Automatic driving vehicle passenger number identification method, system, device and storage medium

Abstract

The application discloses an automatic driving vehicle passenger number identification method and system and a storage medium. In the method, first passenger number information in the vehicle is acquired based on an infrared image, and it is judged whether the first passenger number information is same as second passenger number information in a ticket system. If yes, comparison success information is generated and transmitted to an automatic driving control program. If no, comparison failure information is generated and transmitted to the automatic driving control program. If the comparison success information is received, the automatic driving control program controls the vehicle to open a rear door. If the comparison failure information is received, the automatic driving control program sends unpaid voice prompt information to the vehicle and sends the comparison failure information to a remote control center. The application can supervise and manage the ticket evasion behavior in the operation process and reduce the income loss in the operation process.

Description

Technical Field

[0001] This invention relates to the field of automotive technology, and in particular to a method, system, device, and storage medium for identifying the number of passengers in autonomous vehicles. Background Technology

[0002] With the development of autonomous driving systems, vehicles on some roads can now be controlled by autonomous driving systems, thus replacing human driving operations. This not only reduces the use of human resources but also reduces operating costs. However, in the actual commercial operation of autonomous vehicles, due to the lack of supervision, fare evasion often occurs, which in turn affects their commercial operating revenue. Summary of the Invention

[0003] To address the aforementioned problems, this invention provides a method, system, device, and storage medium for identifying the number of passengers in autonomous vehicles, in order to solve the problem of fare evasion that frequently occurs during the operation of autonomous vehicles in the prior art.

[0004] To achieve the above objectives, the technical solution of the autonomous vehicle passenger number identification method of the present invention includes:

[0005] Step S1: Locate the real-time position of the vehicle and determine whether the vehicle has arrived at the stop. If the vehicle has arrived at the stop, control the vehicle to stop moving and acquire an infrared image inside the vehicle.

[0006] Step S2: Based on the infrared image, obtain the first passenger number information in the vehicle, and determine whether the first passenger number information is the same as the second passenger number information in the ticketing system. If yes, generate a comparison success message and send it to the automatic driving control program; otherwise, generate a comparison failure message and send it to the automatic driving control program.

[0007] Step S3: Determine if there are any passengers who need to get off. If yes, continue to step S4; otherwise, the vehicle restarts and continues moving forward.

[0008] Step S4: If the comparison success information is received, the autonomous driving control program controls the vehicle to open the rear door. If the comparison failure information is received, the autonomous driving control program sends an unpaid fare voice reminder to the vehicle and sends the comparison failure information to the remote control center. After receiving the comparison failure information, the remote control center determines again whether there is an unpaid passenger in the vehicle based on the video playback. If no unpaid passenger is found, the remote control center sends a door opening command to the autonomous driving control program. If an unpaid passenger is found, the remote control center sends a voice reminder to the vehicle to inform the unpaid passenger to pay the fare.

[0009] Furthermore, in step S4, if the passenger who has not paid the fare still has not paid after the voice reminder is sent, the passenger's facial information is recorded.

[0010] Furthermore, in step S1, after the vehicle stops, the current vehicle speed and gear are collected to determine whether the parking requirements are met. If yes, an infrared image inside the vehicle is acquired; otherwise, the vehicle is controlled to stop moving.

[0011] Furthermore, in step S2, obtaining the number of the first passengers inside the vehicle includes the following steps:

[0012] Step S21: Obtain all infrared images in the database, and divide the obtained infrared images into training set and test set according to a certain ratio;

[0013] Step S22: Select the seat area inside the vehicle as the recognition area, extract the recognition area from each infrared image, and normalize the size of the extracted image. Mark images with a seat occlusion rate greater than α as positive samples and images with a seat occlusion rate less than α as negative samples.

[0014] Step S23: Perform color space normalization processing on the recognition region to obtain a normalized image, and extract the orientation gradient histogram features of the normalized image;

[0015] Step S24: Principal component analysis is used to reduce the dimensionality of the histogram of directional gradients features, and the dimensionality-reduced histogram of directional gradients features are input into a Bayesian classifier for classification training;

[0016] Step S25: Input the infrared images actually collected inside the vehicle into the trained Bayesian classifier for classification to obtain the number of the first passengers inside the vehicle.

[0017] Furthermore, in step S3, determining whether a passenger needs to get off the vehicle includes the following steps: obtaining the passenger's line of sight; if the passenger's line of sight is directed toward the rear door of the vehicle, further obtaining the passenger's intention to move; if the passenger's intention to move contains a preset intention, determining that the passenger needs to get off the vehicle.

[0018] Furthermore, in step S4, the autonomous driving control program controlling the vehicle to open the rear door includes the following steps:

[0019] Step S41: Acquire video images of the right side and rear of the vehicle, and determine whether a moving object is approaching based on the video images. If yes, execute step S41; otherwise, control the vehicle to open the rear door.

[0020] Step S42: Obtain the first time t1 from the time the rear door moves to the time it is fully opened, measure the moving speed of the moving object and the straight-line distance from the rear door, calculate the second time t2 from the time the moving object reaches the rear door, and if t1≤t2≤2t1, issue a signal to prohibit opening the rear door.

[0021] Furthermore, in step S4, if a passenger needs to get off the vehicle but does not do so within the third time period after the rear door is opened, the rear door is closed.

[0022] Furthermore, in step S3, there are no passengers in the vehicle, and the autonomous driving control program issues a signal prohibiting the opening of the rear side door.

[0023] This invention also provides an autonomous vehicle passenger number recognition system, which is used to implement the autonomous vehicle passenger number recognition method in the above-mentioned technical solution, including,

[0024] The detection module captures infrared images of the vehicle interior and obtains information about the number of the first passengers.

[0025] The comparison module determines whether the first passenger count information is the same as the second passenger count information in the ticketing system.

[0026] The identification module is used to identify whether a passenger needs to get off the bus.

[0027] The control module controls the opening and closing of the rear door.

[0028] The present invention also provides a storage medium storing the above-described system-executable instructions, which, when executed by a processor included in the autonomous vehicle passenger count recognition system, are used to implement the above-described autonomous vehicle passenger count recognition method.

[0029] Compared with the prior art, the beneficial effects of the present invention are at least as follows:

[0030] 1. After the vehicle stops moving, the system first uses infrared imaging to obtain information on the number of people inside the vehicle, as well as the number of people who paid their fares upon boarding. This information is then compared. If the two numbers are equal, it indicates that every passenger has paid their fare, and the doors can be opened to allow passengers to disembark. If the numbers are inconsistent, it indicates that some passengers have not paid their fares. In this case, a voice message is sent to the passenger compartment to remind them to pay their fare, and a reminder message is sent to the remote control center. The center remotely retrieves the video playback of the relevant time period and verifies which passenger evaded the fare through manual review or facial recognition algorithms. If a specific passenger is identified, they are precisely reminded to pay their parking fee via voice. If no fare evasion is detected, the rear door is opened promptly to prevent other passengers from being unable to disembark.

[0031] 2. When there are many passengers in the vehicle, the varying postures of the people make it difficult to capture clear details in the infrared image, leading to inaccurate passenger count detection. Therefore, the back of the seat is selected as the target area. If someone is in the seat, the back of the seat is obscured, altering the image features and allowing for classification with unoccupied areas. By standardizing the color space of the recognition area, the image contrast can be adjusted, reducing the impact of local shadows and lighting variations, and suppressing noise interference. Attached Figure Description

[0032] Figure 1 This is a flowchart illustrating the method for identifying the number of passengers in an autonomous vehicle according to the present invention.

[0033] Figure 2 This is a schematic diagram illustrating the process of obtaining the number of first passengers in a vehicle according to the present invention. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0035] It is understood that the terms "first," "second," etc., used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are used only to distinguish one element from another. For example, without departing from the scope of this application, a first script may be referred to as a second script, and similarly, a second script may be referred to as a first script.

[0036] like Figure 1 As shown, a method for identifying the number of passengers in an autonomous vehicle includes:

[0037] Step S1: Locate the real-time position of the vehicle, determine whether the vehicle has arrived at the stop, and if the vehicle has arrived at the stop, control the vehicle to stop moving and acquire infrared images inside the vehicle.

[0038] Step S2: Based on the infrared image, obtain the number of the first passengers in the vehicle, and determine whether the number of the first passengers is the same as the number of the second passengers in the ticketing system. If yes, generate a comparison success message and send it to the automatic driving control program; otherwise, generate a comparison failure message and send it to the automatic driving control program.

[0039] Step S3: Determine if there are any passengers who need to get off. If yes, continue to step S4; otherwise, restart the vehicle and continue moving forward.

[0040] Step S4: If a successful comparison message is received, the autonomous driving control program controls the vehicle to open the rear door. If a failed comparison message is received, the autonomous driving control program sends an unpaid fare voice reminder message to the vehicle and sends the failure message to the remote control center. After receiving the failure message, the remote control center determines again whether there is an unpaid passenger in the vehicle based on the video playback. If no unpaid passenger is found, the remote control center sends a door opening command to the autonomous driving control program. If an unpaid passenger is found, the remote control center sends a voice reminder message to the vehicle to inform the unpaid passenger to pay the fare.

[0041] After the vehicle stops moving, the system first uses infrared imaging to obtain information on the number of people inside the vehicle, as well as the number of people who paid their fares upon boarding. This information is then compared. If the two numbers are equal, it indicates that every passenger has paid their fare, and the doors can be opened to allow passengers to disembark. If the numbers are inconsistent, it indicates that some passengers have not paid their fares. In this case, a voice message is sent to the passenger compartment to remind them to pay their fare, and a reminder message is sent to the remote control center. The center remotely retrieves video playback of the relevant time period and verifies which passenger evaded the fare through manual review or facial recognition algorithms. If a specific passenger is identified, they are precisely reminded to pay their parking fee via voice. If no fare evasion is detected, the rear door is opened promptly to prevent other passengers from being unable to disembark.

[0042] In step S4, after sending a voice reminder, if the passenger who has not paid the fare still has not paid, the passenger's facial information is recorded.

[0043] In step S1, after the vehicle stops, the current speed and gear are collected to determine if the parking requirements are met. If yes, an infrared image of the vehicle's interior is acquired; otherwise, the vehicle continues to be controlled to remain stationary. This step prevents the doors from opening before the vehicle has come to a complete stop, thus avoiding passenger injury.

[0044] In step S2, obtaining the number of the first passengers in the vehicle includes the following steps:

[0045] Step S21: Obtain all infrared images in the database, and divide the obtained infrared images into training set and test set according to a certain ratio;

[0046] Step S22: Select the seat area inside the vehicle as the recognition area, crop the recognition area in each infrared image, and normalize the size of the cropped image. Mark images with a seat occlusion rate greater than α as positive samples and images with a seat occlusion rate less than α as negative samples. Specifically, positive samples indicate that there are passengers in the recognition area and negative samples indicate that there are no passengers in the recognition area.

[0047] Step S23: Perform color space normalization on the recognition region to obtain a normalized image, and extract the orientation gradient histogram features of the normalized image;

[0048] Step S24: Principal component analysis is used to reduce the dimensionality of the histogram of directional gradients features, and the dimensionality-reduced histogram of directional gradients features are input into a Bayesian classifier for classification training.

[0049] Step S25: Input the infrared images actually collected inside the vehicle into the trained Bayesian classifier for classification to obtain the number of the first passengers inside the vehicle.

[0050] When there are many passengers in a vehicle, the varying postures of the people make it difficult to capture clear details in the infrared image, leading to inaccurate passenger count detection. Therefore, the seat back is selected as the target area. If someone is in the seat, the seat back is obscured, altering the image characteristics and allowing for classification with unoccupied areas. By standardizing the color space of the recognition area, the image contrast can be adjusted, reducing the impact of local shadows and lighting variations, and suppressing noise interference.

[0051] In step S3, determining whether a passenger needs to get off the vehicle includes the following steps: obtaining the passenger's gaze direction; if the passenger's gaze direction is towards the rear door of the vehicle, further obtaining the passenger's action intention; if the passenger's action intention contains a preset action intention, it is determined that the passenger needs to get off the vehicle. Specifically, the preset action intentions include getting up, turning around, and approaching the door.

[0052] In step S4, the autonomous driving control program controls the vehicle to open the rear side door, including the following steps:

[0053] Step S41: Acquire video images of the right side and rear of the vehicle. Based on the video images, determine whether a moving object is approaching. If yes, execute step S41; otherwise, control the vehicle to open the rear door.

[0054] Step S42: Obtain the first time t1 from the time the rear door moves to the time it is fully opened, measure the moving speed of the moving object and the straight-line distance from the rear door, calculate the second time t2 from the time the moving object reaches the rear door, and if t1≤t2≤2t1, issue a signal to prohibit opening the rear door.

[0055] If the second time interval is less than the first time interval, it indicates that the moving object has already passed the rear door of the vehicle after the door opens, and the passenger can safely disembark at this time. If the second time interval is more than twice the first time interval, it means that the moving object has not yet arrived when the passenger disembarks, and the moving object can observe the passenger disembarking and take evasive action. After recognizing that a passenger needs to disembark, the above method can determine whether the surrounding environment of the vehicle is safe before the passenger disembarks, avoiding collisions between the passenger and moving objects approaching the vehicle, thus preventing passenger injury.

[0056] In step S4, if a passenger needs to disembark but fails to do so within three time intervals after the rear door is opened, the rear door is closed. This step prevents the system from misjudging the situation and causing the vehicle to remain stationary, thus affecting operational efficiency.

[0057] In step S3, if there are no passengers in the vehicle, a signal is issued prohibiting the opening of the rear door. If there are no passengers in the vehicle, there is no need to get off, so closing the rear door at this time can further prevent passengers from boarding through the rear door and thus evading fares.

[0058] The present invention also provides an infrared-based passenger number recognition system for autonomous vehicles, used to implement the above-mentioned autonomous vehicle passenger number recognition method, comprising: a detection module for capturing infrared images inside the vehicle and obtaining first passenger number information therein; a judgment and comparison module for judging whether the first passenger number information is the same as the second passenger number information in the ticketing system; a recognition module for recognizing whether passengers need to get off the vehicle; and a control module for controlling the opening and closing of the rear door.

[0059] This invention also provides an autonomous vehicle passenger number recognition device for implementing the aforementioned autonomous vehicle passenger number recognition method. The device includes a memory for storing a computer program and a processor for executing the computer program to implement the steps of the aforementioned autonomous vehicle passenger number recognition method. Specifically, the autonomous vehicle passenger number recognition device can vary significantly due to different configurations or performance characteristics, and may include one or more processors and memories, and one or more storage media for storing application programs or data. The memory and storage media can be temporary or persistent storage. The program stored in the storage media may include one or more modules, each module including a series of instruction operations on the data processing device.

[0060] The present invention also provides a storage medium storing the above-described system-executable instructions, which, when executed by a processor included in the autonomous vehicle passenger count recognition system, are used to implement the above-described autonomous vehicle passenger count recognition method.

[0061] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0062] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0063] It should be noted that those skilled in the art can make various modifications and improvements without departing from the inventive concept, and these all fall within the scope of protection of this invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

[0064] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for identifying the number of passengers in an autonomous vehicle, characterized in that, include: Step S1: Locate the real-time position of the vehicle and determine whether the vehicle has arrived at the stop. If the vehicle has arrived at the stop, control the vehicle to stop moving and acquire an infrared image inside the vehicle. Step S2: Based on the infrared image, obtain the first passenger number information in the vehicle, and determine whether the first passenger number information is the same as the second passenger number information in the ticketing system. If yes, generate a comparison success message and send it to the automatic driving control program; otherwise, generate a comparison failure message and send it to the automatic driving control program. Step S3: Determine if there are any passengers who need to get off. If yes, continue to step S4; otherwise, the vehicle restarts and continues moving forward. Step S4: If the comparison success information is received, the autonomous driving control program controls the vehicle to open the rear door. If the comparison failure information is received, the autonomous driving control program sends an unpaid fare voice reminder to the vehicle and sends the comparison failure information to the remote control center. After receiving the comparison failure information, the remote control center determines again whether there is an unpaid passenger in the vehicle based on the video playback. If no unpaid passenger is found, the remote control center sends a door opening command to the autonomous driving control program. If an unpaid passenger is found, the remote control center sends a voice reminder to the vehicle to inform the unpaid passenger to pay the fare. In step S2, obtaining the number of the first passengers in the vehicle includes the following steps: Step S21: Obtain all infrared images in the database, and divide the obtained infrared images into training set and test set according to a certain ratio; Step S22: Select the seat area inside the vehicle as the recognition area, extract the recognition area from each infrared image, and normalize the size of the extracted image. Mark images with a seat occlusion rate greater than α as positive samples and images with a seat occlusion rate less than α as negative samples. Step S23: Perform color space normalization processing on the recognition region to obtain a normalized image, and extract the orientation gradient histogram features of the normalized image; Step S24: Principal component analysis is used to reduce the dimensionality of the histogram of directional gradients features, and the dimensionality-reduced histogram of directional gradients features are input into a Bayesian classifier for classification training; Step S25: Input the infrared images actually collected inside the vehicle into the trained Bayesian classifier for classification to obtain the first number of passengers inside the vehicle.

2. The method for identifying the number of passengers in an autonomous vehicle according to claim 1, characterized in that, In step S4, if the passenger who has not paid the fare still has not paid after the voice reminder is sent, the passenger's facial information is recorded.

3. The method for identifying the number of passengers in an autonomous vehicle according to claim 1, characterized in that, In step S1, after the vehicle stops, the current vehicle speed and gear are collected to determine whether the parking requirements are met. If yes, an infrared image inside the vehicle is acquired; otherwise, the vehicle is controlled to stop moving.

4. The method for identifying the number of passengers in an autonomous vehicle according to claim 1, characterized in that, In step S3, determining whether a passenger needs to get off the vehicle includes the following steps: obtaining the passenger's line of sight; if the passenger's line of sight is directed toward the rear door of the vehicle, further obtaining the passenger's intention to move; if the passenger's intention to move contains a preset intention, determining that the passenger needs to get off the vehicle.

5. The method for identifying the number of passengers in an autonomous vehicle according to claim 1, characterized in that, In step S4, the automatic driving control program controls the vehicle to open the rear door, which includes the following steps: Step S41: Acquire video images of the right side and rear of the vehicle, and determine whether a moving object is approaching based on the video images. If yes, execute step S41; otherwise, control the vehicle to open the rear door. Step S42: Obtain the first time t1 from the time the rear door moves to the time it is fully opened, measure the moving speed of the moving object and the straight-line distance from the rear door, calculate the second time t2 from the time the moving object reaches the rear door, and if t1≤t2≤2t1, issue a signal to prohibit opening the rear door.

6. The method for identifying the number of passengers in an autonomous vehicle according to claim 5, characterized in that, In step S4, if a passenger needs to get off the vehicle but does not do so within the third time period after the rear door is opened, the rear door is closed.

7. The method for identifying the number of passengers in an autonomous vehicle according to claim 6, characterized in that, In step S3, there are no passengers in the vehicle, and the automatic driving control program issues a signal prohibiting the opening of the rear side door.

8. A passenger number recognition system for an autonomous vehicle, used to implement the method according to any one of claims 1-7, characterized in that, include: The detection module captures infrared images of the vehicle interior and obtains information about the number of the first passengers. The comparison module determines whether the first passenger count information is the same as the second passenger count information in the ticketing system. The identification module is used to identify whether a passenger needs to get off the bus. The control module controls the opening and closing of the rear door.

9. A storage medium storing executable instructions of the system of claim 8, characterized in that, When executed by a processor included in the system of claim 8, the instructions are used to implement the method for identifying the number of passengers in an autonomous vehicle as described in any one of claims 1-7.

Images