Passenger state determination method and device based on unmanned driving, equipment and medium

Abstract

The invention provides a passenger state determination method and device based on unmanned driving, equipment and a medium, and relates to the field of artificial intelligence, in particular to the field of unmanned driving. The method is applied to an unmanned vehicle, a visual sensor is installed on the vehicle, and the visual sensor is used for collecting image information in the vehicle; the method comprises the steps that when a vehicle reaches a preset terminal point, initial state information of passengers in the vehicle is determined according to vehicle door information and seat information of the vehicle in the driving process; the vehicle door information represents the opening and closing state of the vehicle door, the seat information represents the pressure value of the seat, and the initial state information represents whether the passenger is in the vehicle; acquiring environment monitoring information of the vehicle according to the initial state information; the environment monitoring information comprises vehicle door information, seat information and in-vehicle image information; and according to the environment monitoring information, determining whether the passenger gets off. By combining the information of the vehicle door, the seat and the vision, the passenger getting-off judgment precision is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, and particularly to a method, apparatus, device and medium for determining the state of a passenger based on driverless driving, which can be used in the field of driverless driving. Background Art

[0002] With the development of driverless driving technology, driverless vehicles can be used as online car-hailing vehicles to reach the location of the passenger and take the passenger to the designated location.

[0003] However, without a driver in the vehicle, it is impossible to accurately know the state of the passenger, and there is a situation where the vehicle leaves the designated location before the passenger gets off, which affects the riding experience. Summary of the Invention

[0004] The present disclosure provides a method, apparatus, device and medium for determining the state of a passenger based on driverless driving.

[0005] According to a first aspect of the present disclosure, there is provided a method for determining the state of a passenger based on driverless driving. The method is applied to a driverless vehicle equipped with a vision sensor for collecting in-vehicle image information. The method includes:

[0006] When the vehicle reaches the preset end point of the current ride order, determine the initial state information of the passenger in the vehicle according to the door information of the vehicle during the delivery journey and the seat information of the vehicle during the delivery journey; wherein, the door information represents the open / closed state of the door, the seat information represents the pressure value of the seat, and the initial state information represents whether the passenger is in the vehicle;

[0007] According to the initial state information, obtain the environmental monitoring information of the vehicle within a preset time period; wherein, the environmental monitoring information includes at least one of door information, seat information, and in-vehicle image information;

[0008] According to the environmental monitoring information, determine the target state information of the passenger; wherein, the target state information represents whether the passenger has got off the vehicle.

[0009] According to a second aspect of the present disclosure, there is provided a device for determining the state of a passenger based on driverless driving. The device is applied to a driverless vehicle equipped with a vision sensor for collecting in-vehicle image information. The device includes:

[0010] An initial determination unit, configured to determine initial status information of a passenger in the vehicle based on door information of the vehicle during the trip of delivering a passenger and seat information of the vehicle during the trip of delivering a passenger when the vehicle reaches a preset end point of the current ride order; wherein, the door information represents the open / closed state of the door, the seat information represents the pressure value of the seat, and the initial status information represents whether the passenger is in the vehicle;

[0011] An information acquisition unit, configured to acquire environmental monitoring information of the vehicle within a preset time period according to the initial status information; wherein, the environmental monitoring information includes at least one of door information, seat information, and in-vehicle image information;

[0012] A target determination unit, configured to determine target status information of the passenger according to the environmental monitoring information; wherein, the target status information represents whether the passenger has got off the vehicle.

[0013] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0014] At least one processor; and

[0015] A memory communicatively connected to the at least one processor;

[0016] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in the first aspect of the present disclosure.

[0017] According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, and the computer instructions are used to cause a computer to execute the method described in the first aspect of the present disclosure.

[0018] According to a fifth aspect of the present disclosure, there is provided a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect of the present disclosure are implemented.

[0019] According to a sixth aspect of the present disclosure, there is provided a vehicle, on which a vision sensor is installed, and the vision sensor is used to collect in-vehicle image information, and the vehicle executes the method described in the first aspect of the present disclosure.

[0020] According to the technology of the present disclosure, the accuracy of judging the status of a passenger is improved.

[0021] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. Description of the Drawings

[0022] The accompanying drawings are used to better understand the present solution and do not constitute a limitation to the present disclosure. Among them:

[0023] Figure 1 is a schematic flowchart of a method for determining the passenger status based on driverless driving according to an embodiment of the present disclosure;

[0024] Figure 2 is a schematic flowchart of a method for determining the passenger status based on driverless driving according to an embodiment of the present disclosure;

[0025] Figure 3 is a schematic flowchart of a method for determining the passenger status based on driverless driving according to an embodiment of the present disclosure;

[0026] Figure 4 is a schematic flowchart of a method for determining the passenger status based on driverless driving according to an embodiment of the present disclosure;

[0027] Figure 5 is a logic diagram of an order end process according to an embodiment of the present disclosure;

[0028] Figure 6 is a structural block diagram of a device for determining the passenger status based on driverless driving according to an embodiment of the present disclosure;

[0029] Figure 7 is a structural block diagram of a device for determining the passenger status based on driverless driving according to an embodiment of the present disclosure;

[0030] Figure 8 is a block diagram of an electronic device for implementing the method for determining the passenger status based on driverless driving according to an embodiment of the present disclosure;

[0031] Figure 9 is a block diagram of an electronic device for implementing the method for determining the passenger status based on driverless driving according to an embodiment of the present disclosure. Detailed implementation manners

[0032] The following makes an explanation of the exemplary embodiments of the present disclosure in conjunction with the accompanying drawings. Various details of the embodiments of the present disclosure are included to help understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted below.

[0033] For driverless vehicles, when determining whether a passenger has gotten off, the determination method usually relies on a single sensor, such as whether the car door has been opened or closed, or whether there is seat pressure, etc. This determination method is more vulnerable to various factors and may result in misjudgment. For example, a passenger has opened or closed the car door but has not actually gotten off, or a passenger moves inside the vehicle, etc., which may cause the vehicle to think that the passenger has completed getting off and continue to perform subsequent actions such as driving or receiving new orders, thus bringing a bad experience to the passenger and affecting the normal operation of the vehicle.

[0034] Specifically, the method of using a seat pressure sensor combined with the detection result of the opening and closing of the car door to measure whether there is a passenger on the seat is affected by factors such as the sitting posture change of the passenger and the reliability of the sensor, and there may be a possibility of false alarm. For example, if the seat pressure sensor does not detect pressure and the passenger has performed the action of opening or closing the car door, then the vehicle will misjudge that the passenger has gotten off; if a passenger leaves a heavy object on the seat but has actually gotten off, the vehicle will misjudge that the passenger has not gotten off; if a passenger is too light, such as a child, the pressure sensor may not be triggered. If the parent performs the operation of opening or closing the car door, the vehicle will misjudge that the passenger has gotten off; if the car door sensor fails, it may also lead to an incorrect judgment of the passenger's state, etc.

[0035] The present disclosure provides a method, device, equipment and medium for determining the passenger state based on driverless driving, which is applied to the field of driverless driving in the field of artificial intelligence to improve the accuracy of determining the passenger state.

[0036] It should be noted that the data in this embodiment is not targeted at a specific user and does not reflect the personal information of a specific user. It should be noted that the data in this embodiment comes from a public data set.

[0037] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of the user's personal information and other processes all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0038] To enable readers to more deeply understand the implementation principle of the present disclosure, the following Figures 1-9 is used to further refine the embodiments.

[0039] Figure 1 As a flowchart of a method for determining the passenger state based on driverless driving provided by an embodiment of the present disclosure, this method can be executed by a device for determining the passenger state based on driverless driving. This method is applied to a driverless vehicle, and a vision sensor is installed on the vehicle, and the vision sensor is used to collect image information inside the vehicle. As Figure 1 shown, this method includes the following steps:

[0040] S101. When the vehicle reaches the preset destination of the current ride order, determine the initial status information of the passengers in the vehicle according to the door information of the vehicle during the ride and the seat information of the vehicle during the ride. The door information represents the open / closed state of the door, the seat information represents the pressure value of the seat, and the initial status information represents whether the passengers are in the vehicle.

[0041] Exemplarily, the vehicle in this embodiment is a driverless vehicle, which can receive the ride order of passengers as an online car-hailing service, automatically go to the starting point in the ride order, pick up the passengers and then automatically go to the destination in the ride order to complete the ride order.

[0042] One or more visual sensors can be installed in the vehicle to collect the image information inside the vehicle. For example, one camera can be installed above the front row or rear row seats respectively to collect the image information at the seats. The position and quantity of the visual sensors can be deployed according to the actual vehicle model. For example, for vehicle model A where only passengers are allowed to sit in the rear row, the visual sensor can be installed only above the rear row seats; for vehicle model B where passengers are allowed to sit in the front row and the rear row, visual sensors can be installed above the front row and rear row seats respectively. A door switch sensor can also be installed on the vehicle to detect the open / closed state of the door. A seat pressure sensor can also be installed to detect the pressure value of the seat.

[0043] After picking up the passengers, the vehicle continues to execute the current ride order, and the preset destination set by the passengers is included in the current ride order. The vehicle continuously detects whether it has reached the preset destination. During the driving process, that is, during the ride, the door information and the seat information can also be obtained in real time. The door information represents the open / closed state of the door, that is, it can be determined whether the door has been opened during the ride. The seat information represents the pressure value of the seat, that is, it can be determined whether the pressure borne by the seat has changed during the ride.

[0044] If the vehicle reaches the preset destination of the current ride order, the door information and the seat information collected during the ride can be obtained, and the initial status information of the passengers in the vehicle can be determined according to the door information and / or the seat information of the vehicle during the ride. The initial status information is the preliminary judgment result of whether the passengers are in the vehicle. For example, if the door has been opened during the ride, it is considered that the passengers may no longer be in the vehicle; or if the pressure value on the seat is 0, it is considered that the passengers may no longer be in the vehicle.

[0045] After the vehicle arrives at the preset end point of the current ride order, it can first determine whether the vehicle door is opened within a preset time interval through the door switch sensor. For example, it can be determined whether the vehicle door is opened within 5 seconds. If it is not opened, the initial status information of the passengers in the vehicle is determined based on the door information and the seat information of the vehicle during the ride. That is, it is inferred whether the passengers are still in the vehicle. If the vehicle door is opened within the preset time interval, since it is a driverless vehicle, it means that the passengers are definitely in the vehicle, that is, the initial status information is that the passengers are in the vehicle, and S102 and S103 can be continued to be executed.

[0046] In this embodiment, a display screen is installed on the vehicle, and the display screen is located at the front row seat and / or the rear row seat; the method further includes: after the vehicle arrives at the preset starting point of the current ride order, in response to the confirmation driving instruction issued by the passenger through the display screen, the vehicle travels to the preset end point based on the preset driving trajectory in the current ride order, and the door information of the vehicle during the ride and the seat information of the vehicle during the ride are detected in real time.

[0047] Specifically, a display screen is installed on the vehicle, and the progress of the current ride order can be displayed on the display screen. The display screen can be installed at the front row seat and / or the rear row seat of the vehicle according to different vehicle models. For example, it can be installed on the back of the front row seat for the convenience of the passengers in the rear row seats to view.

[0048] When the vehicle receives the current ride order, it can obtain the preset starting point designated by the passenger from the current ride order. The vehicle arrives at the preset starting point and waits for the passengers to get on the vehicle. After the passengers get on the vehicle, a confirmation driving instruction can be issued through the display screen. For example, the "Drive" control on the display screen can be clicked. When the vehicle receives the confirmation driving instruction and confirms that the passengers have got on the vehicle, it can set off for the preset end point. The current ride order may contain the driving trajectory from the preset starting point to the preset end point. In this embodiment, the planning algorithm for the driving trajectory is not specifically limited. The vehicle travels to the preset end point according to the preset driving trajectory and detects the door information and the seat information in real time during the ride until the current ride order ends.

[0049] The vehicle can also detect in real time whether there are operations of opening and closing the vehicle door, whether there is pressure on the seat, or detect whether there are passengers in the vehicle through a vision sensor, so as to determine whether the passengers have got on the vehicle. For example, after the vehicle arrives at the preset starting point, if there are operations of opening and closing the vehicle door, there is pressure greater than 0 on the seat, and there are people in the vehicle detected through the vision sensor, it is considered that the passengers have got on the vehicle.

[0050] The beneficial effect of such a setting is that when the vehicle arrives at the preset starting point to pick up the passengers, it determines whether the passengers have got on the vehicle by responding to the confirmation driving instruction issued by the passengers. It avoids the vehicle starting before the passengers have got on the vehicle, and improves the riding experience of the passengers.

[0051] S102. Obtain the environmental monitoring information of the vehicle within a preset time period according to the initial state information; wherein, the environmental monitoring information includes at least one of door information, seat information, and in-vehicle image information.

[0052] Exemplarily, after obtaining the initial state information, the environmental monitoring information of the vehicle within a preset time period can be obtained according to the initial state information. The environmental monitoring information is the information obtained by monitoring the in-vehicle environment and different devices in the vehicle. The devices in the vehicle can be doors, seats, etc. The environmental monitoring information can include at least one of door information, seat information, and in-vehicle image information, that is, at least one of the opening and closing state of the door, the pressure value of the seat, and the image collected by the vision sensor can be obtained. The in-vehicle image information collected by the vision sensor can identify whether there is anyone in the vehicle.

[0053] Different initial state information can correspond to different types of environmental monitoring information. For example, if the initial state information is that the passenger is not in the vehicle, the obtained environmental monitoring information can include these three types: door information, seat information, and in-vehicle image information; if the initial state information is that the passenger is in the vehicle, the obtained environmental monitoring information can include these two types: door information and seat information. The association relationship between different initial state information and the types of environmental monitoring information can be preset. According to the preset association relationship, the environmental monitoring information can be obtained targeted, avoiding redundant information acquisition and affecting the determination efficiency of the passenger state.

[0054] S103. Determine the target state information of the passenger according to the environmental monitoring information; wherein, the target state information indicates whether the passenger has got off the vehicle.

[0055] Exemplarily, according to the obtained environmental monitoring information, the target state of the passenger can be determined, that is, it can be determined whether the passenger has got off the vehicle. For example, the environmental monitoring information includes door information, seat information, and in-vehicle image information. If the door information indicates that the door is in the closed state, the seat information indicates that the pressure value on the seat is 0, and the in-vehicle image information indicates that there is no one in the vehicle, it is considered that the passenger has got off the vehicle.

[0056] If it is determined that the passenger has got off the vehicle, the current ride order can be ended, and the next ride order can be continued to pick up new passengers at a new starting point; if it is determined that the passenger has not got off the vehicle, the passenger can be waited for to get off, or a reminder message can be sent to the passenger in the form of playing voice or displaying text on the display screen, etc., to remind the passenger to get off as soon as possible.

[0057] In the embodiments of the present disclosure, the opening and closing state of the vehicle door and the pressure value of the seat can be detected in real time during the driving process. After the vehicle reaches the preset end point, first, based on the opening and closing state of the vehicle door and the seat pressure during the driving process, a preliminary judgment is made on whether the passenger is still in the vehicle. Then, according to the preliminary judgment result, multiple environmental monitoring information such as vehicle door information, seat information, and visual information is obtained. By combining multiple environmental monitoring information, it is determined whether the user has got off the vehicle. Through the fusion of the vehicle door switch sensor, seat pressure sensor, and visual sensor, the accuracy of judging the passenger's state can be improved, the misjudgment of the passenger getting off the vehicle can be avoided, and the riding experience and vehicle operation efficiency can be enhanced.

[0058] Figure 2 FIG. 4 is a schematic flowchart of a method for determining a passenger state based on driverless driving provided by an embodiment of the present disclosure. This embodiment is an optional embodiment based on the above embodiment.

[0059] In this embodiment, based on the vehicle door information during the driving process of the vehicle and the seat information during the driving process of the vehicle, the initial state information of the passengers in the vehicle is determined, including: determining a target time period during the driving process according to the vehicle door information during the driving process of the vehicle; wherein the time length of the target time period is less than or equal to the time length during the driving process; extracting the seat information of the target time period from the seat information during the driving process of the vehicle; and determining the initial state information of the passengers in the vehicle according to the seat information of the target time period.

[0060] As Figure 2 shown, the method includes the following steps:

[0061] S201. When the vehicle reaches the preset end point of the current ride order, determine a target time period during the driving process according to the vehicle door information during the driving process of the vehicle; wherein the time length of the target time period is less than or equal to the time length during the driving process.

[0062] Exemplarily, the vehicle collects vehicle door information and seat information in real time during the driving process. When the vehicle reaches the preset end point, the vehicle door information during the driving process is obtained. According to the vehicle door information during the driving process, a target time period is determined, and the time length of the target time period is less than or equal to the time length during the driving process, that is, the target time period is a time period during the driving process. For example, the target time period is the entire time period during the driving process, or the target time period is a middle period of time during the driving process.

[0063] According to the vehicle door information during the driving process, it is determined whether the vehicle door has been opened during the driving process. According to the opening and closing state of the vehicle door during the driving process, the target time period is determined. For example, if the vehicle door has not been opened during the driving process, the entire time period during the driving process is determined as the target time period; if the vehicle door has been opened during the driving process, the latter two-thirds of the time period during the driving process can be determined as the target time period.

[0064] In this embodiment, determining the target time period during the vehicle delivery trip based on the door information of the vehicle during the delivery trip includes: if it is determined according to the door information of the vehicle during the delivery trip that the vehicle has not performed any door-opening operations during the delivery trip, then the time period corresponding to the delivery trip is determined as the target time period.

[0065] Specifically, according to the door information of the vehicle during the delivery trip, it is determined whether the vehicle has performed any door-opening operations during the delivery trip. If no door-opening operations have been performed, it means that regardless of whether there are passengers in the vehicle, the passengers in the vehicle have not changed during the delivery trip. The time period corresponding to the entire delivery trip can be determined as the target time period.

[0066] The beneficial effect of such a setting is that if the door has not been opened during the journey, the entire delivery trip is determined as the target time period, so as to determine the initial state information of the passengers through the seat information during the entire delivery trip, avoid information omission, and improve the determination accuracy of the initial state information.

[0067] In this embodiment, determining the target time period during the vehicle delivery trip based on the door information of the vehicle during the delivery trip includes: if it is determined according to the door information of the vehicle during the delivery trip that the vehicle has performed a door-opening operation during the delivery trip, then determine the execution time corresponding to the door-opening operation; according to the execution time, determine the target time period from the time period corresponding to the delivery trip; where the target time period represents the time period from the execution time to the moment of reaching the preset end point.

[0068] Specifically, according to the door information of the vehicle during the delivery trip, it is determined whether the vehicle has performed a door-opening operation during the delivery trip. If a door-opening operation has been performed, then determine the time when the door-opening operation is performed, that is, determine the execution time. Starting from this execution time until the vehicle reaches the preset end point, that is, the end moment of the delivery trip, the time period between them is determined as the target time period. That is, the target time period represents the time period from the execution time to the moment of reaching the preset end point.

[0069] After determining that a door-opening operation has been performed during the delivery trip, the number of door-opening operations during the delivery trip can also be determined. If there is only one door-opening operation, then determine the execution time corresponding to the door-opening operation, and according to this execution time, determine the target time period from the time period corresponding to the delivery trip.

[0070] If there are multiple door-opening operations, then the execution time of the last door-opening operation can be determined, and according to the execution time of the last time, determine the target time period from the time period corresponding to the delivery trip. That is, the time period from the execution time of the last time to the moment of reaching the preset end point is determined as the target time period. In this embodiment, the door-opening operation during the delivery trip is the door-opening operation performed before reaching the preset end point.

[0071] The beneficial effect of such a setting is that if the door is opened during the journey, it indicates that the passenger may get off after the door is opened. The time period between the moment when the door is opened midway and the moment when the vehicle reaches the destination is determined as the target time period, reducing information redundancy, avoiding the interference of the seat information before the door is opened on the state judgment, and improving the determination efficiency and accuracy of the initial state information.

[0072] S202. Extract the seat information of the target time period from the seat information of the vehicle during the driving process for delivering the driver.

[0073] Exemplarily, the seat information of the entire driving process for delivering the driver of the vehicle can be collected. After determining the target time period, only the seat information collected during the target time period can be obtained. That is, the seat information of the target time period can be extracted from the seat information of the entire driving process for delivering the driver. For example, the time period of the entire driving process for delivering the driver is from 11:00 to 11:30, and the target time period is from 11:15 to 11:30, then the seat information collected from 11:15 to 11:30 can be extracted.

[0074] S203. Determine the initial state information of the passengers in the vehicle according to the seat information of the target time period.

[0075] Exemplarily, after obtaining the seat information of the target time period, according to the pressure change situation of the seat during the target time period, the initial state information of the passengers is determined, that is, it is determined whether the passengers are still in the vehicle. For example, if the pressure value of the seat remains at a certain large value during the target time period, it is considered that the passengers are still in the vehicle; if the pressure value of the seat suddenly becomes smaller during the target time period, it is considered that the passengers are not in the vehicle. In this embodiment, the seat information of each seat in the vehicle can be monitored to avoid misjudgment caused by passengers changing positions in the vehicle. When determining the initial state information, the pressure values of all seats can be comprehensively judged. For example, if the pressure values of all seats are 0 during the target time period, it is considered that the passengers are not in the vehicle; if there is a large pressure value for one seat during the target time period, it is considered that the passengers are still in the vehicle.

[0076] In this embodiment, according to the opening and closing situation of the door during the driving process for delivering the driver, a part of the information is selected from the seat information of the entire driving process for delivering the driver for judging the initial state information, improving the judgment accuracy of the initial state information, and further improving the judgment accuracy of the target state information.

[0077] In this embodiment, determining the initial state information of the passengers in the vehicle according to the seat information of the target time period includes: if there is a pressure value greater than the preset pressure threshold in the seat information of the target time period, determining that the initial state information of the passengers in the vehicle is that the passengers are in the vehicle; if there is no pressure value greater than the preset pressure threshold in the seat information of the target time period, determining that the initial state information of the passengers in the vehicle is that the passengers are not in the vehicle.

[0078] Specifically, a pressure threshold is preset. For example, the pressure threshold is 0. The pressure values of each seat at each moment are included in the target time period. The pressure values in the target time period are respectively compared with the preset pressure threshold. If there is a pressure value greater than the pressure threshold in the seat information of the target time period, that is, there is a pressure value greater than 0 in the target time period, it is determined that there is an object on the seat in the vehicle, which may be the passenger himself or the passenger's belongings, etc. It can be considered that the initial state information of the passenger is that the passenger is in the vehicle.

[0079] If there is no pressure value greater than the preset pressure threshold in the seat information of the target time period, that is, the pressure values of each seat in the target time period are always 0, it is determined that the initial state information of the passenger in the vehicle is that the passenger is not in the vehicle.

[0080] The beneficial effect of such a setting is that in the target time period, if there has ever been pressure on the seat, it is considered that the passenger is still in the vehicle. For example, the passenger was originally sitting on the seat and then left the seat but is still in the vehicle; if there is no pressure on the seat all the time, it is considered that the passenger is not in the vehicle. The accuracy of determining the initial state information is improved, and the possibility of misjudgment is reduced.

[0081] S204. Obtain the environmental monitoring information of the vehicle in the preset time period according to the initial state information; wherein, the environmental monitoring information includes at least one of door information, seat information, and in-vehicle image information.

[0082] Exemplarily, this step can refer to the above step S102 and will not be elaborated here.

[0083] S205. Determine the target state information of the passenger according to the environmental monitoring information; wherein, the target state information indicates whether the passenger has got off the vehicle.

[0084] Exemplarily, this step can refer to the above step S103 and will not be elaborated here.

[0085] In the embodiments of the present disclosure, the opening and closing state of the door and the pressure value of the seat can be detected in real time during the driving process. When the vehicle reaches the preset end point, first, according to the opening and closing state of the door and the seat pressure during the driving process, it is preliminarily judged whether the passenger is still in the vehicle. Then, according to the preliminary judgment result, various environmental monitoring information such as door information, seat information, and visual information is obtained, and combined with various environmental monitoring information, it is determined whether the user has got off the vehicle. By fusing the door switch sensor, seat pressure sensor, and visual sensor, the judgment accuracy of the passenger state can be improved, the misjudgment of the passenger getting off the vehicle can be avoided, and the riding experience and vehicle operation efficiency can be improved.

[0086] Figure 3Schematic flowchart of a method for determining the passenger status based on driverless driving provided by an embodiment of the present disclosure. This embodiment is an optional embodiment based on the above-mentioned embodiment.

[0087] In this embodiment, according to the initial status information, environmental monitoring information of the vehicle within a preset time period is obtained, including: if the initial status information indicates that the passenger is not in the vehicle, based on a preset information acquisition cycle, door information, seat information, and in-vehicle image information of the vehicle within the preset time period are obtained.

[0088] As Figure 3 shown, the method includes the following steps:

[0089] S301. When the vehicle reaches the preset end point of the current ride order, determine the initial status information of the passenger in the vehicle according to the door information of the vehicle during the delivery journey and the seat information of the vehicle during the delivery journey; wherein, the door information represents the open / closed state of the door, the seat information represents the pressure value of the seat, and the initial status information represents whether the passenger is in the vehicle.

[0090] Exemplarily, this step can refer to the above-mentioned step S101 and will not be elaborated herein.

[0091] S302. If the initial status information indicates that the passenger is not in the vehicle, based on a preset information acquisition cycle, obtain the door information, seat information, and in-vehicle image information of the vehicle within the preset time period.

[0092] Exemplarily, the initial status information represents that the passenger is still in the vehicle or the passenger is not in the vehicle. If it is considered that the passenger is not in the vehicle, the door information, seat information, and in-vehicle image information of the vehicle can be obtained in real time or at regular intervals. That is, for the situation where the passenger may not be in the vehicle, the environmental monitoring information includes the door information, seat information, and in-vehicle image information.

[0093] The information acquisition cycle can be set in advance. For example, the information acquisition cycle can be 1 second to continuously obtain the environmental monitoring information. According to the preset information acquisition cycle, obtain the door information, seat information, and in-vehicle image information of the vehicle within the preset time period. For example, the preset time period can be 5 minutes, that is, continuously obtain the door information, seat information, and in-vehicle image information within 5 minutes to determine whether the passenger has got off the vehicle, so as to avoid the vehicle waiting and judging for a long time, which affects the subsequent vehicle operation.

[0094] The door information can be collected by a door switch sensor, the seat information can be collected by a seat pressure sensor, and the in-vehicle environmental information can be collected by a vision sensor. Among them, seat pressure sensors are installed at each seat to obtain the seat information of each seat.

[0095] In this embodiment, if it is considered that the passenger is not in the vehicle, door information, seat information, and in-vehicle image information are obtained for further judgment, so as to avoid misjudgment of the initial state information and improve the accuracy of confirming the passenger's getting off.

[0096] S303. Determine the target state information of the passenger according to the environmental monitoring information; wherein, the target state information represents whether the passenger has got off.

[0097] Exemplarily, according to the obtained environmental monitoring information, that is, by combining the door information, seat information, and in-vehicle image information, the target state information of the passenger can be determined, that is, it can be determined whether the passenger has finally got off. For example, if it is confirmed that the door is in the closed state, there is no pressure on the seat, and the vision sensor confirms that there is no one in the vehicle, it is determined that the passenger has got off.

[0098] In this embodiment, determining the target state information of the passenger according to the environmental monitoring information includes: determining a visual judgment result according to the in-vehicle image information; wherein, the visual judgment result represents whether there is someone in the vehicle determined by the in-vehicle image; if the door information within a preset time period represents that the door is in the closed state, there is no pressure value greater than a preset pressure threshold in the seat information within the preset time period, and the visual judgment result represents that there is no one in the vehicle, it is determined that the target state information of the passenger is that the passenger has got off.

[0099] Specifically, based on a preset image recognition algorithm, the in-vehicle image information is recognized and processed to determine whether there are people in the in-vehicle image information. For example, the heads in the image can be recognized, the number of heads in the in-vehicle image information can be determined, and the number of heads is determined as the visual judgment result. If the number of heads is 0, it means that the vision sensor determines that there is no one in the vehicle; if the number of heads is greater than 0, it means that the vision sensor determines that there is someone in the vehicle.

[0100] In this embodiment, the preset image recognition algorithm and the head recognition logic are not specifically limited. Different recognition logics can be preset and associated with the opening and closing state of the door, that is, different recognition logics can be selected according to the opening and closing state of the door. For example, when a door is open, the number of heads in the front row, rear row, and trunk can be detected. It is also possible to detect the number of heads when the passenger gets on the vehicle. For example, if it is recognized that the number of heads in the front row is greater than 1 or the number of heads in the rear row is greater than 2, it is considered that the vehicle is overloaded, and a reminder can be sent to the passenger. Different reminders can be sent according to different load states of the vehicle, and the load states of the vehicle can include no load, having passengers and the passenger situation is normal, overloaded, having passengers but not compliant, etc.

[0101] Combining the door information, seat information, and visual judgment result, determine the target status information of the passenger. If the door information indicates that the door is in the closed state, there is no pressure value greater than the preset pressure threshold in the seat information (i.e., the seat pressure value is 0), and the visual judgment result indicates that there is no one in the vehicle, then determine that the target status information of the passenger is that the passenger has got off the vehicle; if the door information indicates that the door is in the open state, there is a pressure value greater than the preset pressure threshold in the seat information, or the visual judgment result indicates that there is someone in the vehicle, then determine that the target status information of the passenger is that the passenger has not got off the vehicle.

[0102] The beneficial effect of such a setting is that by performing image recognition to determine whether there is someone in the vehicle, and combining the door switch state, seat pressure, and image recognition result to determine whether the passenger has got off the vehicle, interference and misjudgment are reduced, and the judgment accuracy of the passenger getting off the vehicle is improved.

[0103] In the embodiments of the present disclosure, the opening and closing state of the door and the pressure value of the seat can be detected in real time during the driving process. When the vehicle reaches the preset end point, first, based on the opening and closing state of the door and the seat pressure during the driving process, preliminarily determine whether the passenger is still in the vehicle. Then, according to the preliminary judgment result, obtain various environmental monitoring information such as door information, seat information, and visual information, and combine various environmental monitoring information to determine whether the user has got off the vehicle. By fusing the door switch sensor, seat pressure sensor, and visual sensor, the judgment accuracy of the passenger status can be improved, the misjudgment of the passenger getting off the vehicle can be avoided, and the riding experience and vehicle operation efficiency can be enhanced.

[0104] Figure 4 It is a schematic flowchart of a method for determining the passenger status based on driverless driving provided by the embodiments of the present disclosure, and this embodiment is an optional embodiment based on the above embodiments.

[0105] In this embodiment, according to the initial state information, obtain the environmental monitoring information of the vehicle within a preset time period, including: if the initial state information is that the passenger is in the vehicle, then detect the current door information of the vehicle in real time; if the current door information of the vehicle indicates that the door is in the open state, then obtain the order type of the current ride order; according to the order type of the current ride order, obtain the environmental monitoring information of the vehicle within a preset time period.

[0106] As Figure 4 shown, the method includes the following steps:

[0107] S401. When the vehicle reaches the preset end point of the current ride order, determine the initial state information of the passenger in the vehicle according to the door information of the vehicle during the driving process and the seat information of the vehicle during the driving process; wherein, the door information indicates the opening and closing state of the door, the seat information indicates the pressure value of the seat, and the initial state information indicates whether the passenger is in the vehicle.

[0108] Exemplarily, this step can refer to the above step S101 and will not be elaborated herein.

[0109] S402. If the initial status information indicates that the passenger is in the vehicle, then the current door information of the vehicle is detected in real time.

[0110] Exemplarily, if the initial status information indicates that the passenger is still in the vehicle, then the current door information of the vehicle is obtained in real time to determine whether the door is opened. If the door has not been opened yet, then the current door information continues to be obtained until the door is opened. The door information can be obtained within a preset time period. For example, the real-time door information is obtained within 10 minutes. If the door has not been opened after 10 minutes, then a voice can be broadcast in the vehicle or a pop-up window can be issued on the in-vehicle display screen to remind the passenger to open the door as soon as possible. It can also be directly considered that the passenger has already got off the vehicle, and the vehicle can leave the preset end point of the current ride order and go to execute the next order.

[0111] S403. If the current door information of the vehicle indicates that the door is in an open state, then the order type of the current ride order is obtained.

[0112] Exemplarily, if it is determined according to the real-time door information that the door is currently opened, then the order type of the current ride order can be obtained. In this embodiment, the order type can indicate the relationship between the current ride order and the next ride order. For example, the order type can include an extreme consecutive order and a non-extreme consecutive order. An extreme consecutive order means that the end point of the current ride order is the start point of the next ride order, that is, after the vehicle arrives at the preset end point of the current ride order, it directly proceeds to the next ride order. Non-extreme consecutive orders can include empty driving orders, single orders, ordinary consecutive orders, etc. A consecutive order refers to two consecutive orders, and the second order has been pre-assigned before the first order ends. An extreme consecutive order specifically refers to the first order in such consecutive orders, and the end point of the first order is the start point of the second order.

[0113] When the vehicle arrives at the preset end point of the current ride order or before arriving at the preset end point, it can know the order type of the current ride order. For example, after the vehicle arrives at the preset end point of the current ride order and has not received the next ride order, then the current ride order is a non-extreme consecutive order. If the vehicle receives a new ride order before arriving at the preset end point of the current ride order, and the preset start point of the new ride order is the preset end point of the current ride order, then the current ride order is an extreme consecutive order.

[0114] S404. According to the order type of the current ride order, the environmental monitoring information of the vehicle within a preset time period is obtained.

[0115] Exemplarily, for different order types, different kinds of environmental monitoring information can be obtained. For example, the association relationship between the order type and the category of environmental monitoring information can be preset, and according to this association relationship, the environmental monitoring information can be obtained specifically. For example, if the order type is an extreme consecutive order, the environmental monitoring information may only include the category of seat information.

[0116] In this embodiment, if it is determined that the passenger is in the vehicle, it is detected whether the door is opened. If the door is opened, the order type is determined, and different monitoring information is obtained corresponding to different order types, so as to realize the judgment of different logics for different order types and improve the determination accuracy of the passenger state.

[0117] In this embodiment, according to the order type of the current ride order, the environmental monitoring information of the vehicle within a preset time period is obtained, including: if the order type is the first type, based on the preset information acquisition period, the seat information of the vehicle within the preset time period is obtained.

[0118] Specifically, the first type may refer to an extreme consecutive order, that is, if the order type is an extreme consecutive order, only the seat information can be obtained. The seat information within the preset time period can be obtained based on the preset information acquisition period.

[0119] That is to say, for an extreme consecutive order, the environmental monitoring information may only include the seat information, and it is not necessary to fuse the visual sensor with the seat pressure and the door state.

[0120] The beneficial effect of such a setting is that the seat information is obtained for the extreme consecutive order, avoiding information redundancy and improving the determination accuracy of the target state information.

[0121] In this embodiment, according to the order type of the current ride order, the environmental monitoring information of the vehicle within a preset time period is obtained, including: if the order type is the second type, based on the preset information acquisition period, the door information, seat information, and in-vehicle image information of the vehicle within the preset time period are obtained.

[0122] Specifically, the second type may refer to a non-extreme consecutive order, that is, if the order type is a non-extreme consecutive order, only the three kinds of information of the door information, seat information, and in-vehicle image information can be obtained. The door information, seat information, and in-vehicle image information within the preset time period can be obtained based on the preset information acquisition period.

[0123] That is to say, for an extreme consecutive order, the environmental monitoring information may include the door information, seat information, and in-vehicle image information, and it is necessary to fuse the visual sensor with the seat pressure and the door state.

[0124] The beneficial effect of such a setting is that it obtains three types of information for the extreme consecutive single orders, avoids information omission and misjudgment, and improves the determination accuracy of the target status information.

[0125] S405. Determine the target status information of the passenger according to the environmental monitoring information; wherein, the target status information indicates whether the passenger has got off the vehicle.

[0126] Exemplarily, according to the obtained environmental monitoring information, the target status information of the passenger can be determined, that is, it is determined whether the passenger gets off the vehicle. If it is determined that the passenger gets off the vehicle, the current ride order can be ended, and the next ride order can be continued to pick up a new passenger at a new starting point; if it is determined that the passenger does not get off the vehicle, the passenger can be waited for to get off the vehicle, or a prompt message can be sent to the passenger in the form of playing voice or displaying text on the display screen to prompt the passenger to get off the vehicle as soon as possible.

[0127] In this embodiment, determining the target status information of the passenger according to the environmental monitoring information includes: if it is determined that there is no pressure value greater than the preset pressure threshold in the seat information within the preset time period, determining that the target status information of the passenger is that the passenger has got off the vehicle.

[0128] Specifically, if the order type is the first type, after obtaining the environmental monitoring information, that is, the seat information, it is determined whether the passenger gets off the vehicle according to the seat information.

[0129] A pressure threshold is set in advance. For example, the pressure threshold is 0. There can be multiple pressure values within the preset time period. If there is no pressure value greater than the preset pressure threshold within the preset time period, that is, the pressure values within the preset time period are all 0, it is determined that the target status information of the passenger is that the passenger has got off the vehicle. If there is a pressure value greater than the preset pressure threshold within the preset time period, it is determined that the target status information of the passenger is that the passenger has not got off the vehicle yet. For example, the preset time period is 5 seconds. If the seat pressure remains 0 for 5 seconds, it is considered that the passenger has got off the vehicle.

[0130] The beneficial effect of such a setting is that for the extreme consecutive single orders, the seat pressure is detected. If the detected seat pressure is empty, it is determined that the passenger gets off the vehicle and the current order is ended, improving the operation efficiency of the driverless vehicle.

[0131] In this embodiment, determining the target status information of the passenger according to the environmental monitoring information includes: determining a visual determination result according to the in-vehicle image information; wherein, the visual determination result indicates whether there is anyone in the vehicle determined through the in-vehicle image; if the door information within the preset time period indicates that the door is in the closed state, there is no pressure value greater than the preset pressure threshold in the seat information within the preset time period, and the visual determination result indicates that there is no one in the vehicle, it is determined that the target status information of the passenger is that the passenger has got off the vehicle.

[0132] Specifically, if the order type is the second type, after obtaining the environmental monitoring information, namely the door information, seat information, and in-vehicle image information, the system determines whether the passenger has alighted by combining the door information, seat information, and in-vehicle image information.

[0133] The judgment process of the target status information corresponding to the second type is the same as that after initially determining that the passenger is not in the vehicle. Both are to determine the target status information of the passenger by combining the door information, seat information, and visual judgment result. If the door information indicates that the door is in the closed state, there is no pressure value greater than the preset pressure threshold in the seat information (i.e., the seat pressure values are all 0), and the visual judgment result indicates that there is no one in the vehicle, then it is determined that the target status information of the passenger is that the passenger has alighted; if the door information indicates that the door is in the open state, there is a pressure value greater than the preset pressure threshold in the seat information, or the visual judgment result indicates that there is someone in the vehicle, then it is determined that the target status information of the passenger is that the passenger has not alighted yet.

[0134] Based on a preset image recognition algorithm, the in-vehicle image information can be recognized and processed to determine whether there are people in the in-vehicle image information. For example, the heads in the image can be recognized to determine the number of heads in the in-vehicle image information, and the number of heads is determined as the visual judgment result. If the number of heads is 0, it means that the visual sensor determines that there is no one in the vehicle; if the number of heads is greater than 0, it means that the visual sensor determines that there is someone in the vehicle.

[0135] The beneficial effect of this setting is that for non-extreme consecutive orders, if it is confirmed that the door is closed, there is no pressure on the seat, and the visual sensor confirms that there is no one, then it is confirmed that the passenger has alighted, ending the current order, realizing the combination of the three types of information, improving the judgment accuracy of the passenger's alighting, and enhancing the passenger's riding experience.

[0136] In the embodiments of the present disclosure, the opening and closing state of the door and the pressure value of the seat can be detected in real time during the driver delivery process. When the vehicle reaches the preset destination, first, based on the opening and closing state of the door and the seat pressure during the driver delivery process, it is preliminarily determined whether the passenger is still in the vehicle. Then, according to the preliminary judgment result, various environmental monitoring information such as door information, seat information, and visual information is obtained, and by combining the various environmental monitoring information, it is determined whether the user has alighted. Through the fusion of the door opening and closing sensor, seat pressure sensor, and visual sensor, the judgment accuracy of the passenger's status can be improved, avoiding misjudgment of the passenger's alighting, and enhancing the riding experience and vehicle operation efficiency.

[0137] Figure 5 This is the logic diagram of the order end process provided for this embodiment. As Figure 5 shown, the determination process of the order end includes:

[0138] S501. The vehicle reaches the preset destination of the current ride order.

[0139] S502. Check whether the vehicle door is open. If it is not open, execute step S503; if it is open, execute step S505.

[0140] S503. Infer whether the passenger is still in the vehicle. If so, continue to execute S502; if not, execute S504.

[0141] Specifically, obtain the door information and seat information of the vehicle during the driver delivery process, and determine the initial status information of the passengers in the vehicle according to the door information and seat information during the driver delivery process. The initial status information indicates whether the passenger is still in the vehicle.

[0142] S504. Determine whether the passenger has got off the vehicle. If so, execute S508 to end the current ride order; if not, continue to execute S502 until it is determined that the passenger has got off the vehicle.

[0143] Specifically, according to the determination logic of the target status information, obtain the environmental monitoring information of the vehicle within a preset time period. The environmental monitoring information includes door information, seat information, and in-vehicle image information. The preset time period can be 60 seconds. Determine the target status information of the passenger according to the environmental monitoring information. The target status information can indicate whether the passenger has got off the vehicle.

[0144] S505. Determine whether the current ride order is a mechanism serial order. If so, execute S506; if not, execute S507.

[0145] S506. Continuously detect whether the seat pressure is empty. If so, execute S508 to end the current ride order; if not, continue to detect whether the seat pressure is empty.

[0146] S507. Determine whether the passenger has got off the vehicle. If so, execute S508 to end the current ride order; if not, continue to determine whether the passenger has got off the vehicle.

[0147] Specifically, according to the determination logic of the target status information, obtain the environmental monitoring information of the vehicle within a preset time period. The environmental monitoring information includes door information, seat information, and in-vehicle image information. The preset time period can be 10 seconds. Determine the target status information of the passenger according to the environmental monitoring information. The target status information can indicate whether the passenger has got off the vehicle.

[0148] For the determination logic of the target status information in this embodiment, different determination logics can be set based on different vehicle types. For example, for vehicle type A that can only seat passengers in the back row, if the doors are closed, there is no seat pressure, and the visual sensor detects no one for a certain period of time, such as 10 seconds, it is considered that the passengers have got off; for vehicle type B that can seat passengers in both the front and back rows, if there are passengers in both the front and back rows, and the front and back doors are each opened and closed once, there is no seat pressure, and the visual sensor detects no one for a certain period of time, it is considered that the passengers have got off; for the case where there is no visual sensor or the visual detection function fails, if the doors are closed and there is no seat pressure for a certain period of time, it is considered that the passengers have got off.

[0149] S508. End the current ride order.

[0150] Figure 6 The block diagram of a determination device for passenger status based on driverless driving provided by an embodiment of the present disclosure. This device is applied to a driverless vehicle, and a visual sensor is installed on the vehicle. The visual sensor is used to collect in-vehicle image information. For the sake of convenience of description, only the parts related to the embodiments of the present disclosure are shown. Referring to Figure 6 Based on this, the determination device 600 for passenger status based on driverless driving includes: an initial determination unit 601, an information acquisition unit 602, and a target determination unit 603.

[0151] The initial determination unit 601 is configured to, when the vehicle reaches the preset end point of the current ride order, determine the initial status information of the passengers in the vehicle according to the door information of the vehicle during the delivery journey and the seat information of the vehicle during the delivery journey; wherein, the door information represents the open / closed state of the doors, the seat information represents the pressure value of the seats, and the initial status information represents whether the passengers are in the vehicle;

[0152] The information acquisition unit 602 is configured to obtain the environmental monitoring information of the vehicle within a preset time period according to the initial status information; wherein, the environmental monitoring information includes at least one of door information, seat information, and in-vehicle image information;

[0153] The target determination unit 603 is configured to determine the target status information of the passengers according to the environmental monitoring information; wherein, the target status information represents whether the passengers have got off.

[0154] Figure 7 The block diagram of a determination device for passenger status based on driverless driving provided by an embodiment of the present disclosure, as Figure 7As shown in the figure, the determination device 700 for the passenger state based on driverless driving includes an initial determination unit 701, an information acquisition unit 702, and a target determination unit 703. Among them, the initial determination unit 701 includes a time determination module 7011, an information extraction module 7012, and an initial determination module 7013.

[0155] The time period determination module 7011 is configured to determine a target time period during the driver delivery process according to the door information of the vehicle during the driver delivery process; wherein, the time length of the target time period is less than or equal to the time length during the driver delivery process;

[0156] The information extraction module 7012 is configured to extract the seat information of the target time period from the seat information of the vehicle during the driver delivery process;

[0157] The initial determination module 7013 is configured to determine the initial state information of the passengers in the vehicle according to the seat information of the target time period.

[0158] In one example, the time period determination module 7011 is specifically configured to:

[0159] If it is determined according to the door information of the vehicle during the driver delivery process that the vehicle has not performed an opening operation during the driver delivery process, the time period corresponding to the driver delivery process is determined as the target time period.

[0160] In one example, the time period determination module 7011 includes:

[0161] The time determination sub-module is configured to determine the execution time corresponding to the opening operation if it is determined according to the door information of the vehicle during the driver delivery process that the vehicle has performed an opening operation during the driver delivery process;

[0162] The time period determination sub-module is configured to determine the target time period from the time period corresponding to the driver delivery process according to the execution time; wherein, the target time period represents the time period from the execution time to the moment of reaching the preset end point.

[0163] In one example, the initial determination module 7013 includes:

[0164] The first determination sub-module is configured to determine that the initial state information of the passengers in the vehicle is that the passengers are in the vehicle if there is a pressure value greater than the preset pressure threshold in the seat information of the target time period;

[0165] The second determination sub-module is configured to determine that the initial state information of the passengers in the vehicle is that the passengers are not in the vehicle if there is no pressure value greater than the preset pressure threshold in the seat information of the target time period.

[0166] In one example, the information acquisition unit 702 includes:

[0167] An information acquisition module, configured to, if the initial status information indicates that the passenger is not in the vehicle, acquire the door information, seat information, and in-vehicle image information of the vehicle within a preset time period based on a preset information acquisition period.

[0168] In one example, the target determination unit 703 includes:

[0169] A vision determination module, configured to determine a vision determination result according to the in-vehicle image information; wherein, the vision determination result represents whether there is anyone in the vehicle determined through the in-vehicle image.

[0170] A status determination module, configured to, if the door information within the preset time period indicates that the door is in a closed state, there is no pressure value greater than a preset pressure threshold in the seat information within the preset time period, and the vision determination result represents that there is no one in the vehicle, determine that the target status information of the passenger is that the passenger has got off the vehicle.

[0171] In one example, the information acquisition unit 702 includes:

[0172] A door detection module, configured to, if the initial status information indicates that the passenger is in the vehicle, detect the current door information of the vehicle in real time.

[0173] An order determination module, configured to, if the current door information of the vehicle indicates that the door is in an open state, acquire the order type of the current ride order.

[0174] An environment monitoring module, configured to acquire the environment monitoring information of the vehicle within a preset time period according to the order type of the current ride order.

[0175] In one example, the environment monitoring module is specifically configured to:

[0176] If the order type is the first type, acquire the seat information of the vehicle within a preset time period based on a preset information acquisition period.

[0177] In one example, the target determination unit 703 includes:

[0178] A pressure judgment module, configured to, if it is determined that there is no pressure value greater than a preset pressure threshold in the seat information within the preset time period, determine that the target status information of the passenger is that the passenger has got off the vehicle.

[0179] In one example, the environment monitoring module is specifically configured to:

[0180] If the order type is the second type, acquire the door information, seat information, and in-vehicle image information of the vehicle within a preset time period based on a preset information acquisition period.

[0181] In one example, the target determination unit 703 includes:

[0182] a vision determination module, configured to determine a vision determination result according to the in-vehicle image information; wherein, the vision determination result represents whether there is anyone in the vehicle determined through the in-vehicle image;

[0183] a status determination module, configured to determine that the target status information of the passenger is that the passenger has got off the vehicle if the door information within the preset time period represents that the door is in a closed state, there is no pressure value greater than a preset pressure threshold in the seat information within the preset time period, and the vision determination result represents that there is no one in the vehicle.

[0184] In one example, a display screen is installed on the vehicle, and the display screen is located at the front row seat and / or the rear row seat; the device further includes:

[0185] a confirmation boarding unit, configured to, after the vehicle arrives at the preset starting point of the current ride order, in response to a confirmation driving instruction sent by the passenger through the display screen, travel to the preset ending point based on the preset driving trajectory in the current ride order, and detect the door information of the vehicle during the driving process of delivering the passenger and the seat information of the vehicle during the driving process of delivering the passenger in real time.

[0186] An embodiment of the present disclosure provides a vehicle, on which a vision sensor is installed, and the vision sensor is used to collect in-vehicle image information, and the vehicle can execute a method for determining the passenger status based on driverless driving.

[0187] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device.

[0188] Figure 8 is a structural block diagram of an electronic device provided by an embodiment of the present disclosure, as Figure 8 shown, the electronic device 800 includes: at least one processor 802; and a memory 801 communicatively connected to the at least one processor 802; wherein, the memory stores instructions executable by the at least one processor 802, and the instructions are executed by the at least one processor 802 so that the at least one processor 802 can execute the method for determining the passenger status based on driverless driving of the present disclosure.

[0189] The electronic device 800 further includes a receiver 803 and a transmitter 804. The receiver 803 is configured to receive instructions and data sent by other devices, and the transmitter 804 is configured to send instructions and data to external devices.

[0190] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0191] According to an embodiment of the present disclosure, the present disclosure also provides a computer program product, which includes: a computer program stored in a readable storage medium, and at least one processor of the electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the electronic device to execute the solution provided in any of the above embodiments.

[0192] Figure 9 FIG. shows a schematic block diagram of an exemplary electronic device 900 that can be used to implement embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, for example, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, for example, personal digital processors, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0193] As Figure 9 shown, the device 900 includes a computing unit 901, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 902 or a computer program loaded from a storage unit 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 can also be stored. The computing unit 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0194] A plurality of components in the device 900 are connected to the I / O interface 905, including: an input unit 906, such as a keyboard, a mouse, etc.; an output unit 907, such as various types of displays, speakers, etc.; a storage unit 908, such as a magnetic disk, an optical disk, etc.; and a communication unit 909, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 909 allows the device 900 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0195] The computing unit 901 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 executes the various methods and processes described above, such as the method for determining the passenger state based on driverless driving. For example, in some embodiments, the method for determining the passenger state based on driverless driving can be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as the storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 900 via the ROM 902 and / or the communication unit 909. When the computer program is loaded into the RAM 903 and executed by the computing unit 901, one or more steps of the method for determining the passenger state based on driverless driving described above can be executed. Alternatively, in other embodiments, the computing unit 901 can be configured to execute the method for determining the passenger state based on driverless driving by any other suitable means (e.g., by means of firmware).

[0196] Various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a dedicated or general-purpose programmable processor, and can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0197] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing devices, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program codes can be executed entirely on the machine, partially on the machine, executed partially on the machine and partially on a remote machine as an independent software package, or executed entirely on a remote machine or server.

[0198] In the context of this disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0199] To provide for interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic, speech, or tactile input).

[0200] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0201] A computer system may include a client and a server. The client and the server are generally far from each other and usually interact via a communication network. The relationship between the client and the server is created by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, and solves the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services ("Virtual Private Server", or simply "VPS"). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0202] It should be understood that various forms of the processes shown above can be used, steps can be reordered, added or deleted. For example, the steps described in the present disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and no limitation is imposed herein.

[0203] The above specific embodiments do not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. A method for determining the passenger status based on driverless driving, which is applied to a driverless vehicle equipped with a vision sensor for collecting in-vehicle image information; the method includes: When the vehicle arrives at the preset end point of the current ride order, determine the initial status information of the passengers in the vehicle according to the door information of the vehicle during the delivery trip and the seat information of the vehicle during the delivery trip; wherein, the door information represents the open / closed state of the door, the seat information represents the pressure value of the seat, and the initial status information represents whether the passengers are in the vehicle. According to the initial status information, obtain the environmental monitoring information of the vehicle within a preset time period; wherein, the environmental monitoring information includes at least one of door information, seat information, and in-vehicle image information. According to the environmental monitoring information, determine the target status information of the passengers; wherein, the target status information represents whether the passengers have got off the vehicle.

2. The method according to claim 1, wherein The step of determining the initial status information of the passengers in the vehicle according to the door information of the vehicle during the delivery trip and the seat information of the vehicle during the delivery trip includes: According to the door information of the vehicle during the delivery trip, determine the target time period during the delivery trip; wherein, the time length of the target time period is less than or equal to the time length of the delivery trip. Extract the seat information of the target time period from the seat information of the vehicle during the delivery trip. According to the seat information of the target time period, determine the initial status information of the passengers in the vehicle.

3. The method according to claim 2, wherein The step of determining the target time period during the delivery trip according to the door information of the vehicle during the delivery trip includes: If it is determined according to the door information of the vehicle during the delivery trip that the vehicle has not performed an opening operation during the delivery trip, then determine the corresponding time period during the delivery trip as the target time period.

4. The method according to claim 2 or 3, wherein, The step of determining the target time period during the delivery trip according to the door information of the vehicle during the delivery trip includes: If it is determined according to the door information of the vehicle during the delivery trip that the vehicle has performed an opening operation during the delivery trip, then determine the execution time corresponding to the opening operation. According to the execution time, determine the target time period from the corresponding time period during the delivery trip; wherein, the target time period represents the time period from the execution time to the moment of arriving at the preset end point.

5. The method according to any one of claims 2-4, wherein The step of determining the initial status information of the passengers in the vehicle according to the seat information of the target time period includes: If there is a pressure value greater than the preset pressure threshold in the seat information of the target time period, then determine that the initial status information of the passengers in the vehicle is that the passengers are in the vehicle. If there is no pressure value greater than the preset pressure threshold in the seat information of the target time period, then determine that the initial status information of the passengers in the vehicle is that the passengers are not in the vehicle.

6. The method according to any one of claims 1-5, wherein The step of obtaining the environmental monitoring information of the vehicle within a preset time period according to the initial status information includes: If the initial status information is that the passengers are not in the vehicle, then based on a preset information acquisition period, obtain the door information, seat information, and in-vehicle image information of the vehicle within the preset time period.

7. The method according to claim 6, wherein, Determining the target status information of the passenger according to the environmental monitoring information includes: Determining a visual determination result according to the in-vehicle image information; wherein, the visual determination result represents whether there is anyone in the vehicle determined through the in-vehicle image; If the door information within the preset time period indicates that the door is in a closed state, there is no pressure value greater than the preset pressure threshold in the seat information within the preset time period, and the visual determination result indicates that there is no one in the vehicle, then determine that the target status information of the passenger is that the passenger has got off the vehicle.

8. The method according to any one of claims 1-7, wherein, Obtaining the environmental monitoring information of the vehicle within a preset time period according to the initial status information includes: If the initial status information is that the passenger is in the vehicle, then detect the current door information of the vehicle in real time; If the current door information of the vehicle indicates that the door is in an open state, then obtain the order type of the current ride order; Obtain the environmental monitoring information of the vehicle within a preset time period according to the order type of the current ride order.

9. The method according to claim 8, wherein Obtaining the environmental monitoring information of the vehicle within a preset time period according to the order type of the current ride order includes: If the order type is the first type, then obtain the seat information of the vehicle within a preset time period based on a preset information acquisition period.

10. The method according to claim 9, wherein Determining the target status information of the passenger according to the environmental monitoring information includes: If it is determined that there is no pressure value greater than the preset pressure threshold in the seat information within the preset time period, then determine that the target status information of the passenger is that the passenger has got off the vehicle.

11. The method according to claim 8, wherein, Obtaining the environmental monitoring information of the vehicle within a preset time period according to the order type of the current ride order includes: If the order type is the second type, then obtain the door information, seat information, and in-vehicle image information of the vehicle within a preset time period based on a preset information acquisition period.

12. The method according to claim 11, wherein, Determining the target status information of the passenger according to the environmental monitoring information includes: Determining a visual determination result according to the in-vehicle image information; wherein, the visual determination result represents whether there is anyone in the vehicle determined through the in-vehicle image; If the door information within the preset time period indicates that the door is in a closed state, there is no pressure value greater than the preset pressure threshold in the seat information within the preset time period, and the visual determination result indicates that there is no one in the vehicle, then determine that the target status information of the passenger is that the passenger has got off the vehicle.

13. According to the method of any one of claims 1-12, a display screen is installed on the vehicle, and the display screen is located at the front row seat and / or the rear row seat; the method further includes: After the vehicle arrives at the preset starting point of the current ride order, in response to a confirmation driving instruction issued by the passenger through the display screen, travel to the preset ending point based on the preset driving trajectory in the current ride order, and detect the door information of the vehicle during the driving process and the seat information of the vehicle during the driving process in real time.

14. A device for determining the status of a passenger based on driverless driving, the device is applied to a driverless vehicle, and a visual sensor is installed on the vehicle, and the visual sensor is used to collect in-vehicle image information; the device includes: An initial determination unit, configured to determine initial status information of a passenger in the vehicle according to door information of the vehicle during the driving process to the current ride order and seat information of the vehicle during the driving process when the vehicle arrives at a preset end point of the current ride order; wherein, the door information represents the open / closed state of the door, the seat information represents the pressure value of the seat, and the initial status information represents whether the passenger is in the vehicle. An information acquisition unit, configured to acquire environmental monitoring information of the vehicle within a preset time period according to the initial status information; wherein, the environmental monitoring information includes at least one of door information, seat information, and in-vehicle image information. A target determination unit, configured to determine target status information of the passenger according to the environmental monitoring information; wherein, the target status information represents whether the passenger has gotten off the vehicle.

15. An electronic device, comprising: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1-13.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the method according to any one of claims 1-13.

17. A computer program product, wherein, Including a computer program, which when executed by a processor implements the steps of the method according to any one of claims 1-13.

18. A vehicle, wherein, A vision sensor is installed on the vehicle, and the vision sensor is used to collect in-vehicle image information, and the vehicle executes the method according to any one of claims 1-13.

Images