Riding intention recognition method and device, storage medium and vehicle

Abstract

The invention provides a riding intention recognition method and device, a storage medium and a vehicle, and relates to the technical field of unmanned driving. The riding intention identification method comprises the following steps: acquiring three-dimensional depth data of a target area corresponding to a target vehicle in real time, wherein the three-dimensional depth data comprises attitude information of each waiting person in the target area; acquiring radar sensing data of the target area in real time, wherein the radar sensing data comprises motion state information of each waiting person in the target area; identifying a target waiting person corresponding to the target vehicle according to the three-dimensional depth data and the radar sensing data; and performing riding intention recognition on the target waiting person to obtain a riding intention recognition result, so as to control the target vehicle based on the riding intention recognition result. According to the invention, automatic identification of the riding intention of the tourist and automatic control of the vehicle are realized, and the limitation of dependence on manual operation in the prior art is solved.

Description

Technical Field

[0001] The present invention relates to the field of unmanned driving technology, and in particular to a method, device, storage medium and vehicle for recognizing passenger intention. Background Art

[0002] At present, low-speed driverless sightseeing vehicles mainly run on fixed routes. When tourists want to take a ride, they need to manually identify the intention of the tourists' gestures and then manually stop the vehicle to pick up the passengers. This method relies on manual operation and has a low degree of automation.

[0003] The existing system needs to rely on manual recognition of tourists' intentions and operation of vehicles, which increases labor costs and reduces the degree of automation of the system. In addition, when manually recognizing tourists' intentions, there are problems such as tourists being ignored or intentions not being recognized in time, resulting in operation delays and affecting tourists' experience. In addition, people have limited energy, and long-term high-intensity work can easily lead to fatigue, thus creating safety hazards. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a method, device, storage medium and vehicle for identifying boarding intention to solve the above technical problems.

[0005] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: a method for identifying boarding intention, comprising: acquiring three-dimensional depth data of a target area corresponding to a target vehicle in real time, the three-dimensional depth data including posture information of each waiting person in the target area; acquiring radar perception data of the target area in real time, the radar perception data including motion status information of each waiting person in the target area; identifying a target waiting person corresponding to the target vehicle based on the three-dimensional depth data and the radar perception data; performing boarding intention identification on the target waiting person to obtain a boarding intention identification result, so as to control the target vehicle based on the boarding intention identification result.

[0006] The beneficial effect of the present invention is that the method identifies the target waiting person corresponding to the target vehicle based on the three-dimensional depth data and radar perception data acquired in real time, and recognizes the intention of the target waiting person to board the vehicle, so as to control the target vehicle according to the recognition result of the intention to board the vehicle. Through the present method, the automatic recognition of the intention of tourists to board the vehicle can be realized, which greatly reduces the occurrence of tourists being ignored or untimely recognized, and solves the limitation of relying on manual operation in the prior art.

[0007] Based on the above technical solution, the present invention can also be improved as follows.

[0008] Furthermore, after the target waiting person's intention to board the bus is identified and the identification result of the intention to board the bus is obtained, it also includes: determining a first position of the target waiting person based on the radar perception data; performing a three-dimensional posture analysis on the target waiting person based on the three-dimensional depth data to obtain a three-dimensional posture analysis result; calculating a second position of the target waiting person based on the three-dimensional posture analysis result; performing a position comparison on the first position and the second position to obtain a position comparison result; and determining that the result of the identification of the intention to board the bus is accurate when the position comparison result satisfies a preset position comparison condition.

[0009] Furthermore, the target waiting person's intention to board the bus is identified to obtain a result of the identification of the intention to board the bus, including: determining whether the target waiting person is in a waving state based on the three-dimensional depth data, and determining whether the target waiting person's hand is facing the target vehicle based on the three-dimensional depth data; when it is determined that the target waiting person is in a waving state, and / or when it is determined that the target waiting person's hand is facing the target vehicle, it is determined that the target waiting person has the intention to board the bus.

[0010] Further, determining whether the target waiter is in a waving state based on the three-dimensional depth data includes: generating a hand coordinate point sequence corresponding to the target waiter based on the three-dimensional depth data, the hand coordinate point sequence being used to describe the hand position of the target waiter; calculating the hand movement amplitude of the target waiter based on the hand coordinate point sequence; when the hand movement amplitude is greater than a preset amplitude threshold, performing a fast Fourier transform calculation based on the hand coordinate point sequence to obtain the hand swing frequency of the target waiter; when the hand swing frequency is within a preset frequency range, determining that the target waiter is in a waving state.

[0011] Further, determining whether the target waiter's hand is facing the target vehicle based on the three-dimensional depth data includes: calculating a direction vector based on the three-dimensional depth data to obtain a first direction vector and a second direction vector; the first direction vector is used to characterize the movement direction of the target waiter's hand relative to the shoulder, and the second direction vector is used to characterize the movement direction of the target waiter's hand relative to the target vehicle; calculating the vector angle between the first direction vector and the second direction vector; in response to the vector angle being less than a preset angle threshold, determining that the target waiter's hand is facing the target vehicle.

[0012] Further, the identifying the target waiting person corresponding to the target vehicle based on the three-dimensional depth data and the radar perception data includes: for each waiting person in the target area, determining, based on the radar perception data, a distance change trend between each waiting person and the target vehicle, a speed direction of each waiting person, and a relative distance between each waiting person and the target vehicle within a preset first time period; and determining as a target waiting person a waiting person whose distance change trend is a decreasing distance, whose speed direction is pointing to the target vehicle, and whose relative distance is less than a preset distance threshold.

[0013] Furthermore, the control of the target vehicle based on the result of the riding intention recognition includes: identifying obstacles in the target area according to the three-dimensional depth data and the radar perception data; predicting the motion trajectory of each obstacle according to the three-dimensional depth data and the radar perception data to obtain the motion trajectory corresponding to each obstacle; identifying collision risks according to the motion trajectory corresponding to each obstacle to obtain a risk identification result; and controlling the movement of the target vehicle according to the risk identification result.

[0014] In order to solve the above technical problems, this embodiment further provides a vehicle boarding intention recognition device, comprising:

[0015] A depth data acquisition module, used to acquire three-dimensional depth data of a target area corresponding to a target vehicle in real time, wherein the three-dimensional depth data includes posture information of each person waiting for the vehicle in the target area;

[0016] A radar data acquisition module, used for acquiring radar sensing data of the target area in real time, wherein the radar sensing data includes motion status information of each person waiting for the bus in the target area;

[0017] A target recognition module, used for identifying a target waiting person corresponding to the target vehicle according to the three-dimensional depth data and the radar perception data;

[0018] The intention recognition module is used to recognize the boarding intention of the target waiting person and obtain the boarding intention recognition result so as to control the target vehicle based on the boarding intention recognition result.

[0019] In order to solve the above technical problems, the present embodiment also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the method for identifying the intention to board a vehicle as described above.

[0020] In order to solve the above technical problems, the present embodiment also provides a vehicle, including a memory, a processor, and a program stored in the memory and running on the processor, and when the processor executes the program, the steps of the method for identifying the intention of riding a vehicle as described above are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 A flow chart of a method for identifying boarding intention according to the present invention;

[0022] Figure 2 Schematic diagram of the vehicle boarding intention recognition device of the present invention. DETAILED DESCRIPTION

[0023] The principles and features of the present invention are described below. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0024] Embodiment 1

[0025] like Figure 1 As shown, this embodiment provides a method for identifying boarding intention, including:

[0026] S101 . Acquire three-dimensional depth data of a target area corresponding to a target vehicle in real time, wherein the three-dimensional depth data includes posture information of each person waiting for the vehicle in the target area.

[0027] The three-dimensional depth data can be a depth map collected by a depth camera. A depth camera, such as a Kinect depth camera, is installed on the target vehicle. The depth camera can capture the depth information in the three-dimensional space and collect the three-dimensional depth data in real time.

[0028] S102: Acquire radar perception data of the target area in real time, wherein the radar perception data includes motion status information of each person waiting for the bus in the target area.

[0029] Radar perception data is data collected by radar. Radar is installed on the target vehicle to collect radar perception data in real time. The radar can use millimeter wave radar, which provides accurate perception of the surrounding environment, especially in complex or severe weather conditions, and can ensure high-precision detection of the target. Radar perception data includes the motion status information of each person waiting in the target area. The motion status information of the person waiting refers to the distance, speed and other data of the person waiting collected by the radar.

[0030] The target area refers to the front and side of the target vehicle, and is set according to actual usage requirements. Specifically, the target area is adjusted and set by adjusting the installation position and angle of the depth camera and radar, so as to collect three-dimensional depth data and radar perception data based on the target area. It should be noted that the setting of the target area should take into account the position and size of the platform, waiting area and other areas corresponding to the target vehicle, so as to ensure that the posture information and motion status information of the waiting people in the platform, waiting area and other areas can be fully and accurately collected.

[0031] S103: Identify a target waiting person corresponding to the target vehicle according to the three-dimensional depth data and the radar perception data.

[0032] S104, performing boarding intention recognition on the target vehicle waiting person to obtain a boarding intention recognition result, so as to control the target vehicle based on the boarding intention recognition result.

[0033] In actual application scenarios, there may be many people waiting for the bus at the platform. That is, when collecting data, the 3D depth data and radar perception data of the target area contain information about all the people waiting for the bus. However, not all the people waiting for the bus want to take the target vehicle. Therefore, it is necessary to identify the target people waiting for the bus based on the 3D depth data and radar perception data. The target people waiting for the bus refer to those who are initially judged to want to take the target vehicle.

[0034] Based on the three-dimensional depth data and radar perception data acquired in real time, this method identifies the target waiting person corresponding to the target vehicle, and identifies the target waiting person's intention to board the vehicle, so as to control the target vehicle based on the recognition result of the intention to board the vehicle. Through this method, the automatic recognition of tourists' intention to board the vehicle can be realized, which greatly reduces the occurrence of tourists being ignored or untimely recognized, and solves the limitation of relying on manual operation in the existing technology. This method is applicable to a variety of usage scenarios, such as sightseeing car scenarios, ordinary bus scenarios and other vehicle control scenarios.

[0035] Optionally, in an embodiment, after the target waiting person's boarding intention is identified and the boarding intention identification result is obtained, the method further includes: determining the first position of the target waiting person based on the radar perception data; performing a three-dimensional posture analysis on the target waiting person based on the three-dimensional depth data to obtain a three-dimensional posture analysis result; calculating the second position of the target waiting person based on the three-dimensional posture analysis result; performing a position comparison on the first position and the second position to obtain a position comparison result; and determining that the boarding intention identification result is accurate when the position comparison result satisfies a preset position comparison condition.

[0036] To improve the accuracy of intention recognition, radar perception data and three-dimensional depth data are combined for cross-validation. The radar perception data collected by the radar can obtain the position of the target waiting person relative to the target vehicle, that is, the first position. In this embodiment, the radar perception data and the three-dimensional depth data use the same coordinate system, that is, the vehicle is used as the origin and the XOY plane is constructed with the horizontal plane.

[0037] The three-dimensional depth data collected by the depth camera can generate a posture skeleton graph, that is, the three-dimensional posture analysis result. Based on the existing image processing technology, a posture skeleton graph containing all people can be generated, or a separate posture skeleton graph can be generated for each person. In this embodiment, it is necessary to generate a separate posture skeleton graph for each person for a frame of the picture taken by the depth camera. The posture skeleton graph consists of a series of key points (such as shoulders, elbows, wrists, hips, knees and ankles, etc.) and lines connecting these points. It is an image that characterizes the corresponding postures of the key points of the human skeleton. Based on the posture skeleton graph, the position of the target waiting person relative to the target vehicle, that is, the second position, is obtained.

[0038] For each target waiting person, the first position and the second position corresponding to the target waiting person need to be calculated respectively. For each target waiting person, the first position and the second position corresponding to the target waiting person are compared to determine whether the first position and the second position are consistent, and the position comparison result is obtained. The position comparison result is consistent or inconsistent. The consistent position indicates that the distance between the first position and the second position is within the preset error range, and the inconsistent position indicates that the distance between the first position and the second position exceeds the preset error range.

[0039] In the case where the position comparison result satisfies the preset position comparison conditions, the boarding intention recognition result is determined to be accurate, specifically: for a target waiting person with boarding intention, if the position comparison result corresponding to the target waiting person is position consistency, then the boarding intention recognition result of the target waiting person is determined to be accurate; if the position comparison result corresponding to the target waiting person is position inconsistency, then the boarding intention recognition result of the target waiting person is determined to be inaccurate. For a target waiting person without boarding intention, if the position comparison result corresponding to the target waiting person is position inconsistency, then the boarding intention recognition result of the target waiting person is determined to be accurate; if the position comparison result corresponding to the target waiting person is position consistency, then the boarding intention recognition result of the target waiting person is determined to be inaccurate.

[0040] In some embodiments, on the basis of position comparison, the accuracy of the boarding intention recognition result is judged in combination with direction comparison, specifically: the position comparison result is calculated in the above manner. The moving direction of the target waiting person is determined based on the radar perception data. The gesture direction of the target waiting person is determined based on the three-dimensional posture analysis result. A direction comparison is performed on the moving direction and the gesture direction to obtain a direction comparison result. When the position comparison result satisfies the preset position comparison condition and the direction comparison result satisfies the preset direction comparison condition, it is determined that the boarding intention recognition result is accurate.

[0041] Millimeter wave radar can calculate the moving direction of the target person by analyzing the relative position change between the target person and the target vehicle. The Kinect depth camera calculates the direction of the gesture through the three-dimensional coordinates in the 3D space, and can directly determine the direction of the gesture based on the vector angle.

[0042] For each target waiting person, the moving direction and gesture direction corresponding to the target waiting person need to be calculated respectively. For each target waiting person, the moving direction and gesture direction corresponding to the target waiting person are compared to determine whether the moving direction and the gesture direction are consistent, and the direction comparison result is obtained. The direction comparison result is consistent or inconsistent. The consistent direction indicates that the angle between the moving direction and the gesture direction is within the preset angle error range, and the inconsistent direction indicates that the angle between the moving direction and the gesture direction exceeds the preset angle error range.

[0043] In the case where the position comparison result satisfies the preset position comparison condition and the direction comparison result satisfies the preset direction comparison condition, the boarding intention recognition result is determined to be accurate, specifically: for a target waiting person with boarding intention, if the position comparison result corresponding to the target waiting person is position consistency and the direction comparison result is direction consistency, then the boarding intention recognition result of the target waiting person is determined to be accurate; otherwise, the boarding intention recognition result of the target waiting person is determined to be inaccurate. For a target waiting person without boarding intention, if the position comparison result corresponding to the target waiting person is position inconsistency and the direction comparison result is direction inconsistency, then the boarding intention recognition result of the target waiting person is determined to be accurate; otherwise, the boarding intention recognition result of the target waiting person is determined to be inaccurate.

[0044] Optionally, in an embodiment, the target waiting person's intention to board the bus is identified to obtain a result of the identification of the target waiting person's intention to board the bus, including: determining whether the target waiting person is in a waving state based on the three-dimensional depth data, and determining whether the target waiting person's hand is facing the target vehicle based on the three-dimensional depth data; when it is determined that the target waiting person is in a waving state, and / or when it is determined that the target waiting person's hand is facing the target vehicle, it is determined that the target waiting person has the intention to board the bus.

[0045] For each target waiting person, gesture recognition needs to be performed in the above manner to determine whether the target waiting person has the intention to board the bus. The target waiting person can be determined to have the intention to board the bus if one of the conditions is met, that is, the target waiting person is in a waving state or the target waiting person's hand is facing the target vehicle, then the target waiting person is determined to have the intention to board the bus; if the target waiting person is not in a waving state and the target waiting person's hand is not facing the target vehicle, then the target waiting person has no intention to board the bus. It is also possible to determine that the target waiting person has the intention to board the bus if two conditions are met at the same time, that is, the target waiting person is in a waving state and the target waiting person's hand is facing the target vehicle, then the target waiting person is determined to have the intention to board the bus; otherwise, the target waiting person has no intention to board the bus.

[0046] Optionally, in an embodiment, determining whether the target waiter is in a waving state based on the three-dimensional depth data includes: generating a hand coordinate point sequence corresponding to the target waiter based on the three-dimensional depth data, the hand coordinate point sequence being used to describe the hand position of the target waiter; calculating the hand movement amplitude of the target waiter based on the hand coordinate point sequence; when the hand movement amplitude is greater than a preset amplitude threshold, performing a fast Fourier transform calculation based on the hand coordinate point sequence to obtain the hand swing frequency of the target waiter; when the hand swing frequency is within a preset frequency range, determining that the target waiter is in a waving state.

[0047] The Kinect depth camera is used to capture the three-dimensional posture and gesture of the waiting person, and uses its depth perception ability to accurately model the position, posture, gesture, etc. of the waiting person. The hand coordinate point sequence is generated in the following way: based on the three-dimensional depth data, a posture skeleton image sequence for the target waiting person is generated. The posture skeleton image sequence includes multiple continuous posture skeleton images, and the number of posture skeleton images is set according to the usage requirements, for example, 30 frames. The hand coordinates are extracted from each frame image of the posture skeleton image sequence to generate a hand coordinate point sequence.

[0048] According to the hand coordinate point sequence, the hand movement amplitude of the target waiting person is calculated, specifically: the hand movement amplitude is the horizontal movement amplitude of the hand, that is, the change of the hand coordinate point in the X-axis direction. According to the hand coordinate point sequence, the X-coordinate difference of two adjacent hand coordinate points is first calculated, and then the average value of each X-coordinate difference is calculated to obtain the hand movement amplitude.

[0049] Both the amplitude threshold and the frequency range can be set according to actual conditions. In this embodiment, the amplitude threshold is set to 0.2 meters, and the frequency range is set to 0.5Hz to 2Hz. When the hand movement amplitude is greater than the preset amplitude threshold, further calculate and determine whether the hand swing frequency is within the preset frequency range; if the hand movement amplitude is less than or equal to the preset amplitude threshold, the target waiting person is determined to be in a non-waving state. For the hand swing frequency, when the hand swing frequency is within the preset frequency range, the target waiting person is determined to be in a waving state; otherwise, the target waiting person is determined to be in a non-waving state.

[0050] Optionally, in an embodiment, determining whether the target waiter's hand is facing the target vehicle based on the three-dimensional depth data includes: calculating a direction vector based on the three-dimensional depth data to obtain a first direction vector and a second direction vector; the first direction vector is used to characterize the movement direction of the target waiter's hand relative to the shoulder, and the second direction vector is used to characterize the movement direction of the target waiter's hand relative to the target vehicle; calculating the vector angle between the first direction vector and the second direction vector; in response to the vector angle being less than a preset angle threshold, determining that the target waiter's hand is facing the target vehicle.

[0051] When calculating the first direction vector and the second direction vector, the calculation is performed based on the posture skeleton diagram corresponding to the current moment, which refers to the latest posture skeleton diagram in time. The shoulder coordinates and hand coordinates of the target waiting person are extracted from the posture skeleton diagram. In this coordinate system, the coordinates of the vehicle corresponding to the origin of the coordinate system, that is, the vehicle are (0, 0, 0). In this embodiment, the angle threshold is set to 36°. For the calculation of the vector angle, the vector angle formula can be used for calculation. For each target waiting person, it is necessary to judge whether the hand is facing the target vehicle by the above method. When the vector angle is less than the preset angle threshold, it is determined that the hand of the target waiting person is facing the target vehicle; otherwise, it is determined that the hand of the target waiting person is not facing the target vehicle.

[0052] Optionally, in an embodiment, the identifying the target waiter corresponding to the target vehicle based on the three-dimensional depth data and the radar perception data includes: for each waiter in the target area, determining, based on the radar perception data, a distance change trend between each waiter and the target vehicle, a speed direction of each waiter, and a relative distance between each waiter and the target vehicle within a preset first time period; and determining a waiter whose distance change trend is a decreasing distance, whose speed direction is pointing to the target vehicle, and whose relative distance is less than a preset distance threshold as a target waiter.

[0053] For distance, millimeter-wave radar emits electromagnetic waves and receives reflected signals to detect the distance of the target object. When the propagation time of the electromagnetic waves becomes shorter and shorter, the reflected signal returns faster, which means the target is closer to the radar.

[0054] For the speed direction, the radar analyzes the time difference between two consecutive return signals and calculates the target's moving speed. If the target's speed component points to the vehicle (that is, the target is approaching the vehicle), its speed value will be negative, indicating that the target is moving toward the radar (vehicle). If the target's speed component moves away from the vehicle, the speed value is positive, indicating that the target is moving away from the radar.

[0055] In this embodiment, a waiting person who simultaneously satisfies the three conditions that the distance change trend is a decreasing distance, the speed direction is directed toward the target vehicle, and the relative distance is less than a preset distance threshold is determined as a target waiting person. In some embodiments, the target waiting person can also be identified only by the distance change trend and the speed direction, and a waiting person who simultaneously satisfies the two conditions that the distance change trend is a decreasing distance and the speed direction is directed toward the target vehicle is determined as a target waiting person.

[0056] The control of the target vehicle based on the boarding intention recognition result includes: generating a vehicle control instruction according to the boarding intention recognition result to control the target vehicle. Specifically, if a target waiting person with the boarding intention is identified, a parking instruction is generated to control the target vehicle to stop, ensuring that the target vehicle stops accurately at the target waiting person's predetermined position. After the target vehicle stops, a reminder instruction is generated to control the vehicle display screen or voice system to issue a reminder message to inform the target waiting person that the target vehicle has stopped and remind him to get on and off the vehicle.

[0057] Optionally, in an embodiment, controlling the target vehicle based on the result of the riding intention recognition includes: identifying obstacles in the target area according to the three-dimensional depth data and the radar perception data; predicting the motion trajectory of each obstacle according to the three-dimensional depth data and the radar perception data to obtain the motion trajectory corresponding to each obstacle; identifying collision risks according to the motion trajectory corresponding to each obstacle to obtain a risk identification result; and controlling the movement of the target vehicle according to the risk identification result.

[0058] During the driving process, the target vehicle can be controlled through the automatic obstacle avoidance mechanism. The Kinect depth camera and millimeter wave radar perceive the surrounding environment of the vehicle in real time, and detect the position, distance and motion state of obstacles. According to the three-dimensional depth information of obstacles captured by the Kinect depth camera and the radar perception data measured by the millimeter wave radar, the movement of pedestrians is predicted through Kalman filtering, and whether there is a risk of collision is predicted. When there is a risk of collision, the path of the target vehicle is optimized, and the speed of the target vehicle is adjusted or an emergency stop is initiated through proportional, integral, and differential control (PID).

[0059] For example, the depth camera detects the position of the pedestrian as (3, 2, 0), and the millimeter-wave radar detects that the distance to the obstacle is decreasing, the speed is 1.5m / s, and the direction is the vehicle's forward direction. The above data is fused through Kalman filtering, and it is predicted that the pedestrian will move to the target vehicle's driving path in the next 2 seconds. Then the collision time is calculated to be 1.2s, which is less than the threshold of 2s, and the potential danger of collision is detected. The path needs to be optimized, and the speed of the vehicle is adjusted to 1m / s through PID control to bypass smoothly. If the collision time is detected to be less than 0.6s, an emergency stop is made to ensure that the vehicle safely bypasses the obstacle and avoids collision. During the bypass process, continuous monitoring of the environment is maintained and the vehicle's driving path is dynamically adjusted.

[0060] In summary, the bus waiter approaches the target vehicle and makes a specific ride gesture to indicate that he / she wants to board the target vehicle. The Kinect depth camera captures the posture information of the target bus waiter in real time, while the millimeter wave radar collects real-time data of the surrounding environment. The millimeter wave radar is combined with the Kinect depth camera to provide multiple redundancies, which improves the accuracy and reliability of recognition. Through this method, high-precision, all-weather tourist intention recognition and automatic parking are achieved, which overcomes the limitations of traditional image processing, improves the intelligence and safety of unmanned sightseeing vehicles, and enhances the experience of tourists. Through this method, automatic control of vehicles can be achieved, which is especially suitable for low-speed unmanned sightseeing vehicles, significantly improves the intelligence and automation level of low-speed unmanned sightseeing vehicles, and solves the limitations of relying on manual operation in the existing technology.

[0061] Embodiment 2

[0062] like Figure 2 As shown, this embodiment provides a vehicle-riding intention recognition device 200, comprising:

[0063] A depth data acquisition module 201 is used to acquire three-dimensional depth data of a target area corresponding to a target vehicle in real time, wherein the three-dimensional depth data includes posture information of each person waiting for the vehicle in the target area;

[0064] A radar data acquisition module 202, for acquiring radar sensing data of the target area in real time, wherein the radar sensing data includes motion status information of each person waiting for the bus in the target area;

[0065] A target recognition module 203 is used to recognize a target waiting person corresponding to the target vehicle according to the three-dimensional depth data and the radar perception data;

[0066] The intention recognition module 204 is used to recognize the boarding intention of the target waiting person and obtain a boarding intention recognition result so as to control the target vehicle based on the boarding intention recognition result.

[0067] Optionally, in an embodiment, after the intention recognition module 204, a cross-validation module is further included, and the cross-validation module is used to:

[0068] Determine the first position of the target waiting person based on the radar perception data; perform a three-dimensional posture analysis on the target waiting person based on the three-dimensional depth data to obtain a three-dimensional posture analysis result; calculate the second position of the target waiting person based on the three-dimensional posture analysis result; perform a position comparison on the first position and the second position to obtain a position comparison result; and determine that the result of the boarding intention recognition is accurate when the position comparison result meets a preset position comparison condition.

[0069] Optionally, in an embodiment, the intention recognition module 204 includes:

[0070] a gesture recognition unit, configured to determine whether the target waiting person is waving according to the three-dimensional depth data, and determine whether the hand of the target waiting person is facing the target vehicle according to the three-dimensional depth data;

[0071] The intention recognition unit is used to determine that the target waiting person has the intention to board the vehicle when it is determined that the target waiting person is in a waving state and / or when it is determined that the hand of the target waiting person is facing the target vehicle.

[0072] Optionally, in an embodiment, the gesture recognition unit includes:

[0073] A sequence generation subunit, used for generating a hand coordinate point sequence corresponding to the target waiting person according to the three-dimensional depth data, wherein the hand coordinate point sequence is used for describing the hand position of the target waiting person;

[0074] An amplitude calculation subunit, used for calculating the hand movement amplitude of the target waiting person according to the hand coordinate point sequence;

[0075] A frequency calculation subunit, configured to perform a fast Fourier transform calculation based on the hand coordinate point sequence to obtain the hand swing frequency of the target waiting person when the hand movement amplitude is greater than a preset amplitude threshold;

[0076] The waving identification subunit is used to determine that the target waiting person is in a waving state when the hand swing frequency is within a preset frequency range.

[0077] Optionally, in an embodiment, the gesture recognition unit further includes:

[0078] a direction vector calculation subunit, configured to calculate a direction vector according to the three-dimensional depth data to obtain a first direction vector and a second direction vector; the first direction vector is used to represent the movement direction of the hand of the target waiting person relative to the shoulder, and the second direction vector is used to represent the movement direction of the hand of the target waiting person relative to the target vehicle;

[0079] An angle calculation subunit, used to calculate the vector angle between the first direction vector and the second direction vector;

[0080] The hand direction recognition subunit is used to determine that the hand of the target waiting person is facing the target vehicle in response to the vector angle being less than a preset angle threshold.

[0081] Optionally, in an embodiment, the target identification module 203 includes:

[0082] A data processing unit is used to determine, for each of the waiting persons in the target area, a distance change trend between each waiting person and the target vehicle within a preset first time period, a speed direction of each waiting person, and a relative distance between each waiting person and the target vehicle according to the radar sensing data;

[0083] The target identification unit is used to determine a waiting person whose distance change trend is decreasing, whose speed direction is pointing to the target vehicle, and whose relative distance is less than a preset distance threshold as a target waiting person.

[0084] Optionally, in an embodiment, the intention recognition module 204 further includes:

[0085] an obstacle identification unit, configured to identify obstacles in the target area according to the three-dimensional depth data and the radar perception data;

[0086] A trajectory calculation unit, configured to predict a motion trajectory of each obstacle according to the three-dimensional depth data and the radar perception data, and obtain a motion trajectory corresponding to each obstacle;

[0087] A risk identification unit, used to identify the collision risk according to the motion trajectory corresponding to each obstacle, and obtain a risk identification result;

[0088] A vehicle control unit is used to control the movement of the target vehicle according to the risk identification result.

[0089] Embodiment 3

[0090] This embodiment provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the method for identifying boarding intention as described in Example 1.

[0091] Embodiment 4

[0092] This embodiment provides a vehicle, including a memory, a processor, and a program stored in the memory and running on the processor, and when the processor executes the program, the steps of the method for identifying the boarding intention as described in the first embodiment are implemented.

[0093] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0094] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A method for identifying a person's intention to board a vehicle, characterized in that: include: Acquire three-dimensional depth data of a target area corresponding to a target vehicle in real time, wherein the three-dimensional depth data includes posture information of each person waiting for the vehicle in the target area; Acquire radar sensing data of the target area in real time, wherein the radar sensing data includes motion status information of each person waiting for a bus in the target area; Identify a target waiting person corresponding to the target vehicle according to the three-dimensional depth data and the radar perception data; The target vehicle waiting person's intention to board the vehicle is identified to obtain a result of the identification of the intention to board the vehicle, so as to control the target vehicle based on the result of the identification of the intention to board the vehicle.

2. The method for identifying the boarding intention according to claim 1, characterized in that: After the boarding intention of the target waiting person is recognized and the boarding intention recognition result is obtained, the method further includes: Determining the first position of the target waiting person according to the radar sensing data; Performing a three-dimensional posture analysis on the target waiting person according to the three-dimensional depth data to obtain a three-dimensional posture analysis result; Calculating the second position of the target waiting person according to the three-dimensional posture analysis result; Performing a position comparison on the first position and the second position to obtain a position comparison result; When the position comparison result satisfies a preset position comparison condition, it is determined that the boarding intention recognition result is accurate.

3. The method for identifying the boarding intention according to claim 1, characterized in that: The step of performing boarding intention recognition on the target waiting person to obtain a boarding intention recognition result includes: Determining whether the target waiting person is waving according to the three-dimensional depth data, and determining whether the hand of the target waiting person is facing the target vehicle according to the three-dimensional depth data; When it is determined that the target waiting person is waving, and / or when it is determined that the hand of the target waiting person is facing the target vehicle, it is determined that the target waiting person has the intention to board the vehicle.

4. The method for identifying the boarding intention according to claim 3, characterized in that: The determining whether the target waiting person is in a waving state according to the three-dimensional depth data includes: Generating a hand coordinate point sequence corresponding to the target waiting person according to the three-dimensional depth data, wherein the hand coordinate point sequence is used to describe the hand position of the target waiting person; Calculating the hand movement amplitude of the target waiting person according to the hand coordinate point sequence; When the hand movement amplitude is greater than a preset amplitude threshold, a fast Fourier transform calculation is performed based on the hand coordinate point sequence to obtain the hand swing frequency of the target waiting person; When the hand swing frequency is within a preset frequency range, it is determined that the target waiting person is in a waving state.

5. The method for identifying the boarding intention according to claim 3, characterized in that: The determining, according to the three-dimensional depth data, whether the hand of the target waiting person is facing the target vehicle comprises: Calculating a direction vector according to the three-dimensional depth data to obtain a first direction vector and a second direction vector; the first direction vector is used to represent the movement direction of the hand of the target waiting person relative to the shoulder, and the second direction vector is used to represent the movement direction of the hand of the target waiting person relative to the target vehicle; Calculating a vector angle between the first direction vector and the second direction vector; In response to the vector angle being less than a preset angle threshold, it is determined that the hand of the target waiting person is facing the target vehicle.

6. The method for identifying the boarding intention according to claim 1, characterized in that: The step of identifying a target waiting person corresponding to the target vehicle according to the three-dimensional depth data and the radar perception data includes: For each of the waiting persons in the target area, according to the radar sensing data, determining the distance change trend between each waiting person and the target vehicle, the speed direction of each waiting person, and the relative distance between each waiting person and the target vehicle within a preset first time period; A waiting person whose distance change trend is decreasing, whose speed direction is pointing to the target vehicle, and whose relative distance is less than a preset distance threshold is determined as a target waiting person.

7. The method for identifying boarding intention according to claim 1, characterized in that: The controlling the target vehicle based on the riding intention recognition result comprises: identifying obstacles in the target area according to the three-dimensional depth data and the radar perception data; Predicting the motion trajectory of each obstacle according to the three-dimensional depth data and the radar perception data to obtain the motion trajectory corresponding to each obstacle; According to the motion trajectories corresponding to the obstacles, collision risk identification is performed to obtain risk identification results; The movement of the target vehicle is controlled according to the risk identification result.

8. A vehicle boarding intention recognition device, characterized in that: include: A depth data acquisition module, used to acquire three-dimensional depth data of a target area corresponding to a target vehicle in real time, wherein the three-dimensional depth data includes posture information of each person waiting for the vehicle in the target area; A radar data acquisition module, used for acquiring radar sensing data of the target area in real time, wherein the radar sensing data includes motion status information of each person waiting for the bus in the target area; A target recognition module, used for identifying a target waiting person corresponding to the target vehicle according to the three-dimensional depth data and the radar perception data; The intention recognition module is used to recognize the boarding intention of the target waiting person and obtain the boarding intention recognition result so as to control the target vehicle based on the boarding intention recognition result.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable a computer to execute the method for identifying boarding intention as described in any one of claims 1 to 7.

10. A vehicle, characterized in that: The method comprises a memory, a processor and a program stored in the memory and running on the processor, wherein when the processor executes the program, the steps of the method for identifying the boarding intention as described in any one of claims 1 to 7 are implemented.

Images