Method, device, storage medium and vehicle for identifying boarding intention

Abstract

The present invention proposes a method, device, storage medium and vehicle for identifying boarding intentions, and relates to the field of unmanned driving technology. The boarding intention identification method includes: acquiring three-dimensional depth data of a target area corresponding to a target vehicle in real time, the three-dimensional depth data including posture information of each waiting person in the target area; acquiring radar perception data of the target area in real time, the radar perception data including motion status information of each waiting person in the target area; identifying the target waiting person corresponding to the target vehicle based on the three-dimensional depth data and the radar perception data; performing boarding intention identification on the target waiting person to obtain a boarding intention identification result, so as to control the target vehicle based on the boarding intention identification result. The present invention realizes automatic identification of tourists' boarding intentions and automatic control of vehicles, solving the limitation of relying on manual operation in the prior art.

Description

Technical Field

[0001] The present invention relates to the field of unmanned driving technology, and in particular to a method, device, storage medium and vehicle for recognizing boarding intention. Background Art

[0002] Currently, low-speed driverless sightseeing buses primarily operate on fixed routes. When a tourist wishes to board, they must manually identify their gestures and then manually stop the vehicle to board. This method relies on manual operation and has a low degree of automation.

[0003] The existing system relies on manual effort to identify visitors' intentions and control the vehicles, increasing labor costs and reducing the system's level of automation. Furthermore, manual identification of visitors' intentions can lead to overlooked or untimely recognition of their intentions, resulting in operational delays and a negative impact on the visitor experience. Furthermore, human energy is limited, and prolonged, high-intensity work can easily lead to fatigue, creating safety risks. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method, device, storage medium and vehicle for identifying boarding intention to solve the above technical problems.

[0005] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: a method for identifying boarding intention, comprising: acquiring three-dimensional depth data of a target area corresponding to a target vehicle in real time, the three-dimensional depth data including posture information of each waiting person in the target area; acquiring radar perception data of the target area in real time, the radar perception data including motion status information of each waiting person in the target area; identifying the target waiting person corresponding to the target vehicle based on the three-dimensional depth data and the radar perception data; performing boarding intention identification on the target waiting person to obtain a boarding intention identification result, so as to control the target vehicle based on the boarding intention identification result.

[0006] The present invention has the following beneficial effects: Based on real-time three-dimensional depth data and radar perception data, this method identifies the target vehicle and the target passenger, and then recognizes the target passenger's intention to board the vehicle, so that the target vehicle can be controlled based on the recognition result. This method can automatically identify the passenger's intention to board the vehicle, greatly reducing the incidence of passengers being overlooked or unrecognized, and overcoming the limitations of existing technologies that rely on manual operation.

[0007] On the basis of the above technical solution, the present invention can also be improved as follows.

[0008] Furthermore, after the target waiting person's intention to board the bus is identified and the boarding intention identification result is obtained, it also includes: determining the first position of the target waiting person based on the radar perception data; performing a three-dimensional posture analysis on the target waiting person based on the three-dimensional depth data to obtain a three-dimensional posture analysis result; calculating the second position of the target waiting person based on the three-dimensional posture analysis result; performing a position comparison on the first position and the second position to obtain a position comparison result; and determining that the boarding intention identification result is accurate when the position comparison result meets a preset position comparison condition.

[0009] Furthermore, the target waiting person's intention to board the bus is identified to obtain a result of the identification of the intention to board the bus, including: determining whether the target waiting person is in a waving state based on the three-dimensional depth data, and determining whether the target waiting person's hand is facing the target vehicle based on the three-dimensional depth data; when it is determined that the target waiting person is in a waving state, and / or when it is determined that the target waiting person's hand is facing the target vehicle, it is determined that the target waiting person has the intention to board the bus.

[0010] Furthermore, determining whether the target waiter is in a waving state based on the three-dimensional depth data includes: generating a hand coordinate point sequence corresponding to the target waiter based on the three-dimensional depth data, the hand coordinate point sequence being used to describe the hand position of the target waiter; calculating the hand movement amplitude of the target waiter based on the hand coordinate point sequence; when the hand movement amplitude is greater than a preset amplitude threshold, performing a fast Fourier transform calculation based on the hand coordinate point sequence to obtain the hand swing frequency of the target waiter; when the hand swing frequency is within a preset frequency range, determining that the target waiter is in a waving state.

[0011] Furthermore, determining whether the target waiter's hand is facing the target vehicle based on the three-dimensional depth data includes: performing direction vector calculation based on the three-dimensional depth data to obtain a first direction vector and a second direction vector; the first direction vector is used to characterize the movement direction of the target waiter's hand relative to the shoulder, and the second direction vector is used to characterize the movement direction of the target waiter's hand relative to the target vehicle; calculating the vector angle between the first direction vector and the second direction vector; in response to the vector angle being less than a preset angle threshold, determining that the target waiter's hand is facing the target vehicle.

[0012] Furthermore, the identifying of the target waiting person corresponding to the target vehicle based on the three-dimensional depth data and the radar perception data includes: for each waiting person in the target area, determining, based on the radar perception data, a distance change trend between each waiting person and the target vehicle, a speed direction of each waiting person, and a relative distance between each waiting person and the target vehicle within a preset first time period; and determining as a target waiting person a waiting person whose distance change trend is a decreasing distance, whose speed direction is pointing to the target vehicle, and whose relative distance is less than a preset distance threshold.

[0013] Furthermore, the controlling of the target vehicle based on the result of the recognition of the intention to ride includes: identifying obstacles in the target area according to the three-dimensional depth data and the radar perception data; predicting the motion trajectory of each obstacle according to the three-dimensional depth data and the radar perception data to obtain the motion trajectory corresponding to each obstacle; identifying collision risks according to the motion trajectory corresponding to each obstacle to obtain a risk identification result; and controlling the movement of the target vehicle according to the risk identification result.

[0014] To solve the above technical problems, this embodiment further provides a vehicle boarding intention recognition device, comprising:

[0015] A depth data acquisition module is used to acquire three-dimensional depth data of a target area corresponding to a target vehicle in real time, wherein the three-dimensional depth data includes posture information of each waiting person in the target area;

[0016] A radar data acquisition module, configured to acquire radar sensing data of the target area in real time, wherein the radar sensing data includes motion status information of each person waiting for the bus in the target area;

[0017] A target recognition module is used to identify a target waiting person corresponding to the target vehicle based on the three-dimensional depth data and the radar perception data;

[0018] The intention recognition module is used to recognize the boarding intention of the target vehicle waiting person and obtain a boarding intention recognition result so as to control the target vehicle based on the boarding intention recognition result.

[0019] In order to solve the above technical problems, this embodiment also provides a non-transitory computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable a computer to execute the method for identifying the intention to board a vehicle as described above.

[0020] To solve the above technical problems, this embodiment also provides a vehicle, including a memory, a processor, and a program stored in the memory and running on the processor. When the processor executes the program, the steps of the method for identifying the intention to board a vehicle as described above are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a flow chart of the method for identifying boarding intention of the present invention;

[0022] Figure 2 Schematic diagram of the vehicle boarding intention recognition device of the present invention. DETAILED DESCRIPTION

[0023] The principles and features of the present invention are described below. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0024] Example 1

[0025] like Figure 1 As shown, this embodiment provides a method for identifying boarding intention, including:

[0026] S101 , acquiring three-dimensional depth data of a target area corresponding to a target vehicle in real time, wherein the three-dimensional depth data includes posture information of each person waiting for the vehicle in the target area.

[0027] 3D depth data can be a depth map collected by a depth camera. A depth camera, such as a Kinect depth camera, is installed on the target vehicle. The depth camera can capture depth information in 3D space and collect 3D depth data in real time.

[0028] S102 : Acquire radar sensing data of the target area in real time, where the radar sensing data includes motion status information of each person waiting for the bus in the target area.

[0029] Radar perception data is collected by radar. A radar installed on the target vehicle collects this data in real time. Millimeter-wave radar can be used, providing precise environmental awareness, ensuring high-precision target detection, particularly in complex or adverse weather conditions. Radar perception data includes the motion status of each passenger within the target area. This information includes data such as distance and speed collected by the radar.

[0030] The target area refers to the front and side of the target vehicle, and is set according to actual usage requirements. Specifically, by adjusting the installation position and angle of the depth camera and radar, the target area can be adjusted and set to collect three-dimensional depth data and radar perception data based on the target area. It should be noted that the setting of the target area should take into account the position and size of the platform, waiting area and other areas corresponding to the target vehicle to ensure that the posture information and movement status information of the waiting people in the platform, waiting area and other areas can be fully and accurately collected.

[0031] S103: Identify a target waiting person corresponding to the target vehicle according to the three-dimensional depth data and the radar perception data.

[0032] S104: Identify the boarding intention of the target vehicle waiting person to obtain a boarding intention recognition result, so as to control the target vehicle based on the boarding intention recognition result.

[0033] In real-world applications, there may be many people waiting for the bus at a station. This means that during data collection, the 3D depth data and radar perception data obtained for the target area contain information about all of the people waiting for the bus. However, not all of these people intend to board the target vehicle. Therefore, target person identification is necessary based on the 3D depth and radar perception data. Target people are those who are initially determined to want to board the target vehicle.

[0034] This method uses real-time three-dimensional depth data and radar perception data to identify the target vehicle's corresponding passenger and then recognize the passenger's intention to board the vehicle. This recognition allows for control of the target vehicle based on the boarding intention result. This method automatically identifies tourists' boarding intentions, significantly reducing the likelihood of tourists being overlooked or unrecognized, and addresses the limitations of existing technologies that rely on manual control. This method is applicable to a variety of scenarios, such as sightseeing bus and regular bus control.

[0035] Optionally, in an embodiment, after the target waiting person's intention to board the bus is identified and the boarding intention identification result is obtained, the method further includes: determining the first position of the target waiting person based on the radar perception data; performing a three-dimensional posture analysis on the target waiting person based on the three-dimensional depth data to obtain a three-dimensional posture analysis result; calculating the second position of the target waiting person based on the three-dimensional posture analysis result; performing a position comparison on the first position and the second position to obtain a position comparison result; and determining that the boarding intention identification result is accurate when the position comparison result meets a preset position comparison condition.

[0036] To improve the accuracy of intent recognition, radar perception data and 3D depth data are cross-validated. Radar perception data collected by the radar can determine the position of the target passenger relative to the target vehicle, i.e., the first position. In this embodiment, both radar perception data and 3D depth data use the same coordinate system, with the vehicle as the origin and the horizontal plane as the XOY plane.

[0037] The three-dimensional depth data collected by the depth camera can generate a posture skeleton graph, that is, the three-dimensional posture analysis result. Based on the existing image processing technology, a posture skeleton graph containing all people can be generated, or a separate posture skeleton graph can be generated for each person. In this embodiment, it is necessary to generate a separate posture skeleton graph for each person for a frame of picture taken by the depth camera. The posture skeleton graph consists of a series of key points (such as shoulders, elbows, wrists, hips, knees and ankles, etc.) and lines connecting these points. It is an image that characterizes the corresponding posture of the key points of the human skeleton. Based on the posture skeleton graph, the position of the target waiting person relative to the target vehicle, that is, the second position, is obtained.

[0038] For each target passenger, the first and second positions corresponding to the target passenger need to be calculated. A position comparison is performed on the first and second positions to determine whether the first and second positions are consistent. A position comparison result is obtained, which can be either consistent or inconsistent. A consistent position indicates that the distance between the first and second positions is within a preset error range, while an inconsistent position indicates that the difference between the first and second positions exceeds the preset error range.

[0039] If the position comparison result satisfies the preset position comparison conditions, the boarding intention recognition result is determined to be accurate. Specifically, for a target waiting person with boarding intention, if the position comparison result corresponding to the target waiting person is position consistency, the boarding intention recognition result of the target waiting person is determined to be accurate; if the position comparison result corresponding to the target waiting person is position inconsistency, the boarding intention recognition result of the target waiting person is determined to be inaccurate. For a target waiting person without boarding intention, if the position comparison result corresponding to the target waiting person is position inconsistency, the boarding intention recognition result of the target waiting person is determined to be accurate; if the position comparison result corresponding to the target waiting person is position consistency, the boarding intention recognition result of the target waiting person is determined to be inaccurate.

[0040] In some embodiments, based on position comparison, the accuracy of the boarding intention recognition result is determined in combination with direction comparison, specifically: the position comparison result is calculated using the above method. The movement direction of the target waiting person is determined based on the radar perception data. The gesture direction of the target waiting person is determined based on the three-dimensional posture analysis result. The movement direction and the gesture direction are compared to obtain a direction comparison result. If the position comparison result satisfies a preset position comparison condition and the direction comparison result satisfies a preset direction comparison condition, the boarding intention recognition result is determined to be accurate.

[0041] Millimeter-wave radar can calculate the target person's direction of movement by analyzing the relative position changes between the target vehicle and the target person. The Kinect depth camera calculates the direction of a gesture using three-dimensional coordinates in 3D space, directly determining the direction of the gesture based on the angle between the vectors.

[0042] For each target passenger, the corresponding movement direction and gesture direction must be calculated. A direction comparison is performed on each target passenger to determine whether the movement direction and gesture direction are consistent. A direction comparison result is obtained, which is either consistent or inconsistent. A consistent direction indicates that the angle between the movement direction and the gesture direction is within a preset angular error range, while an inconsistent direction indicates that the angle between the movement direction and the gesture direction exceeds the preset angular error range.

[0043] If the position comparison result satisfies a preset position comparison condition and the direction comparison result satisfies a preset direction comparison condition, the boarding intention recognition result is determined to be accurate. Specifically, for a target waiting person with boarding intention, if the position comparison result corresponding to the target waiting person is position consistency and the direction comparison result is direction consistency, then the boarding intention recognition result of the target waiting person is determined to be accurate; otherwise, the boarding intention recognition result of the target waiting person is determined to be inaccurate. For a target waiting person without boarding intention, if the position comparison result corresponding to the target waiting person is position inconsistency and the direction comparison result is direction inconsistency, then the boarding intention recognition result of the target waiting person is determined to be accurate; otherwise, the boarding intention recognition result of the target waiting person is determined to be inaccurate.

[0044] Optionally, in an embodiment, the target waiting person's intention to board the bus is identified to obtain a boarding intention identification result, including: determining whether the target waiting person is in a waving state based on the three-dimensional depth data, and determining whether the target waiting person's hand is facing the target vehicle based on the three-dimensional depth data; when it is determined that the target waiting person is in a waving state, and / or when it is determined that the target waiting person's hand is facing the target vehicle, it is determined that the target waiting person has the intention to board the bus.

[0045] For each target passenger, gesture recognition is performed using the aforementioned method to determine whether the target passenger intends to board the vehicle. The target passenger's intention to board can be determined if one of the following conditions is met: that is, the target passenger is waving or their hand is facing the target vehicle. If the target passenger is not waving and their hand is not facing the target vehicle, the target passenger has no intention to board the vehicle. Alternatively, the target passenger's intention to board can be determined if both conditions are met: that is, the target passenger is waving and their hand is facing the target vehicle. Otherwise, the target passenger has no intention to board the vehicle.

[0046] Optionally, in an embodiment, determining whether the target waiter is in a waving state based on the three-dimensional depth data includes: generating a hand coordinate point sequence corresponding to the target waiter based on the three-dimensional depth data, the hand coordinate point sequence being used to describe the hand position of the target waiter; calculating the hand movement amplitude of the target waiter based on the hand coordinate point sequence; when the hand movement amplitude is greater than a preset amplitude threshold, performing a fast Fourier transform calculation based on the hand coordinate point sequence to obtain the hand swing frequency of the target waiter; when the hand swing frequency is within a preset frequency range, determining that the target waiter is in a waving state.

[0047] The Kinect depth camera is used to capture the 3D posture and gestures of the person waiting for the bus, leveraging its depth perception capabilities to accurately model the person's position, posture, and gestures. The hand coordinate point sequence is generated by generating a posture skeleton sequence for the target person based on the 3D depth data. The posture skeleton sequence consists of multiple consecutive posture skeleton images, with the number of posture skeleton images set based on usage requirements, for example, 30 frames. The hand coordinates are extracted from each frame of the posture skeleton sequence to generate a hand coordinate point sequence.

[0048] Based on the hand coordinate point sequence, the hand motion amplitude of the target passenger is calculated. Specifically, the hand motion amplitude is the horizontal motion amplitude of the hand, that is, the change in the hand coordinate point along the X-axis. Based on the hand coordinate point sequence, the X-coordinate difference between two adjacent hand coordinate points is first calculated, and then the average of these X-coordinate differences is calculated to obtain the hand motion amplitude.

[0049] Both the amplitude threshold and the frequency range can be set according to actual conditions. In this embodiment, the amplitude threshold is set to 0.2 meters, and the frequency range is set to 0.5Hz to 2Hz. When the amplitude of the hand movement is greater than the preset amplitude threshold, the hand swing frequency is further calculated and judged whether it is within the preset frequency range; if the hand movement amplitude is less than or equal to the preset amplitude threshold, the target waiting person is determined to be in a non-waving state. For the hand swing frequency, when the hand swing frequency is within the preset frequency range, the target waiting person is determined to be in a waving state; otherwise, the target waiting person is determined to be in a non-waving state.

[0050] Optionally, in an embodiment, determining whether the target waiter's hand is facing the target vehicle based on the three-dimensional depth data includes: performing direction vector calculation based on the three-dimensional depth data to obtain a first direction vector and a second direction vector; the first direction vector is used to characterize the movement direction of the target waiter's hand relative to the shoulder, and the second direction vector is used to characterize the movement direction of the target waiter's hand relative to the target vehicle; calculating the vector angle between the first direction vector and the second direction vector; in response to the vector angle being less than a preset angle threshold, determining that the target waiter's hand is facing the target vehicle.

[0051] When calculating the first direction vector and the second direction vector, the calculation is performed based on the posture skeleton diagram corresponding to the current moment, which refers to the latest posture skeleton diagram in time. The shoulder coordinates and hand coordinates of the target waiting person are extracted from the posture skeleton diagram. In this coordinate system, the coordinates of the vehicle corresponding to the coordinate system origin, that is, the vehicle are (0, 0, 0). In this embodiment, the angle threshold is set to 36°. For the calculation of the vector angle, the vector angle formula can be used for calculation. For each target waiting person, it is necessary to judge whether the hand is facing the target vehicle in the above manner. When the vector angle is less than the preset angle threshold, it is determined that the hand of the target waiting person is facing the target vehicle; otherwise, it is determined that the hand of the target waiting person is not facing the target vehicle.

[0052] Optionally, in an embodiment, the identifying the target waiter corresponding to the target vehicle based on the three-dimensional depth data and the radar perception data includes: for each of the waiters in the target area, determining the distance change trend between each waiter and the target vehicle, the speed direction of each waiter, and the relative distance between each waiter and the target vehicle within a preset first time period based on the radar perception data; and determining the waiter whose distance change trend is a decreasing distance, whose speed direction is pointing to the target vehicle, and whose relative distance is less than a preset distance threshold as the target waiter.

[0053] For distance, millimeter-wave radar emits electromagnetic waves and receives reflected signals to detect the distance of the target object. As the propagation time of the electromagnetic waves becomes shorter and shorter, the reflected signal returns faster, indicating that the target is closer to the radar.

[0054] For velocity, the radar analyzes the time difference between two consecutive return signals to calculate the target's velocity. If the target's velocity component points toward the vehicle (i.e., the target is approaching the vehicle), the velocity value will be negative, indicating that the target is moving toward the radar (vehicle). If the target's velocity component points away from the vehicle, the velocity value will be positive, indicating that the target is moving away from the radar.

[0055] In this embodiment, a passenger who simultaneously meets the following conditions: a decreasing distance trend, a speed direction pointing toward the target vehicle, and a relative distance less than a preset distance threshold is identified as a target passenger. In some embodiments, a target passenger may be identified based solely on the distance trend and speed direction; a passenger who simultaneously meets the following conditions: a decreasing distance trend and a speed direction pointing toward the target vehicle is identified as a target passenger.

[0056] Controlling the target vehicle based on the boarding intention recognition result includes generating a vehicle control command based on the boarding intention recognition result to control the target vehicle. Specifically, upon identifying a target waiting person intending to board the vehicle, a stop command is generated to control the target vehicle to stop, ensuring that the target vehicle stops accurately at the target waiting person's predetermined location. After the target vehicle stops, a reminder command is generated to control the vehicle's display screen or voice system to issue a reminder message, notifying the target waiting person that the target vehicle has stopped and prompting them to board or alight.

[0057] Optionally, in an embodiment, controlling the target vehicle based on the result of the riding intention recognition includes: identifying obstacles in the target area based on the three-dimensional depth data and the radar perception data; predicting the motion trajectory of each obstacle based on the three-dimensional depth data and the radar perception data to obtain the motion trajectory corresponding to each obstacle; identifying collision risks based on the motion trajectory corresponding to each obstacle to obtain a risk identification result; and controlling the movement of the target vehicle based on the risk identification result.

[0058] During driving, the target vehicle can be controlled through an automatic obstacle avoidance mechanism. The Kinect depth camera and millimeter-wave radar provide real-time perception of the vehicle's surroundings, detecting the position, distance, and motion of obstacles. Based on the three-dimensional depth information of obstacles captured by the Kinect depth camera and radar perception data from the millimeter-wave radar, a Kalman filter is used to predict pedestrian movement and determine whether there is a collision risk. If a collision risk exists, the target vehicle's path is optimized, and its speed is adjusted or an emergency stop is initiated using proportional, integral, and differential control (PID).

[0059] For example, the depth camera detects a pedestrian's position as (3, 2, 0), while the millimeter-wave radar detects an decreasing distance to an obstacle at a speed of 1.5 m / s, in the vehicle's forward direction. By fusing this data through a Kalman filter, it is predicted that the pedestrian will move into the target vehicle's path within the next two seconds. The time to collision is then calculated to be 1.2 seconds, which is less than the 2-second threshold, detecting a potential collision. Path optimization is required, and the vehicle's speed is reduced to 1 m / s through PID control for a smooth detour. If the time to collision is detected to be less than 0.6 seconds, an emergency stop is initiated to ensure the vehicle safely circumvents the obstacle and avoids a collision. During the detour, the vehicle maintains continuous environmental monitoring and dynamically adjusts its path.

[0060] In summary, a passenger approaches a target vehicle and makes a specific boarding gesture to indicate their desire to board. The Kinect depth camera captures the target passenger's posture information in real time, while the millimeter-wave radar collects real-time data of the surrounding environment. The combination of millimeter-wave radar and Kinect depth camera provides multiple redundancies, improving the accuracy and reliability of recognition. This method achieves high-precision, all-weather tourist intention recognition and automatic parking, overcoming the limitations of traditional image processing, improving the intelligence and safety of driverless sightseeing vehicles, and enhancing the tourist experience. This method enables automatic vehicle control, which is particularly suitable for low-speed driverless sightseeing vehicles, significantly improving the intelligence and automation level of low-speed driverless sightseeing vehicles, and overcoming the limitations of existing technologies that rely on manual operation.

[0061] Example 2

[0062] like Figure 2 As shown, this embodiment provides a vehicle boarding intention recognition device 200, comprising:

[0063] A depth data acquisition module 201 is used to acquire three-dimensional depth data of a target area corresponding to a target vehicle in real time, wherein the three-dimensional depth data includes posture information of each waiting person in the target area;

[0064] A radar data acquisition module 202 is configured to acquire radar sensing data of the target area in real time, wherein the radar sensing data includes motion status information of each person waiting for the bus in the target area;

[0065] A target recognition module 203 is configured to recognize a target waiting person corresponding to the target vehicle based on the three-dimensional depth data and the radar sensing data;

[0066] The intention recognition module 204 is used to recognize the boarding intention of the target vehicle waiting person and obtain a boarding intention recognition result so as to control the target vehicle based on the boarding intention recognition result.

[0067] Optionally, in an embodiment, after the intention recognition module 204, a cross-validation module is further included, wherein the cross-validation module is configured to:

[0068] Based on the radar perception data, the first position of the target waiting person is determined; based on the three-dimensional depth data, a three-dimensional posture analysis is performed on the target waiting person to obtain a three-dimensional posture analysis result; based on the three-dimensional posture analysis result, the second position of the target waiting person is calculated; a position comparison is performed on the first position and the second position to obtain a position comparison result; when the position comparison result meets the preset position comparison condition, it is determined that the boarding intention recognition result is accurate.

[0069] Optionally, in an embodiment, the intention recognition module 204 includes:

[0070] a gesture recognition unit, configured to determine whether the target waiting person is waving according to the three-dimensional depth data, and determine whether the hand of the target waiting person is facing the target vehicle according to the three-dimensional depth data;

[0071] The intention recognition unit is used to determine that the target waiting person has the intention to board the vehicle when it is determined that the target waiting person is in a waving state and / or when it is determined that the hand of the target waiting person is facing the target vehicle.

[0072] Optionally, in an embodiment, the gesture recognition unit includes:

[0073] A sequence generation subunit, configured to generate a hand coordinate point sequence corresponding to the target waiting person according to the three-dimensional depth data, wherein the hand coordinate point sequence is used to describe the hand position of the target waiting person;

[0074] an amplitude calculation subunit, configured to calculate the hand movement amplitude of the target bus waiter according to the hand coordinate point sequence;

[0075] a frequency calculation subunit, configured to perform a fast Fourier transform calculation based on the hand coordinate point sequence to obtain the hand swing frequency of the target waiting person when the hand movement amplitude is greater than a preset amplitude threshold;

[0076] The waving recognition subunit is used to determine that the target waiting person is in a waving state when the hand swing frequency is within a preset frequency range.

[0077] Optionally, in an embodiment, the gesture recognition unit further includes:

[0078] a direction vector calculation subunit, configured to calculate a direction vector based on the three-dimensional depth data to obtain a first direction vector and a second direction vector; the first direction vector is used to represent the direction of movement of the target waiting person's hand relative to the shoulder, and the second direction vector is used to represent the direction of movement of the target waiting person's hand relative to the target vehicle;

[0079] An angle calculation subunit, configured to calculate a vector angle between the first direction vector and the second direction vector;

[0080] The hand direction recognition subunit is configured to determine that the hand of the target vehicle waiter is facing the target vehicle in response to the vector angle being less than a preset angle threshold.

[0081] Optionally, in an embodiment, the target identification module 203 includes:

[0082] a data processing unit configured to determine, for each of the waiting persons in the target area, based on the radar sensing data, a distance change trend between each waiting person and the target vehicle, a speed direction of each waiting person, and a relative distance between each waiting person and the target vehicle within a preset first time period;

[0083] The target recognition unit is used to determine a waiting person whose distance change trend is decreasing, whose speed direction is pointing to the target vehicle, and whose relative distance is less than a preset distance threshold as a target waiting person.

[0084] Optionally, in an embodiment, the intention recognition module 204 further includes:

[0085] an obstacle identification unit, configured to identify obstacles within the target area based on the three-dimensional depth data and the radar perception data;

[0086] a trajectory calculation unit, configured to predict a motion trajectory of each obstacle based on the three-dimensional depth data and the radar perception data, and obtain a motion trajectory corresponding to each obstacle;

[0087] a risk identification unit, configured to identify collision risks according to the motion trajectories corresponding to the obstacles and obtain risk identification results;

[0088] A vehicle control unit is used to control the movement of the target vehicle according to the risk identification result.

[0089] Example 3

[0090] This embodiment provides a non-transitory computer-readable storage medium, which stores computer instructions. The computer instructions are used to enable a computer to execute the method for identifying boarding intention as described in Example 1.

[0091] Example 4

[0092] This embodiment provides a vehicle, including a memory, a processor, and a program stored in the memory and running on the processor. When the processor executes the program, the steps of the method for identifying boarding intention as described in Example 1 are implemented.

[0093] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0094] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A method for identifying boarding intention, characterized in that: include: Acquire three-dimensional depth data of a target area corresponding to a target vehicle in real time, wherein the three-dimensional depth data includes posture information of each person waiting for the vehicle in the target area; Acquire radar sensing data of the target area in real time, wherein the radar sensing data includes motion status information of each person waiting for the bus in the target area; Identify a target waiting person corresponding to the target vehicle according to the three-dimensional depth data and the radar perception data; performing boarding intention recognition on the target vehicle waiting person to obtain a boarding intention recognition result, so as to control the target vehicle based on the boarding intention recognition result; The step of performing boarding intention recognition on the target bus waiter to obtain a boarding intention recognition result includes: determining whether the target waiting person is waving according to the three-dimensional depth data, and determining whether the hand of the target waiting person is facing the target vehicle according to the three-dimensional depth data; When it is determined that the target waiting person is waving, and / or when it is determined that the hand of the target waiting person is facing the target vehicle, it is determined that the target waiting person has the intention to board the vehicle; The determining, based on the three-dimensional depth data, whether the target waiting person is in a waving state includes: generating a hand coordinate point sequence corresponding to the target waiting person according to the three-dimensional depth data, wherein the hand coordinate point sequence is used to describe the hand position of the target waiting person; Calculating the hand movement amplitude of the target bus waiter according to the hand coordinate point sequence; When the hand movement amplitude is greater than a preset amplitude threshold, a fast Fourier transform calculation is performed based on the hand coordinate point sequence to obtain the hand swing frequency of the target waiting person; When the hand swing frequency is within a preset frequency range, it is determined that the target waiting person is in a waving state.

2. The method for recognizing boarding intention according to claim 1, characterized in that: After the boarding intention of the target waiting person is recognized and a boarding intention recognition result is obtained, the method further includes: determining a first position of the target person waiting for the bus according to the radar sensing data; Performing a three-dimensional posture analysis on the target waiting person according to the three-dimensional depth data to obtain a three-dimensional posture analysis result; Calculating a second position of the target person waiting for the bus according to the three-dimensional posture analysis result; Performing a position comparison on the first position and the second position to obtain a position comparison result; When the position comparison result satisfies a preset position comparison condition, it is determined that the boarding intention recognition result is accurate.

3. The method for recognizing boarding intention according to claim 1, characterized in that: The determining, based on the three-dimensional depth data, whether the hand of the target waiting person is facing the target vehicle includes: Calculating a direction vector based on the three-dimensional depth data to obtain a first direction vector and a second direction vector; the first direction vector is used to represent the direction of movement of the target waiting person's hand relative to the shoulder, and the second direction vector is used to represent the direction of movement of the target waiting person's hand relative to the target vehicle; Calculating a vector angle between the first direction vector and the second direction vector; In response to the vector angle being less than a preset angle threshold, it is determined that the hand of the target waiting person is facing the target vehicle.

4. The method for recognizing boarding intention according to claim 1, characterized in that: The identifying a target waiting person corresponding to the target vehicle according to the three-dimensional depth data and the radar perception data includes: For each of the waiting persons in the target area, determining, based on the radar sensing data, a distance change trend between each waiting person and the target vehicle, a speed direction of each waiting person, and a relative distance between each waiting person and the target vehicle within a preset first time period; A waiting person whose distance change trend is decreasing, whose speed direction is pointing to the target vehicle, and whose relative distance is less than a preset distance threshold is determined as a target waiting person.

5. The method for recognizing the boarding intention according to claim 1, characterized in that: The controlling the target vehicle based on the boarding intention recognition result includes: identifying obstacles within the target area based on the three-dimensional depth data and the radar perception data; Predicting the motion trajectory of each obstacle based on the three-dimensional depth data and the radar perception data to obtain the motion trajectory corresponding to each obstacle; Perform collision risk identification based on the motion trajectory corresponding to each obstacle to obtain a risk identification result; The movement of the target vehicle is controlled according to the risk identification result.

6. A vehicle boarding intention recognition device, characterized in that: include: A depth data acquisition module is used to acquire three-dimensional depth data of a target area corresponding to a target vehicle in real time, wherein the three-dimensional depth data includes posture information of each waiting person in the target area; A radar data acquisition module, configured to acquire radar sensing data of the target area in real time, wherein the radar sensing data includes motion status information of each person waiting for the bus in the target area; A target recognition module is used to identify a target waiting person corresponding to the target vehicle based on the three-dimensional depth data and the radar perception data; an intention recognition module, configured to recognize the boarding intention of the target vehicle waiting person and obtain a boarding intention recognition result, so as to control the target vehicle based on the boarding intention recognition result; The intention recognition module includes: a gesture recognition unit, configured to determine whether the target waiting person is waving according to the three-dimensional depth data, and determine whether the hand of the target waiting person is facing the target vehicle according to the three-dimensional depth data; an intention recognition unit, configured to determine that the target waiting person has an intention to board the vehicle when it is determined that the target waiting person is waving and / or when it is determined that the target waiting person's hand is facing the target vehicle; The gesture recognition unit includes: A sequence generation subunit, configured to generate a hand coordinate point sequence corresponding to the target waiting person according to the three-dimensional depth data, wherein the hand coordinate point sequence is used to describe the hand position of the target waiting person; an amplitude calculation subunit, configured to calculate the hand movement amplitude of the target bus waiter according to the hand coordinate point sequence; a frequency calculation subunit, configured to perform a fast Fourier transform calculation based on the hand coordinate point sequence to obtain the hand swing frequency of the target waiting person when the hand movement amplitude is greater than a preset amplitude threshold; The waving recognition subunit is used to determine that the target waiting person is in a waving state when the hand swing frequency is within a preset frequency range.

7. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable a computer to execute the method for recognizing boarding intention according to any one of claims 1 to 5.

8. A vehicle, characterized in that: The method comprises a memory, a processor and a program stored in the memory and running on the processor, wherein when the processor executes the program, the steps of the method for recognizing the intention to board a vehicle as described in any one of claims 1 to 5 are implemented.

Images