Obstacle prediction method and device based on millimeter wave radar, and storage medium

Abstract

The invention discloses an obstacle prediction method and device based on a millimeter wave radar and a storage medium, and relates to the field of radar data processing.The method comprises the steps that in the vehicle driving process, the size information and speed information of a target object are detected through the millimeter wave radar; and determining a horizontal position and a relative height of the target object relative to the millimeter wave radar of the vehicle, wherein the relative height is obtained through pitching information detected by the millimeter wave radar. And according to the detected size information and speed information of the target object, and the horizontal position and relative height of the target object to the vehicle, predicting the comprehensive probability that the target object is the obstacle of the vehicle. According to the invention, the technical problems of low accuracy of obstacle detection and high requirement on system hardware caused by lack of height information for predicting obstacles in the prior art are solved.

Description

Technical Field

[0001] The present application relates to the field of radar data processing, and in particular to an obstacle prediction method, device and storage medium based on millimeter wave radar. Background Art

[0002] In advanced driver assistant systems (ADAS) or autonomous driving (AD) systems, multiple sensors are usually configured, such as radar, sonar, ultrasonic sensors, and visual sensors (such as cameras), etc., to perceive the surrounding environment and target information. By perceiving the surrounding environment and targets, target recognition can be completed, thereby effectively avoiding obstacles and increasing the comfort and safety of autonomous driving.

[0003] When some targets meet certain conditions at a height, such as being higher than the vehicle height, they will not affect the normal driving of the vehicle. However, the current obstacle avoidance method will identify them as obstacles, causing false alarms, thereby leading to unnecessary braking operations or safety accidents such as rear-end collisions. Therefore, how to effectively use sensor data to detect the above obstacles, especially suspended objects, is an urgent problem to be solved. In addition, the current identification of obstacles mostly requires multi-sensor data collection, which has high requirements for system hardware. Summary of the invention

[0004] The embodiments of the present application provide a millimeter-wave radar-based obstacle prediction method, device, and storage medium to solve the technical problems in the prior art of lacking the use of altitude information to predict obstacles, resulting in low obstacle detection accuracy and high requirements for system hardware.

[0005] To achieve the above objectives, the embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides an obstacle prediction method based on millimeter wave radar, the method comprising:

[0007] During the driving of the vehicle, the millimeter wave radar is used to detect the size information and speed information of the target object, and the horizontal position and relative height of the target object relative to the millimeter wave radar of the vehicle are determined, wherein the relative height is obtained by the pitch information detected by the millimeter wave radar;

[0008] The comprehensive probability that the target object is an obstacle to the vehicle is predicted based on the size information, speed information, horizontal position and relative height of the detected target object from the vehicle.

[0009] In combination with the first aspect, in a possible design manner, predicting the comprehensive probability that the target object is an obstacle to the vehicle based on the detected size information, speed information, and horizontal position and relative height from the vehicle of the target object includes:

[0010] Predicting a first probability that the target object is an obstacle to the vehicle based on the horizontal position;

[0011] predicting a second probability that the target object is an obstacle to the vehicle based on the size information;

[0012] predicting, by using the speed information, a third probability that the target object is an obstacle to the vehicle;

[0013] predicting a fourth probability that the target object is an obstacle to the vehicle based on the relative height;

[0014] The first probability, the second probability, the third probability, and the fourth probability are combined to determine a comprehensive probability that the target object is an obstacle to the vehicle.

[0015] In combination with the first aspect, in a possible design manner, determining the horizontal position of the target object relative to the millimeter wave radar of the vehicle includes:

[0016] Determining a driving trajectory state of the vehicle;

[0017] When the driving trajectory state indicates that the vehicle is turning, determining the longitudinal distance of the target object relative to the millimeter-wave radar, calculating the distance of the target object relative to the vehicle's motion trajectory using the turning radius, and obtaining the horizontal position of the target object relative to the millimeter-wave radar of the vehicle;

[0018] When the driving trajectory state indicates that the vehicle is moving straight, the longitudinal distance of the target object relative to the millimeter-wave radar is determined, and the distance of the target object relative to the vehicle's motion trajectory is calculated using the lateral distance of the target object relative to the millimeter-wave radar to obtain the horizontal position of the target object relative to the vehicle's millimeter-wave radar.

[0019] In combination with the first aspect, in a possible design manner, determining a relative height of a target object relative to a millimeter wave radar of the vehicle includes:

[0020] The height of the target object at the current moment is predicted by the Kalman filter algorithm in combination with the initial relative height and covariance of the target object at the previous moment to obtain the relative height.

[0021] In combination with the first aspect, in a possible design manner, predicting a fourth probability that the target object is an obstacle to the vehicle according to the relative height includes:

[0022] According to the relative height determined by the vehicle at the current moment, a fourth probability F(h) that the target object is an obstacle to the vehicle at the current moment is obtained based on the following relationship:

[0023]

[0024] Among them, a high is the first threshold of the target height, b high is the second threshold value of the target height, c is the third threshold value of the target height, d is the fourth threshold value of the target height, and h is the target height after Kalman filtering.

[0025] In combination with the first aspect, in a possible design manner, predicting a second probability that the target object is an obstacle to the vehicle according to the size information includes:

[0026] According to the target frame length in the size information, the second probability F(x len ):

[0027]

[0028] Among them, a len is the target frame length threshold parameter, x len The target frame length.

[0029] In combination with the first aspect, in a possible design manner, determining the driving trajectory state of the vehicle includes:

[0030] According to the current vehicle speed and yaw rate, the turning radius r of the vehicle at the current moment is obtained based on the following relationship:

[0031] r = vego / yawRate;

[0032] Among them, vego is the current speed of the vehicle, yawRate is the yaw rate of the vehicle;

[0033] In the case where the driving trajectory state indicates that the vehicle is turning, determining the longitudinal distance of the target object relative to the millimeter-wave radar, calculating the distance of the target object relative to the vehicle's motion trajectory using a turning radius, and obtaining a horizontal position of the target object relative to the millimeter-wave radar of the vehicle; and predicting a first probability that the target object is an obstacle to the vehicle based on the horizontal position, including:

[0034] According to the lateral distance, longitudinal distance, and turning radius of the target object relative to the millimeter wave radar, the first probability F(x, y) is obtained based on the following relationship:

[0035]

[0036] Among them, α is the first setting parameter, λ is the second setting parameter, x is the longitudinal distance, y is the lateral distance, r dif is the distance of the target object relative to the vehicle's motion trajectory;

[0037] When the driving trajectory state indicates that the vehicle is traveling straight, determining the longitudinal distance of the target object relative to the millimeter-wave radar, calculating the distance of the target object relative to the vehicle's motion trajectory using the lateral distance of the target object relative to the millimeter-wave radar, and obtaining the horizontal position of the target object relative to the millimeter-wave radar of the vehicle; and predicting a first probability that the target object is an obstacle to the vehicle according to the horizontal position, including:

[0038] According to the longitudinal distance of the target object relative to the millimeter-wave radar and the distance of the vehicle's motion trajectory, the first probability F(x, y) is obtained based on the following relationship:

[0039]

[0040] Among them, x max is the maximum detection distance of the millimeter wave radar.

[0041] In combination with the first aspect, in a possible design manner, predicting the third probability that the target object is an obstacle to the vehicle by using the speed information includes:

[0042] According to the speed information, the third probability is obtained based on the following relationship:

[0043]

[0044] Among them, p is the penalty factor, a v is the first speed threshold parameter, b v is the second speed threshold parameter, x v is the speed information.

[0045] In combination with the first aspect, in a possible design manner, combining the first probability, the second probability, the third probability, and the fourth probability to determine the comprehensive probability that the target object is an obstacle to the vehicle includes:

[0046] Determine the smallest obstacle probability x among the first probability, the second probability, and the third probability min , the minimum probability weight w is obtained based on the following relationship:

[0047] w=1-0.5*x min 1.7 ;

[0048] The mixing probability p is obtained based on the following relationship mix :

[0049]

[0050] Among them, x2 and x3 are the smallest obstacle probability x among the first probability, the second probability and the third probability. min The probability outside

[0051] The comprehensive probability is obtained based on the following relationship:

[0052] p=a w *p pitch +(1-a w )*p mix ;

[0053] Among them, p is the comprehensive probability, p pitch is the fourth probability, p mix is the mixing probability, a w The size of increases as the pitch angle measurement accuracy of the millimeter-wave radar increases.

[0054] In a second aspect, an embodiment of the present application provides an obstacle prediction device based on millimeter wave radar, comprising:

[0055] Memory for storing computer programs;

[0056] Millimeter wave radar, used to receive or send electromagnetic wave signals;

[0057] A processor is used to execute the computer program to implement the method of the first aspect and possible design manner thereof.

[0058] In a third aspect, an embodiment of the present application provides a storage medium, in which a computer program is stored, wherein the computer program is configured to execute the method of the first aspect and possible design methods thereof when running.

[0059] Compared with the prior art, the obstacle prediction method, device and storage medium based on millimeter wave radar provided in the embodiment of the present application, by first detecting the size information and speed information of the target object by millimeter wave radar during the driving process of the vehicle, and determining the horizontal position and relative height of the target object relative to the millimeter wave radar of the vehicle, wherein the relative height is obtained by the pitch information detected by the millimeter wave radar; according to the size information, speed information, and horizontal position and relative height of the detected target object from the vehicle, the comprehensive probability of the target object being an obstacle to the vehicle is predicted. In the embodiment of the present application, only the radar information collected by the 4D millimeter wave radar is needed to obtain the size information and speed information of the target object, and the horizontal position of the target object relative to the millimeter wave radar of the vehicle, that is, there is no need to collect information by multiple sensors, so the requirements for system hardware are low. And by using the 4D millimeter wave radar to obtain the relative height, and predicting the comprehensive probability that the target object is an obstacle based on the relative height, the accuracy and reliability of obstacle detection can be improved. Therefore, the present application solves the technical problems in the related art that the obstacle is not predicted by using height information, resulting in low accuracy of obstacle detection, and high requirements for system hardware.

[0060] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0062] Figure 1 A schematic diagram of an obstacle prediction method provided in an embodiment of the present application is shown;

[0063] Figure 2 A schematic diagram of the hardware structure of an obstacle prediction device based on millimeter-wave radar provided in an embodiment of the present application is shown;

[0064] Figure 3 A flowchart of an obstacle prediction method based on millimeter-wave radar provided in an embodiment of the present application is shown;

[0065] Figure 4 A probability graph of a horizontal obstacle model output when a vehicle is turning provided in an embodiment of the present application is shown;

[0066] Figure 5 A probability graph of a horizontal obstacle model output when the vehicle is traveling straight ahead provided by an embodiment of the present application is shown;

[0067] Figure 6A probability graph output by a track box obstacle model provided in an embodiment of the present application is shown;

[0068] Figure 7 A probability graph output by a speed obstacle model provided in an embodiment of the present application is shown;

[0069] Figure 8 A probability graph output by a pitch obstacle model provided in an embodiment of the present application is shown;

[0070] Fig. 9 A weight diagram of a minimum obstacle probability provided by an embodiment of the present application is shown;

[0071] Fig.10 A distribution diagram of the comprehensive probability that a billboard provided by an embodiment of the present application is an obstacle to a real vehicle is shown;

[0072] Fig.11 A software structure block diagram of an obstacle prediction device based on millimeter-wave radar provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0073] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0074] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the", "these" and the like in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether directly or indirectly. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there can be three relationships. For example, "A and / or B" can mean: A exists alone, A and B exist at the same time, and B exists alone. Usually, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0075] With the rapid development of intelligent driving technology, the performance of sensors in intelligent driving systems has gradually improved. Among them, millimeter wave radar has gradually upgraded from 3D millimeter wave radar (three-dimensional millimeter wave radar) to 4D millimeter wave radar (four-dimensional millimeter wave radar). 4D millimeter wave radar can not only provide distance, speed, and azimuth information, but also detect the pitch information of the target. How to use the pitch information detected by 4D millimeter wave radar to achieve accurate target obstacle recognition is a problem that needs to be solved urgently.

[0076] The published patent "An Effective Obstacle Target Determination Method, Device and Vehicle" (Application Number: [CN201910088527.7]) proposes an effective target obstacle determination method, which first screens the stationary targets in the lane, and then continuously obtains the target RCS value after screening the target. Calculate the target RCS change rate, and calculate the target obstacle probability based on the RCS change rate. This scheme can effectively calculate the target obstacle probability when the target RCS estimate is accurate, but when the target RCS estimate is inaccurate, the accuracy of the target obstacle probability calculation will be seriously reduced. In addition, this scheme does not utilize the target pitch information output by the 4D millimeter wave radar, and the resources are not fully utilized.

[0077] The published patent "Obstacle Detection Method, Obstacle Detection Device and Storage Medium" (application number: CN202211311141.6) proposes an obstacle detection method, which takes the point cloud image, grid map and detection image of the area to be detected, fuses the panoramic segmentation semantics of the detection image with the point cloud image, and then calculates the fused obstacle probability by matching the fused image and the grid map. This solution has the following problems: first, it requires multi-sensor input and cannot achieve the target obstacle probability output by a single radar; second, it requires large computing power and storage space, and has high hardware requirements.

[0078] The published patent "Fusion of Obstacle Detection Using Radar and Camera" (application number [201310330571.7]) divides the same field of view into an occupancy grid with multiple observation cells. The fusion module receives data from the radar and imaging systems at the same time, and projects the occupancy grid and the associated radar data onto the captured image, then extracts the features of each corresponding cell, and finally uses the main classifier to confirm whether it is an obstacle. The essence of this solution is obstacle estimation with multi-sensor input.

[0079] The published patent "A vehicle-mounted multi-obstacle classification device and method based on Bayesian classifier" (application number [201510981972.8]) uses Kalman filtering to detect obstacle targets in the video image in front of the vehicle collected by the camera, then extracts features of the detected obstacle targets, and finally uses a naive Bayesian classifier based on the obstacle target features to obtain the category of the obstacle target. This solution still requires multi-sensor input, and the system has poor real-time performance, making it difficult to implement in engineering.

[0080] As can be seen from the above, most of the existing obstacle estimation schemes require multi-sensor input and high system hardware requirements. In addition, pitch information is not used to assist obstacle identification. Therefore, when some static suspended objects such as height limit poles, gantries, toll barriers, vehicle pass poles / car barriers or overpass culverts meet certain conditions at a height, such as higher than the vehicle height, these schemes usually directly regard the suspended objects as obstacles, causing false alarms, and when some targets that are very low to the ground, such as manhole covers and speed bumps, are also identified as obstacles by these schemes, so the detection reliability is low.

[0081] Based on this, the present application embodiment provides an obstacle prediction method based on millimeter wave radar, please refer to Figure 1 , Figure 1 A schematic diagram of an obstacle prediction method provided by an embodiment of the present application is shown, Figure 1 As shown, in this method, the radar information collected by the 4D millimeter wave radar and the vehicle information are used as input to calculate the horizontal obstacle model probability (equivalent to the first probability), the track frame obstacle model probability (equivalent to the second probability), the speed obstacle model probability (equivalent to the third probability) and the pitch obstacle model probability (fourth probability) of the target object. The size information, speed information, horizontal position and relative height of the target object from the vehicle are comprehensively used to predict the comprehensive probability that the target object is an obstacle to the vehicle.

[0082] In the embodiment of the present application, only the radar information collected by the 4D millimeter wave radar is needed to obtain the size information and speed information of the target object and the horizontal position of the target object relative to the millimeter wave radar of the vehicle, that is, there is no need to collect information with multiple sensors, so the requirements for system hardware are low. In addition, the relative height is determined by using the pitch information, and the fourth probability is predicted based on the relative height. The comprehensive probability of estimating the target object being an obstacle in combination with the fourth probability can improve the accuracy and reliability of obstacle detection.

[0083] In the embodiment of the present application, the 4D millimeter wave radar (hereinafter referred to as the millimeter wave radar) transmits and receives the reflected electromagnetic wave signal. The distance, direction and speed of the target around the vehicle can be calculated by referring to the Doppler effect formula, and the target size is determined based on the target outline in the point cloud map of the millimeter wave radar, and the relative height is determined using the pitch information. In the present application, the vehicle can be replaced by a ship, an airplane, a drone, a satellite or an intelligent body, that is, the millimeter wave radar can be installed on a ship, an airplane, a drone, a satellite or an intelligent body to implement the solution provided in the embodiment of the present application. The following continues to explain the example of the vehicle being the carrier of the millimeter wave radar. During the movement of the vehicle, the millimeter wave radar obtains the height information or pitch angle information of the target object in the surrounding environment relative to the millimeter wave radar or relative to the vehicle, thereby effectively avoiding false alarms and improving the detection reliability by increasing the height perception of the surrounding environment.

[0084] Please refer to Figure 2 , Figure 2 FIG. 1 is a schematic diagram showing the hardware structure of an obstacle prediction device based on a millimeter wave radar provided in an embodiment of the present application. Figure 2 As shown, the device includes a processor 201, a millimeter wave radar 202, a memory 203, etc. These components can communicate through one or more communication buses or signal lines, such as a controller area network (CAN) bus ( Figure 2 Those skilled in the art will appreciate that Figure 2 The hardware structure shown in the figure does not constitute a limitation on the obstacle prediction device. The obstacle prediction device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0085] The millimeter wave radar 202 transmits an electromagnetic wave signal and receives a reflected wave of the electromagnetic wave signal reflected by a target object to obtain detection data (or radar information).

[0086] The memory 203 is used to store computer programs or data.

[0087] The processor 201 executes various functions and data processing by running the computer program in the memory 203. For example, based on the detection data acquired by the millimeter wave radar 202, the probability that the target object is an obstacle to the vehicle is predicted.

[0088] It should be understood that the above Figure 2 The hardware modules included in the illustrated device are merely described for exemplary purposes and do not constitute a limitation to the present application.

[0089] The obstacle prediction method based on millimeter wave radar provided in the embodiment of the present application can be applied to Figure 2The method is described in detail below.

[0090] Please refer to Figure 3 , Figure 3 A flowchart of an obstacle prediction method based on millimeter wave radar provided in an embodiment of the present application is shown. Figure 3 As shown, the method includes:

[0091] Step S301: During the driving process of the vehicle, the size information and speed information of the target object are detected by the millimeter wave radar, and the horizontal position and relative height of the target object relative to the millimeter wave radar of the vehicle are determined.

[0092] In this step, the size information, speed information, horizontal position, and relative height are determined based on the detection data obtained by the millimeter wave radar at the same time. Specifically, the size information can be determined by the target outline in the point cloud image returned by the millimeter wave radar at the current moment, the speed information and horizontal position can be calculated by the time difference between the transmitted electromagnetic wave signal and the received reflected wave at the current moment, and the relative height can be converted by the pitch information at the current moment.

[0093] Step S302: predicting a first probability that the target object is an obstacle to the vehicle according to the horizontal position.

[0094] In this step, considering the actual driving scenario, the vehicle (or vehicle) focuses on the self-lane and the adjacent and separated lane targets, that is, when the lateral distance between the target object and the vehicle is closer and the longitudinal distance is closer, the probability that the target object is an obstacle is higher. On the contrary, when the lateral distance between the target object and the vehicle is far and the longitudinal distance is far, the target object is almost impossible to cause an obstacle to the vehicle, so the probability that the target object is an obstacle should be low or 0 at this time. In other words, the horizontal position takes into account the two elements of longitudinal distance and lateral distance, and determines the size of the first probability based on the distance between the target object and the horizontal position relative to the vehicle. Specifically, if the horizontal position is far, the first probability is small, and if the horizontal position is close, the first probability is large.

[0095] In some of the embodiments, considering that a vehicle has two states when driving: turning and going straight, when the vehicle turns, it is necessary to determine the horizontal distance based on the turning radius of the vehicle. When the vehicle goes straight, it is only necessary to consider whether the vehicle will hit obstacles on both sides when passing. Therefore, the lateral distance between the vehicle and the target object is used as the horizontal distance.

[0096] Specifically, determining the driving trajectory state of the vehicle;

[0097] When the driving trajectory status indicates that the vehicle is turning, the longitudinal distance of the target object relative to the millimeter-wave radar is determined, and the distance of the target object relative to the vehicle's motion trajectory is calculated using the turning radius to obtain the horizontal position of the target object relative to the vehicle's millimeter-wave radar; the first probability that the target object is an obstacle to the vehicle is predicted based on the horizontal position.

[0098] When the driving trajectory status indicates that the vehicle is going straight, determine the longitudinal distance of the target object relative to the millimeter-wave radar, use the lateral distance of the target object relative to the vehicle to calculate the distance of the target object relative to the vehicle's motion trajectory, and obtain the horizontal position of the target object relative to the vehicle's millimeter-wave radar; predict the first probability that the target object is an obstacle to the vehicle based on the horizontal position.

[0099] The vehicle's driving trajectory state is determined by the yaw rate of the vehicle, and the unit of the yaw rate of the vehicle is rad / s. If the yaw rate of the vehicle is 0, the vehicle trajectory state indicates that the vehicle is going straight, and if the yaw rate of the vehicle is greater than 0, the vehicle trajectory state indicates that the vehicle is turning.

[0100] Illustratively, this embodiment constructs a target level obstacle model, and calculates the first probability based on the model.

[0101] First, the turning radius of the vehicle at the current moment is calculated based on equation (1):

[0102] r = vego / yawRate(1)

[0103] In equation (1), r is the turning radius of the ego vehicle at the current moment, in meters; vego is the current speed of the ego vehicle, in meters per second; and yawRate is the yaw rate of the ego vehicle, in rad per second.

[0104] Based on the turning radius, the target horizontal obstacle model is constructed, and the first probability F(x, y) is predicted based on equation (2):

[0105]

[0106] In equation (2), α is the first setting parameter, λ is the second setting parameter, x is the longitudinal distance, y is the lateral distance, and r dif is the distance of the target object relative to the trajectory of the vehicle (or the trajectory distance of the vehicle): When turning, the trajectory of the vehicle is approximately an arc, r dif Indicates the distance from the target object to the arc; when driving straight, the trajectory of the vehicle is a straight line, and the distance from the target to the straight line.

[0107] Equation (2) is the first probability predicted when the vehicle turns. If the vehicle is moving straight, equation (2) can be simplified to obtain:

[0108]

[0109] In equation (3), x max is the maximum detection distance of millimeter wave radar.

[0110] The following is an explanation of the model output results. The vehicle is turning, and the current speed of the vehicle is vego = 20m / s, and the yaw rate of the vehicle is yawRate = 0.1rad / s. Please refer to Figure 4 , Figure 4 The probability graph of the horizontal obstacle model output when the vehicle is turning is shown in an embodiment of the present application. From the probability distribution, it can be seen that the probability distribution presents a curvature, which corresponds to the driving trajectory of the vehicle. The closer the lateral distance y and the longitudinal distance x between the vehicle and the target object at each moment of the turning process, the higher the first probability, and vice versa.

[0111] The vehicle is moving straight, the yaw rate yawRate = 0 rad / s, and the maximum detection distance of the radar is x max =200m, please refer to Figure 5 , Figure 5 A probability graph of the horizontal obstacle model output when the vehicle is traveling straight ahead provided in an embodiment of the present application is shown. From the probability distribution, it can be seen that the closer the lateral distance y and the closer the longitudinal distance x between the vehicle and the target object at each moment in the straight-moving process, the higher the first probability, and vice versa.

[0112] Step S302 considers the influence of the vehicle's driving trajectory state on the identification of whether the target object is an obstacle, and outputs the first probability that the target object is an obstacle when the vehicle is turning and going straight through equations (2) and (3), respectively, with high prediction accuracy and reliability.

[0113] Step S303: predicting a second probability that the target object is an obstacle to the vehicle based on the size information.

[0114] During driving, vision has poor or no recognition effect on targets such as large vehicles stationary on the road. Due to the dense point cloud of millimeter-wave radar, the outline information (i.e., track frame) of such targets can be roughly constructed. For the vehicle, the larger the target size, the higher the probability that it is an obstacle.

[0115] Exemplarily, this embodiment constructs a track box obstacle model, and calculates the second probability based on the model. The second probability increases as the target size increases. The second probability is shown in relational equation (4):

[0116]

[0117] In equation (4), a len is the target frame length threshold parameter, x lenThe target frame length is in m. It should be noted that the target frame length can be the length of any side of the target track frame, or the length of the longest side.

[0118] The following is an explanation of the model output results. Figure 6 , Figure 6 A probability graph of a track box obstacle model output provided by an embodiment of the present application is shown. From the probability distribution, it can be seen that when the size of the box is less than 5m, the second probability increases with the increase of the target box length, until the second probability is equal to 1 after the size of the box is greater than or equal to 5m.

[0119] Step S303 obtains the size information of the target object based on the characteristic that the millimeter wave radar can return a relatively dense point cloud, and predicts the second probability that the target object is an obstacle to the vehicle based on the target frame length in the size information.

[0120] Step S304: predicting a third probability that the target object is an obstacle to the vehicle based on the speed information.

[0121] The third probability increases with the increase of speed information, so the third probability that the target object is an obstacle to the vehicle can be predicted based on the speed information. Specifically, before the speed information reaches the threshold, the greater the speed information, the greater the third probability. After the speed information reaches the threshold, the speed information increases and the third probability is equal to 1.

[0122] In some embodiments, a speed obstacle model is constructed to predict a third probability. The third probability is shown in equation (5):

[0123]

[0124] In equation (5), p is the penalty factor, a v is the first speed threshold parameter, b v is the second speed threshold parameter, x v For speed information.

[0125] This embodiment sets two speed threshold parameters, namely, threshold parameter a when the speed of the target object is low and threshold parameter b when the speed of the target object is low. v , and the threshold parameter b when the target object has a high velocity v. This is because when the target object and the vehicle are on the same horizontal plane, the probability that the moving target is an obstacle is high, but when the target changes from a stationary state to a moving state (i.e., the speed is very low), it is unreasonable to identify it as a non-obstacle, or to say that the value of the third probability is very small. Considering the speed change of the target object, when the speed of the target object is low, the third probability is equal to 0.5, and when the speed of the target object is high, the fourth probability is equal to 0.8. It should be noted that the parameter setting of the model in the embodiment of the present application is only an exemplary description and does not constitute a limitation to the present application.

[0126] The following is an explanation of the model output results. Figure 7 , Figure 7 FIG. 1 shows a probability diagram of a speed obstacle model output provided by an embodiment of the present application. From the probability distribution, it can be seen that a v =0.75m / s, b v =1.05m / s, when the speed information (speed) is less than 0.75m / s, the third probability is equal to 0.5, when the speed information is greater than 1.05m / s, the third probability is equal to 1, and when the speed information is between 0.75m / s and 1.05m / s, the third probability is proportional to the speed information.

[0127] Step S304 predicts the third probability that the target object is an obstacle in combination with the speed information detected by the millimeter wave radar, and by setting a fixed threshold parameter for the target moving at a low speed, the accuracy and reliability of obstacle prediction for the target object moving at a low speed are improved.

[0128] Step S305: predicting a fourth probability that the target object is an obstacle to the vehicle based on the relative height.

[0129] Since the millimeter wave detection radar can detect the pitch information of the target, and the pitch information can determine the relative height, the introduction of pitch information helps to improve the estimation accuracy of the obstacle probability. During actual driving, the target object is close to the height of the vehicle, the higher the probability that it is an obstacle, and vice versa. For example, when the height of static suspended objects such as height limit poles, gantries, toll barriers, vehicle pass poles / blocking poles or overpass culverts meets certain conditions, such as being higher than the height of the vehicle, it has no effect on the passage of the vehicle, so the probability that such targets are obstacles is low or 0.

[0130] The pitch information is the pitch angle of the target object relative to the millimeter wave radar, and the pitch angle can be converted into a relative height. As an example, if the pitch angle is 0, the relative height is 0; if the pitch angle is a positive number, indicating a pitch angle, then the relative height is positive, indicating that the first height is higher than the vehicle; if the pitch angle is a negative number, indicating a pitch angle, then the relative height is negative, indicating that the first height is lower than the vehicle.

[0131] Since the detected pitch information is uncertain, the error caused by only relying on the pitch information of a single frame to determine the height information of the target is large. Based on this, in some embodiments, it is considered to improve its stability and accuracy through Kalman filtering. The filtering process can be described as predicting the target height and its covariance at time k-1 based on the target height and its covariance at time k, where the target height is the relative height of the target object relative to the vehicle.

[0132] Specifically, the height of the target object at the current moment is predicted by using the Kalman filter algorithm in combination with the initial relative height and covariance of the target object at the previous moment to obtain the relative height.

[0133] As an example, the Kalman filter algorithm for obtaining relative height is described in combination with the following equations (6) to (10).

[0134]

[0135] Among them, the state transfer matrix F = 1, the process noise covariance Q k is a 1-dimensional variable.

[0136] The radar data processing module outputs each frame of detection data to the processor (or radar data processing module), which calculates the target height Z based on the target height Z at time k. k Update the current frame target height h k and their covariance.

[0137]

[0138] The measurement matrix H = 1, the measurement noise covariance R k is a 1-dimensional variable, K k is the 1-dimensional Kalman gain, and the unit matrix I=1.

[0139] Obstacle probability prediction is performed based on the relative height processed by Kalman filtering or other filtering methods, or based on the original relative height. As an example, this embodiment constructs a pitch obstacle probability model, and its predicted fourth probability F(h) is shown in relational formula (11):

[0140]

[0141] In equation (11), a high is the first threshold of the target height, b high is the second threshold of the target height, c is the third threshold of the target height, d is the fourth threshold of the target height, and h is the target height after Kalman filtering, that is, h in equation (9): k .

[0142] The following is an explanation of the model output results. Figure 8 , Figure 8 A probability graph of a pitch obstacle model output provided by an embodiment of the present application is shown. From the probability distribution, it can be seen that when the relative height is less than 0.1m or the relative height is greater than 4.4m, the fourth probability is very small and close to 0. When the relative height is between 0.1m and 0.2m, the greater the relative height, the greater the fourth probability. When the relative height is in the range of 0.2m to 3.5m, the fourth probability is equal to 1. When the relative height is between 3.5m and 4.4m, the greater the relative height, the smaller the fourth probability.

[0143] This step makes full use of the pitch information detected by the millimeter-wave radar, converts the pitch information into relative height, and estimates the probability that the target object is an obstacle based on the relative height through the pitch obstacle probability model, thereby improving the accuracy and reliability of the prediction results.

[0144] Step S306: Determine the comprehensive probability that the target object is an obstacle to the vehicle by combining the first probability, the second probability, the third probability and the fourth probability.

[0145] In this step, the comprehensive probability is related to the first probability, the second probability, the third probability and the fourth probability. Specifically, when the other three probabilities remain unchanged, if another probability is large, the comprehensive probability is large, and if another probability is small, the comprehensive probability is small.

[0146] In some embodiments, weights may be assigned to the first probability, the second probability, the third probability, and the fourth probability, and a comprehensive probability may be obtained based on the product of each probability and the weight corresponding to each probability. The weight may be a fixed weight, and the weight corresponding to each probability may be different, such as the weight of the first probability is 0.25, the weight of the second probability is 0.2, the weight of the third probability is 0.15, and the weight of the fourth probability is 0.4. In addition, the weight may be a dynamic weight, such as the size of the weight may be related to the accuracy of each probability, and the higher the accuracy, the greater the weight. Specifically, the error value between the predicted probability and the actual probability is first calculated, and the error value represents the accuracy of the probability. If the error value of the first probability is less than the error value of the second probability, it means that the accuracy of the first probability is higher than the accuracy of the second probability. Based on this, the weight of the first probability can be set higher than the weight of the second probability.

[0147] In some of the embodiments, weights are comprehensively selected in an adaptive manner, and the comprehensive probability that the target object is an obstacle is finally calculated in combination with the weights.

[0148] Specifically, in actual driving, if the relative height has the strongest correlation with passability, then the focus is on the pitch obstacle probability model, and the comprehensive probability is shown in equation (12):

[0149] p=a w *p pitch +(1-a w )*p mix (12)

[0150] In equation (12), p is the comprehensive probability, that is, the final probability that the target object calculated in the current frame is an obstacle, p pitch is the fourth probability, p mix is the mixed probability.

[0151] Then determine the weights of the probabilities output by the other three models respectively. As an example, first select the smallest obstacle probability among the three models, that is, determine the smallest obstacle probability x among the first probability, the second probability, and the third probability. min Then calculate the minimum obstacle probability x min The weight w is calculated using equation (13):

[0152] w=1-0.5*x min 1.7 (13)

[0153] In equation (13), x min is the minimum value among the first probability, the second probability, and the third probability, and w is the probability corresponding to the minimum value.

[0154] Please refer to Fig. 9 , Fig. 9 FIG. 4 shows a weight diagram of a minimum obstacle probability provided by an embodiment of the present application. Fig. 9 As can be seen from the figure, the minimum obstacle probability x min The relationship between the weight w of the minimum obstacle probability and x is as follows: min The larger , the smaller w.

[0155] Then based on expression (14), we get the mixing probability p mix :

[0156]

[0157] Among them, x2 and x3 are the smallest obstacle probability x among the first probability, the second probability and the third probability. min Probability outside of .

[0158] In some embodiments, a w The size of increases with the increase of the pitch angle measurement accuracy of the millimeter-wave radar.

[0159] Please refer to Fig.10 , Fig.10A distribution diagram of the comprehensive probability that a billboard provided by an embodiment of the present application is an obstacle to a real vehicle is shown. Fig.10 In the figure, the horizontal axis is the number of frames, and the vertical axis is the percentage of the comprehensive probability. During the driving process of the vehicle, the millimeter wave radar obtains detection data frame by frame, and the radar data processing module predicts the comprehensive probability that the billboard is an obstacle frame by frame. It can be seen that the comprehensive probability decreases frame by frame and reaches a lower value at the 5th frame. Based on this value, it can be determined that the billboard is not an obstacle, indicating that the comprehensive probability determined by this method has a high accuracy in predicting the target object.

[0160] Please refer to Fig.11 , Fig.11 FIG. 4 shows a software structure block diagram of an obstacle prediction device based on millimeter wave radar provided in an embodiment of the present application. Fig.11 As shown, the device comprises:

[0161] The acquisition module 1101 is used to detect the size information and speed information of the target object through the millimeter wave radar during the vehicle's driving process, and to determine the horizontal position and relative height of the target object relative to the millimeter wave radar of the vehicle.

[0162] The first probability prediction module 1102 is used to predict the first probability that the target object is an obstacle to the vehicle according to the horizontal position.

[0163] The second probability prediction module 1103 is used to predict the second probability that the target object is an obstacle to the vehicle according to the size information.

[0164] The third probability prediction module 1104 is used to predict the third probability that the target object is an obstacle to the vehicle based on the speed information.

[0165] The fourth probability prediction module 1105 is used to predict the fourth probability that the target object is an obstacle to the vehicle according to the relative height.

[0166] The comprehensive probability prediction module 1106 is used to determine the comprehensive probability that the target object is an obstacle to the vehicle by combining the first probability, the second probability, the third probability and the fourth probability.

[0167] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0168] In addition, in combination with the method provided in the above embodiment, a storage medium may be provided in this embodiment to implement the method. The storage medium stores a computer program; when the computer program is executed by a processor, any obstacle prediction method based on millimeter wave radar in the above embodiment is implemented.

[0169] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0170] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0171] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0172] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. An obstacle prediction method based on millimeter wave radar, characterized in that: The method comprises: During the driving of the vehicle, the millimeter wave radar is used to detect the size information and speed information of the target object, and the horizontal position and relative height of the target object relative to the millimeter wave radar of the vehicle are determined, wherein the relative height is obtained by the pitch information detected by the millimeter wave radar; The comprehensive probability that the target object is an obstacle to the vehicle is predicted based on the size information, speed information, horizontal position and relative height of the detected target object from the vehicle.

2. The obstacle prediction method based on millimeter wave radar according to claim 1, characterized in that: The method of predicting the comprehensive probability that the target object is an obstacle to the vehicle based on the size information, speed information, horizontal position and relative height of the detected target object from the vehicle comprises: Predicting a first probability that the target object is an obstacle to the vehicle based on the horizontal position; predicting a second probability that the target object is an obstacle to the vehicle based on the size information; Predicting a third probability that the target object is an obstacle to the vehicle based on the speed information; predicting a fourth probability that the target object is an obstacle to the vehicle based on the relative height; The first probability, the second probability, the third probability, and the fourth probability are combined to determine a comprehensive probability that the target object is an obstacle to the vehicle.

3. The obstacle prediction method based on millimeter wave radar according to claim 2, characterized in that: The determining the horizontal position of the target object relative to the millimeter wave radar of the vehicle comprises: Determining a driving trajectory state of the vehicle; When the driving trajectory state indicates that the vehicle is turning, determining the longitudinal distance of the target object relative to the millimeter-wave radar, calculating the distance of the target object relative to the vehicle's motion trajectory using the turning radius, and obtaining the horizontal position of the target object relative to the millimeter-wave radar of the vehicle; When the driving trajectory state indicates that the vehicle is moving straight, the longitudinal distance of the target object relative to the millimeter-wave radar is determined, and the distance of the target object relative to the vehicle's motion trajectory is calculated using the lateral distance of the target object relative to the millimeter-wave radar to obtain the horizontal position of the target object relative to the vehicle's millimeter-wave radar.

4. The obstacle prediction method based on millimeter wave radar according to claim 1, characterized in that: Determining a relative height of a target object relative to a millimeter wave radar of the vehicle, comprising: The height of the target object at the current moment is predicted by the Kalman filter algorithm in combination with the initial relative height and covariance of the target object at the previous moment to obtain the relative height.

5. The obstacle prediction method based on millimeter wave radar according to claim 2, characterized in that: The predicting, according to the relative height, a fourth probability that the target object is an obstacle to the vehicle comprises: According to the relative height determined by the vehicle at the current moment, a fourth probability F(h) that the target object is an obstacle to the vehicle at the current moment is obtained based on the following relationship: Among them, a high is the first threshold of the target height, b high is the second threshold value of the target height, c is the third threshold value of the target height, d is the fourth threshold value of the target height, and h is the target height after Kalman filtering.

6. The obstacle prediction method based on millimeter wave radar according to claim 2, characterized in that: The predicting, according to the size information, a second probability that the target object is an obstacle to the vehicle comprises: According to the target frame length in the size information, the second probability F(x len ): Among them, a len is the target frame length threshold parameter, x len The target frame length.

7. The obstacle prediction method based on millimeter wave radar according to claim 3, characterized in that: The determining the driving track state of the vehicle includes: According to the current vehicle speed and yaw rate, the turning radius r of the vehicle at the current moment is obtained based on the following relationship: r = vego / yawRate; Among them, vego is the current speed of the vehicle, yawRate is the yaw rate of the vehicle; In the case where the driving trajectory state indicates that the vehicle is turning, determining the longitudinal distance of the target object relative to the millimeter-wave radar, calculating the distance of the target object relative to the vehicle's motion trajectory using a turning radius, and obtaining a horizontal position of the target object relative to the millimeter-wave radar of the vehicle; and predicting a first probability that the target object is an obstacle to the vehicle based on the horizontal position, including: According to the lateral distance, longitudinal distance, and turning radius of the target object relative to the millimeter wave radar, the first probability F(x, y) is obtained based on the following relationship: Among them, α is the first setting parameter, λ is the second setting parameter, x is the longitudinal distance, y is the lateral distance, r dif is the distance of the target object relative to the vehicle's motion trajectory; When the driving trajectory state indicates that the vehicle is traveling straight, determining the longitudinal distance of the target object relative to the millimeter-wave radar, calculating the distance of the target object relative to the vehicle's motion trajectory using the lateral distance of the target object relative to the millimeter-wave radar, and obtaining the horizontal position of the target object relative to the millimeter-wave radar of the vehicle; and predicting a first probability that the target object is an obstacle to the vehicle according to the horizontal position, including: According to the longitudinal distance of the target object relative to the millimeter-wave radar and the distance of the vehicle's motion trajectory, the first probability F(x, y) is obtained based on the following relationship: Among them, x max is the maximum detection distance of the millimeter wave radar.

8. The obstacle prediction method based on millimeter wave radar according to claim 2, characterized in that: The predicting, by using the speed information, a third probability that the target object is an obstacle to the vehicle comprises: According to the speed information, the third probability is obtained based on the following relationship: Among them, p is the penalty factor, a v is the first speed threshold parameter, b v is the second speed threshold parameter, x v is the speed information.

9. The obstacle prediction method based on millimeter wave radar according to claim 2, characterized in that: The combining the first probability, the second probability, the third probability, and the fourth probability to determine the comprehensive probability that the target object is an obstacle to the vehicle includes: Determine the smallest obstacle probability x among the first probability, the second probability, and the third probability min , the minimum probability weight w is obtained based on the following relationship: w=1-0.5*x min 1.7 ; The mixing probability p is obtained based on the following relationship mix : Among them, x2 and x3 are the smallest obstacle probability x among the first probability, the second probability and the third probability. min The probability outside The comprehensive probability is obtained based on the following relationship: p=a w *p pitch +(1-a w )*p mix ; Among them, p is the comprehensive probability, p pitch is the fourth probability, p mix is the mixing probability, a w The size of increases as the pitch angle measurement accuracy of the millimeter-wave radar increases.

10. An obstacle prediction device based on millimeter wave radar, characterized in that: The device comprises: Memory for storing computer programs; Millimeter wave radar, used to receive or send electromagnetic wave signals; A processor, configured to execute the computer program to implement the obstacle prediction method based on millimeter-wave radar as described in any one of claims 1 to 9.

11. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute the obstacle prediction method based on millimeter-wave radar according to any one of claims 1 to 9 when running.

Images