Method, device and computer equipment for associating obstacles and trajectory estimates

Abstract

The present application relates to a method, apparatus, computer device, storage medium, and computer program product for associating obstacles and trajectory estimates. The method includes: obtaining vehicle operation data and initial trajectories of multiple obstacles; determining a vehicle operating condition based on the vehicle operation data, wherein the vehicle operating condition includes a steady state or a non-steady state; obtaining multiple trajectory estimates for the kth time based on the state data of the multiple obstacles obtained for the kth time and the multiple trajectory estimates for the kth time; and obtaining a Mahalanobis distance for the kth time based on the vehicle operating condition, the state data of the multiple obstacles, the multiple trajectory estimates for the kth time, and a preset covariance matrix, where k-1 is greater than or equal to 1; and determining an association relationship between the multiple trajectory estimates for the kth time and the multiple obstacles based on the Mahalanobis distance for the kth time and a data association algorithm. This method can improve the accuracy of the association relationship.

Description

Technical Field

[0001] The present application relates to the field of intelligent vehicle technology, and in particular to a method, apparatus, computer equipment, storage medium, and computer program product for associating obstacles and trajectory estimation. Background Art

[0002] With the development of intelligent driving technology, the demand for vehicle path planning and path tracking is becoming more and more urgent. When planning or tracking a vehicle's path, other vehicles and pedestrians in the lane can be regarded as obstacles. Trajectory estimation is the predicted value of the obstacle trajectory. Associating obstacles and trajectory estimation helps with vehicle path planning or path tracking.

[0003] Currently, the method for associating obstacles and trajectory estimates is to directly associate them through a data association algorithm. However, this method does not consider the actual operating conditions of the vehicle. Using the same data association method under different operating conditions can easily lead to large errors and low association accuracy. Summary of the Invention

[0004] Based on this, it is necessary to provide a method, device, computer equipment, computer-readable storage medium and computer program product for associating obstacle and trajectory estimation, which can improve the accuracy of the association relationship between obstacle and trajectory estimation, in order to address the technical problem that the existing method of associating obstacle and trajectory estimation has low correlation accuracy under actual vehicle operating conditions.

[0005] In a first aspect, the present application provides a method for associating obstacles and trajectory estimates. The method includes:

[0006] Obtain vehicle operation data and initial trajectories of multiple obstacles;

[0007] Determine the vehicle operating condition based on the vehicle operation data, where the vehicle operating condition includes a stable operating condition or a non-steady operating condition;

[0008] Obtaining first multiple trajectory estimates based on the state data of the multiple obstacles and the initial trajectories of the multiple obstacles obtained for the first time, and obtaining first Mahalanobis distances based on the vehicle operating condition, the state data of the multiple obstacles, the first multiple trajectory estimates, and a preset covariance matrix; and determining associations between the first multiple trajectory estimates and the multiple obstacles based on the first Mahalanobis distance and a data association algorithm;

[0009] Based on the state data of the multiple obstacles obtained at the kth time and the multiple trajectory estimates obtained at the k-1th time, the multiple trajectory estimates are obtained at the kth time, and based on the vehicle operating condition, the state data of the multiple obstacles, the multiple trajectory estimates at the kth time, and a preset covariance matrix, a Mahalanobis distance is obtained at the kth time, where k-1 is greater than or equal to 1; and based on the kth Mahalanobis distance and a data association algorithm, an association relationship between the multiple trajectory estimates at the kth time and the multiple obstacles is determined.

[0010] In one embodiment, the vehicle operating data includes: vehicle acceleration, accelerator pedal angle, and brake pedal angle; determining the vehicle operating condition based on the vehicle operating data includes:

[0011] If the vehicle acceleration is 0, the accelerator pedal angle is less than or equal to the first preset angle, and the brake pedal angle is less than or equal to the second preset angle, then the vehicle operating condition is determined to be a stable operating condition;

[0012] If the vehicle acceleration is greater than 0 and the accelerator pedal angle is greater than a first preset angle, or if the vehicle acceleration is less than 0 and the brake pedal angle is greater than a second preset angle, the vehicle operating condition is determined to be a non-stationary operating condition.

[0013] In one embodiment, obtaining the kth Mahalanobis distance based on the vehicle operating condition, state data of multiple obstacles, multiple trajectory estimates of the kth time, and a preset covariance matrix includes:

[0014] Traversing the k-th multiple trajectory estimates, for the current trajectory estimate traversed to, obtaining multiple residual vectors based on the state data of the multiple obstacles and the current trajectory estimate;

[0015] If the vehicle operating condition is a stationary condition, obtaining a Mahalanobis distance between the current trajectory estimate and each of the multiple obstacles under the stationary condition based on each of the multiple residual vectors and a preset covariance matrix;

[0016] If the vehicle operating condition is a non-stationary operating condition, adjusting each of the plurality of residual vectors according to a first preset value to obtain a plurality of adjusted residual vectors; and obtaining a Mahalanobis distance between the current trajectory estimate and each of the plurality of obstacles under the non-stationary operating condition based on the plurality of adjusted residual vectors and a preset covariance matrix;

[0017] The k-th Mahalanobis distance is obtained according to the Mahalanobis distance between each trajectory in the k-th plurality of trajectory estimates and each obstacle in the plurality of obstacles.

[0018] In one embodiment, a plurality of residual vectors are obtained based on the state data of the plurality of obstacles and the current trajectory estimate, including:

[0019] The current trajectory estimate is transformed by a matrix to obtain the transformed trajectory estimate;

[0020] Subtracting the transformed trajectory estimate from the state data of each obstacle in the state data of the plurality of obstacles to obtain a plurality of residual vectors.

[0021] In one embodiment, determining the association between the k-th multiple trajectory estimates and the multiple obstacles based on the k-th Mahalanobis distance and the data association algorithm includes:

[0022] According to the Mahalanobis distance of the kth time and the preset threshold, the cost matrix of the kth multiple trajectories estimated to multiple obstacles is obtained;

[0023] The cost matrix is input into a data association algorithm to obtain an association matrix between the k-th multiple trajectory estimates and the multiple obstacles. Based on the association matrix, an association relationship between the k-th multiple trajectory estimates and the multiple obstacles is determined.

[0024] In one embodiment, a method for obtaining status data of multiple obstacles includes:

[0025] Obtain status data of multiple obstacles detected by radar and status data of multiple obstacles detected by camera;

[0026] If the vehicle operating condition is a non-stationary operating condition, the state data of each obstacle in the state data of the multiple obstacles detected by the camera is longitudinally corrected to obtain a plurality of corrected camera state data; the plurality of corrected camera state data and the state data of the multiple obstacles detected by the radar are coordinate-converted to obtain a plurality of obstacle state data;

[0027] If the vehicle operating condition is a stable operating condition, the state data of the multiple obstacles detected by the radar and the state data of the multiple obstacles detected by the camera are subjected to coordinate conversion to obtain the state data of the multiple obstacles.

[0028] In a second aspect, the present application also provides a device for associating obstacles with trajectory estimation. The device includes:

[0029] A data acquisition module, used to obtain vehicle operation data and initial trajectories of multiple obstacles;

[0030] A working condition determination module is used to determine the vehicle working condition based on the vehicle operation data, where the vehicle working condition includes a stable working condition or a non-steady working condition;

[0031] a first association module, configured to obtain first multiple trajectory estimates based on the state data of the multiple obstacles and the initial trajectories of the multiple obstacles obtained for the first time, obtain first Mahalanobis distances based on the vehicle operating condition, the state data of the multiple obstacles, the first multiple trajectory estimates, and a preset covariance matrix, and determine an association relationship between the first multiple trajectory estimates and the multiple obstacles based on the first Mahalanobis distance and a data association algorithm;

[0032] The second association module is configured to obtain a kth multiple trajectory estimates based on the state data of the multiple obstacles obtained at the kth time and the multiple trajectory estimates obtained at the k-1th time, and obtain a kth Mahalanobis distance based on the vehicle operating condition, the state data of the multiple obstacles, the multiple trajectory estimates obtained at the kth time, and a preset covariance matrix, where k-1 is greater than or equal to 1; and determine an association relationship between the multiple trajectory estimates obtained at the kth time and the multiple obstacles based on the kth Mahalanobis distance and a data association algorithm.

[0033] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the following steps are performed:

[0034] Obtain vehicle operation data and initial trajectories of multiple obstacles;

[0035] Determine the vehicle operating condition based on the vehicle operation data, where the vehicle operating condition includes a stable operating condition or a non-steady operating condition;

[0036] Obtaining first multiple trajectory estimates based on the state data of the multiple obstacles and the initial trajectories of the multiple obstacles obtained for the first time, and obtaining first Mahalanobis distances based on the vehicle operating condition, the state data of the multiple obstacles, the first multiple trajectory estimates, and a preset covariance matrix; and determining associations between the first multiple trajectory estimates and the multiple obstacles based on the first Mahalanobis distance and a data association algorithm;

[0037] Based on the state data of the multiple obstacles obtained at the kth time and the multiple trajectory estimates obtained at the k-1th time, the multiple trajectory estimates are obtained at the kth time, and based on the vehicle operating condition, the state data of the multiple obstacles, the multiple trajectory estimates at the kth time, and a preset covariance matrix, a Mahalanobis distance is obtained at the kth time, where k-1 is greater than or equal to 1; and based on the kth Mahalanobis distance and a data association algorithm, an association relationship between the multiple trajectory estimates at the kth time and the multiple obstacles is determined.

[0038] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following steps:

[0039] Obtain vehicle operation data and initial trajectories of multiple obstacles;

[0040] Determine the vehicle operating condition based on the vehicle operation data, where the vehicle operating condition includes a stable operating condition or a non-steady operating condition;

[0041] Obtaining first multiple trajectory estimates based on the state data of the multiple obstacles and the initial trajectories of the multiple obstacles obtained for the first time, and obtaining first Mahalanobis distances based on the vehicle operating condition, the state data of the multiple obstacles, the first multiple trajectory estimates, and a preset covariance matrix; and determining associations between the first multiple trajectory estimates and the multiple obstacles based on the first Mahalanobis distance and a data association algorithm;

[0042] Based on the state data of the multiple obstacles obtained at the kth time and the multiple trajectory estimates obtained at the k-1th time, the multiple trajectory estimates are obtained at the kth time, and based on the vehicle operating condition, the state data of the multiple obstacles, the multiple trajectory estimates at the kth time, and a preset covariance matrix, a Mahalanobis distance is obtained at the kth time, where k-1 is greater than or equal to 1; and based on the kth Mahalanobis distance and a data association algorithm, an association relationship between the multiple trajectory estimates at the kth time and the multiple obstacles is determined.

[0043] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the following steps:

[0044] Obtain vehicle operation data and initial trajectories of multiple obstacles;

[0045] Determine the vehicle operating condition based on the vehicle operation data, where the vehicle operating condition includes a stable operating condition or a non-steady operating condition;

[0046] Obtaining first multiple trajectory estimates based on the state data of the multiple obstacles and the initial trajectories of the multiple obstacles obtained for the first time, and obtaining first Mahalanobis distances based on the vehicle operating condition, the state data of the multiple obstacles, the first multiple trajectory estimates, and a preset covariance matrix; and determining associations between the first multiple trajectory estimates and the multiple obstacles based on the first Mahalanobis distance and a data association algorithm;

[0047] Based on the state data of the multiple obstacles obtained at the kth time and the multiple trajectory estimates obtained at the k-1th time, the multiple trajectory estimates are obtained at the kth time, and based on the vehicle operating condition, the state data of the multiple obstacles, the multiple trajectory estimates at the kth time, and a preset covariance matrix, a Mahalanobis distance is obtained at the kth time, where k-1 is greater than or equal to 1; and based on the kth Mahalanobis distance and a data association algorithm, an association relationship between the multiple trajectory estimates at the kth time and the multiple obstacles is determined.

[0048] The above-described method, apparatus, computer device, storage medium, and computer program product for associating obstacle and trajectory estimates determine the vehicle operating condition, including either a steady or unsteady operating condition, based on acquired vehicle operation data. The vehicle operating condition can be associated with the obstacle and trajectory estimate based on the vehicle operating condition, thereby improving the accuracy of the association relationship. The method of obtaining the current trajectory estimate by combining the acquired obstacle state data and the previous trajectory estimate fully utilizes the acquired obstacle state data and trajectory estimate, making the trajectory estimate more accurate and facilitating improved association accuracy. The Mahalanobis distance and data association algorithm can effectively eliminate interference from correlations between data, further improving the accuracy of the association relationship. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A diagram illustrating an application environment of a method for associating obstacles with trajectory estimation in one embodiment;

[0050] Figure 2 Schematic diagram of a method for associating obstacles and trajectory estimation in one embodiment Figure 1 ;

[0051] Figure 3 Schematic diagram of a method for associating obstacles and trajectory estimation in one embodiment Figure 2 ;

[0052] Figure 4 Schematic diagram of a method for associating obstacles and trajectory estimation in one embodiment Figure 3 ;

[0053] Figure 5 Schematic diagram of a sub-process of S810 in one embodiment;

[0054] Figure 6 Schematic diagram of a method for associating obstacles and trajectory estimation in one embodiment Figure 4 ;

[0055] Figure 7 Schematic diagram of a method for associating obstacles and trajectory estimation in one embodiment Figure 5 ;

[0056] Figure 8 A schematic diagram of a process for fusing obstacles detected by radar and camera in one embodiment;

[0057] Figure 9 is a schematic diagram of a process for associating obstacle and trajectory estimation under different vehicle operating conditions in one embodiment;

[0058] Figure 10 is a structural block diagram of an apparatus for associating obstacle and trajectory estimation in one embodiment;

[0059] Figure 11 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0061] The method for associating obstacles and trajectory estimation provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the vehicle 104 through the network. The data storage system can store the data that the terminal 102 needs to process. The data storage system can be integrated on the terminal 102, or it can be placed on the cloud or other network servers. A sensor 106 is installed on the vehicle 104, and the sensor 106 is used to obtain the status data of multiple obstacles 108. The terminal 102 obtains the operating data of the vehicle 104 and the initial trajectories of multiple obstacles; according to the operating data of the vehicle 104, the vehicle operating condition is determined, and the vehicle operating condition includes a stable operating condition or a non-stationary operating condition; based on the status data of the multiple obstacles 108 and the initial trajectories of the multiple obstacles 108 obtained by the sensor 106 for the first time, the first multiple trajectory estimates are obtained, and based on the vehicle operating condition, the status data of the multiple obstacles 108, the first multiple trajectory estimates and the preset covariance matrix, the first Mahalanobis distance is obtained; based on the first Mahalanobis distance and the data association calculation The method determines the association between the first plurality of trajectory estimates and the plurality of obstacles 108; obtains the kth plurality of trajectory estimates based on the kth state data of the plurality of obstacles 108 acquired by the sensor 106 and the k-1th plurality of trajectory estimates; and obtains the kth Mahalanobis distance based on the vehicle operating condition, the state data of the plurality of obstacles 108, the kth plurality of trajectory estimates, and a preset covariance matrix, where k-1 is greater than or equal to 1; and determines the association between the kth plurality of trajectory estimates and the plurality of obstacles 108 based on the kth Mahalanobis distance and the data association algorithm. The terminal 102 may be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices may include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, and the like. Portable wearable devices may include smart watches, smart bracelets, head-mounted devices, and the like.

[0062] In one embodiment, Figure 2 As shown, a method for associating obstacles and trajectory estimation is provided, and the method is applied to Figure 1 Taking the terminal 102 in FIG. 1 as an example, the method includes the following steps:

[0063] S200: Acquire vehicle operation data and initial trajectories of multiple obstacles.

[0064] The vehicle operation data is data collected in real time while the vehicle is in operation. The collection method can be through sensors or a CAN (Controller Area Network) bus. Sensors that collect vehicle operation data include inertial measurement units (IMUs), gyroscopes, accelerometers, magnetometers, and pressure sensors. Obstacles are obstacles around the vehicle when it is running in the lane. Obstacles mainly include other vehicles and pedestrians. Obstacle detection methods include radar sensor detection and camera acquisition. The initial trajectory of the obstacle is the state data of the obstacle initially obtained and stored locally. The state data of the obstacle includes the obstacle type, obstacle speed, obstacle acceleration, and the coordinates of the obstacle location. The number of obstacles includes at least one. Therefore, the initial trajectories of multiple obstacles are obtained, and each of the multiple obstacles corresponds to an initial trajectory.

[0065] S400: Determine a vehicle operating condition based on vehicle operation data, where the vehicle operating condition includes a stable operating condition or a non-steady operating condition.

[0066] Among them, the stable operating condition is the vehicle state when the vehicle is running at a uniform speed, and the non-stationary operating condition is the vehicle state when the vehicle is accelerating or braking. The vehicle operation data is different when the vehicle is running at a uniform speed, accelerating or braking. For example, when the vehicle is running at a uniform speed, the vehicle speed remains unchanged, the vehicle acceleration is close to 0, the accelerator pedal angle and the brake pedal angle are close to 0. When the vehicle is accelerating, the accelerator is increased, the accelerator pedal angle is greater than 0, and the vehicle travels at a speed with an acceleration greater than 0. When the vehicle is decelerating, the brake is stepped on, the brake pedal angle is greater than 0, and the vehicle travels at a speed with an acceleration less than 0. The vehicle state can be determined as a stable operating condition or a non-stationary operating condition based on the acquired vehicle operation data.

[0067] S600: Based on the state data of the multiple obstacles and the initial trajectories of the multiple obstacles obtained for the first time, obtain a first multiple trajectory estimate, and based on the vehicle operating condition, the state data of the multiple obstacles, the first multiple trajectory estimates, and a preset covariance matrix, obtain a first Mahalanobis distance; based on the first Mahalanobis distance and a data association algorithm, determine an association relationship between the first multiple trajectory estimates and the multiple obstacles.

[0068] The trajectory estimate is a predicted value of the obstacle state data. The obstacle state data can be acquired in real time. The first acquired state data of the multiple obstacles is the first acquired state data of the multiple obstacles after the initial trajectory of the obstacle is saved. Based on the state data of each obstacle in the first acquired state data of the multiple obstacles and the initial trajectory of each obstacle in the initial trajectories of the multiple obstacles, the first multiple trajectory estimates are obtained. The number of the first trajectory estimates is the product of the number of initial trajectories and the number of obstacles in the first acquired multiple obstacles. Preferably, the present application can use a Kalman filter algorithm to obtain the first multiple trajectory estimates. The state data of each obstacle in the first acquired state data of the multiple obstacles and the initial trajectory of each obstacle in the initial trajectories of the multiple obstacles are input into the Kalman filter algorithm to obtain the first multiple trajectory estimates. It should be noted that the present application does not limit the specific algorithm type used to obtain the first trajectory estimates. The preset covariance matrix is derived from the performance parameters of the sensor. The Mahalanobis distance is a distance metric that takes into account the relationship between various characteristics and is independent of the measurement scale. It is used to represent the covariance distance of data and is an effective method for calculating the similarity between two unknown sample sets. The association between the first multiple trajectory estimates and the multiple obstacles reflects the interdependence and correlation between each trajectory estimate in the first multiple trajectory estimates and each obstacle in the first multiple obstacles. The data association algorithm is used to detect the association between each trajectory estimate in the first multiple trajectory estimates and each obstacle in the first multiple obstacles. Based on the vehicle operating condition, under different vehicle operating conditions, the Mahalanobis distance between each trajectory estimate in the first multiple trajectory estimates and each obstacle in the first multiple obstacles is obtained based on the state data of the multiple obstacles, the first multiple trajectory estimates, and a preset covariance matrix. This is the first Mahalanobis distance. Based on the first Mahalanobis distance and the data association algorithm, the association between each trajectory estimate in the first multiple trajectory estimates and each obstacle in the multiple obstacles is determined. Preferably, the data association algorithm used in this application may be a GNN (Global Nearest Neighbor) algorithm. This application does not specifically limit the type of data association algorithm used.

[0069] S800: Based on the k-th acquired state data of the multiple obstacles and the k-1-th multiple trajectory estimates, obtain the k-th multiple trajectory estimates, and based on the vehicle operating condition, the state data of the multiple obstacles, the k-th multiple trajectory estimates, and a preset covariance matrix, obtain the k-th Mahalanobis distance, where k-1 is greater than or equal to 1; and based on the k-th Mahalanobis distance and a data association algorithm, determine an association relationship between the k-th multiple trajectory estimates and the multiple obstacles.

[0070] In which, based on the state data of each obstacle in the state data of the multiple obstacles obtained for the kth time and each trajectory estimate in the multiple trajectory estimates for the k-1th time, the multiple trajectory estimates for the kth time are obtained, wherein the multiple trajectory estimates for the k-1th time are the trajectory estimates at the previous moment of the multiple trajectory estimates for the kth time, k-1 is greater than or equal to 1, and the number of the multiple trajectory estimates for the kth time is the product of the number of trajectory estimates in the multiple trajectory estimates for the k-1th time and the number of obstacles in the multiple obstacles obtained for the kth time. Preferably, the present application can use a Kalman filter algorithm to obtain the multiple trajectory estimates for the kth time, and input the state data of each obstacle in the state data of the multiple obstacles obtained for the kth time and each trajectory estimate in the multiple trajectory estimates for the k-1th time into the Kalman filter algorithm to obtain the multiple trajectory estimates for the kth time. It should be noted that the present application does not limit the specific algorithm type used to obtain the kth trajectory estimate. The association relationship between the k-th multiple trajectory estimates and the multiple obstacles reflects the interdependence and correlation between each trajectory estimate in the k-th multiple trajectory estimates and each obstacle in the k-th acquired multiple obstacles. The data association algorithm is used to detect the association relationship between each trajectory estimate in the first multiple trajectory estimates and each obstacle in the k-th acquired multiple obstacles. Based on the vehicle operating condition, under different vehicle operating conditions, the Mahalanobis distance between each trajectory estimate in the k-th multiple trajectory estimates and each obstacle in the k-th acquired multiple obstacles is obtained based on the state data of the multiple obstacles, the k-th multiple trajectory estimates, and a preset covariance matrix. This is the k-th Mahalanobis distance. Based on the k-th Mahalanobis distance and the data association algorithm, the association relationship between each trajectory estimate in the k-th multiple trajectory estimates and each obstacle in the k-th acquired multiple obstacles is determined.

[0071] In the above-mentioned method for associating obstacle and trajectory estimation, the vehicle operating condition is determined by obtaining vehicle operation data. The vehicle operating condition includes a stable operating condition or a non-stationary operating condition. The obstacle and trajectory estimation can be associated according to the vehicle operating condition, thereby improving the accuracy of the association relationship. The method of obtaining the current trajectory estimate by combining the obstacle state data obtained each time and the previous trajectory estimate fully utilizes the obstacle state data and trajectory estimate obtained each time, making the trajectory estimate more accurate and facilitating the improvement of the accuracy of the association relationship. The Mahalanobis distance and data association algorithm can effectively eliminate the interference of correlation between data, further improving the accuracy of the association relationship.

[0072] In one embodiment, Figure 3 As shown, the vehicle operation data includes: vehicle acceleration, accelerator pedal angle and brake pedal angle; based on the vehicle operation data, the vehicle operating condition is determined, including:

[0073] S420, if the vehicle acceleration is 0, the accelerator pedal angle is less than or equal to the first preset angle, and the brake pedal angle is less than or equal to the second preset angle, then determining that the vehicle operating condition is a stable operating condition;

[0074] S440: If the vehicle acceleration is greater than 0 and the accelerator pedal angle is greater than a first preset angle, or if the vehicle acceleration is less than 0 and the brake pedal angle is greater than a second preset angle, the vehicle operating condition is determined to be a non-stationary operating condition.

[0075] In this embodiment, the vehicle operation data includes: vehicle acceleration, accelerator pedal angle and brake pedal angle. If the vehicle acceleration detected by the sensor is 0, the accelerator pedal angle is less than or equal to the first preset angle and the brake pedal angle is less than or equal to the second preset angle, the vehicle operating condition is determined to be a stable operating condition, wherein the first preset angle and the second preset angle are both close to 0. When the vehicle acceleration is 0, the accelerator pedal angle and the brake pedal angle are both close to 0, the vehicle's operating speed remains unchanged, and the vehicle operating condition is determined to be a stable operating condition; when the vehicle acceleration is greater than 0 and the accelerator pedal angle is greater than the first preset angle, the vehicle is accelerating, and the vehicle operating condition is determined to be a non-stable state; when the vehicle acceleration is less than 0 and the brake pedal angle is greater than the second preset angle, the vehicle is braking, and the vehicle operating condition is determined to be a non-stable condition.

[0076] The solution of the above embodiment determines whether the vehicle operating condition is in a stable condition or a non-steady condition by judging the vehicle's acceleration, accelerator pedal angle, and brake pedal angle. Based on the determined vehicle operating condition, the accuracy of the association relationship between obstacle and trajectory estimation can be improved.

[0077] In one embodiment, Figure 4 As shown, based on the vehicle operating conditions, the state data of multiple obstacles, the k-th multiple trajectory estimates and the preset covariance matrix, the k-th Mahalanobis distance is obtained, including:

[0078] S810, traversing the k-th multiple trajectory estimates, and for the current trajectory estimate currently traversed, obtaining multiple residual vectors based on the state data of the multiple obstacles and the current trajectory estimate;

[0079] S820, if the vehicle operating condition is a stationary condition, obtaining a Mahalanobis distance between the current trajectory estimate and each of the multiple obstacles under the stationary condition based on each of the multiple residual vectors and a preset covariance matrix;

[0080] S830: If the vehicle operating condition is a non-stationary operating condition, adjusting each of the plurality of residual vectors according to a first preset value to obtain a plurality of adjusted residual vectors; and obtaining a Mahalanobis distance between the current trajectory estimate and each of the plurality of obstacles under the non-stationary operating condition based on the plurality of adjusted residual vectors and a preset covariance matrix.

[0081] S840 , obtaining a k-th Mahalanobis distance according to a Mahalanobis distance between each trajectory in the k-th plurality of trajectory estimates and each obstacle in the plurality of obstacles.

[0082] In this embodiment, the k-th multiple trajectory estimates include multiple trajectory estimates of multiple obstacles. A traversal method is used to traverse the k-th multiple trajectory estimates. For the current trajectory estimate currently traversed, the current trajectory estimate is subtracted from the state data of each obstacle in the state data of the multiple obstacles obtained for the k-th time to obtain multiple residual vectors. Each residual vector in the multiple residual vectors is the difference between the current trajectory estimate and the state data of each obstacle in the state data of the multiple obstacles obtained for the k-th time. The data types in each of the multiple residual vectors include the location coordinates of the obstacle, the obstacle velocity, and the obstacle acceleration.

[0083] The Mahalanobis distance calculation formula is: 2 (y k )=y T S -1 y, where y = y k -y k ', if the vehicle is in a stable working condition, then according to each residual vector in the multiple residual vectors and the preset covariance matrix, the Mahalanobis distance between the current trajectory estimate and each obstacle in the multiple obstacles in the stable working condition is obtained. Specifically, the state data of each obstacle in the multiple obstacles is sequentially brought into y k , bring the current trajectory estimate into y k ', obtain the residual vector y of the current trajectory estimation and the state data of the corresponding obstacle obtained for the kth time, S is the preset covariance matrix, and the residual vector y is successively brought into the Mahalanobis distance calculation formula, and the d 2 (y k ) is the Mahalanobis distance between the current trajectory estimate and the k-th obtained trajectory to each of the multiple obstacles under stable conditions.

[0084] If the vehicle operating condition is a non-stationary operating condition, each residual vector in the plurality of residual vectors is adjusted according to the first preset value to obtain a plurality of adjusted residual vectors. For example, the residual vector is recorded as y = [x, y, v x ,v y ], where x and y are the horizontal and vertical coordinates of the obstacle’s location, respectively, and v xand v y are the speeds of the obstacle in the horizontal and vertical directions respectively. The first preset value is k, and the adjusted residual vector is y t =[kx,y,v x ,v y ]. According to the multiple adjusted residual vectors and the preset covariance matrix, the Mahalanobis distance between the current trajectory and each of the multiple obstacles under non-stationary working conditions is obtained, and each adjusted residual vector y in the multiple adjusted residual vectors is converted to t Substitute y into the Mahalanobis distance calculation formula in sequence, and obtain d in sequence 2 (y k ) is the Mahalanobis distance between the current trajectory estimate and the k-th obtained trajectory to each of the multiple obstacles under non-stationary conditions.

[0085] Continue traversing the k-th multiple trajectory estimates, and sequentially obtain the Mahalanobis distance between each trajectory estimate in the k-th multiple trajectory estimates and each obstacle in the multiple obstacles, to obtain the k-th Mahalanobis distance. The number of the k-th Mahalanobis distances is the product of the number of trajectory estimates in the k-th multiple trajectory estimates and the number of obstacles in the multiple obstacles obtained for the k-th time.

[0086] The solution of the above embodiment, by traversing multiple trajectory estimates for the kth time, for the current trajectory estimate currently traversed, multiple residual vectors are obtained according to the state data of multiple obstacles and the current trajectory estimate. Under a stationary working condition, the Mahalanobis distance between the current trajectory estimate and each of the multiple obstacles under the stationary working condition is obtained according to each residual vector in the multiple residual vectors and a preset covariance matrix. Under a non-stationary working condition, each residual vector in the multiple residual vectors is adjusted according to a first preset value to obtain multiple adjusted residual vectors. According to the multiple adjusted residual vectors and the preset covariance matrix, the non-stationary working condition is obtained. The Mahalanobis distance between the current trajectory and each of the multiple obstacles is calculated, and the kth Mahalanobis distance is obtained based on the Mahalanobis distance between each trajectory in the kth multiple trajectory estimates and each of the multiple obstacles. By adjusting the residual vector under different vehicle operating conditions to obtain the Mahalanobis distance under different vehicle operating conditions, the Mahalanobis distance can be adjusted according to different vehicle operating conditions when associating obstacles and trajectory estimates, which is beneficial to improving the accuracy of the association relationship. At the same time, data association based on the obtained Mahalanobis distance can effectively eliminate the interference of correlation between data, which is beneficial to improving the accuracy of the association relationship.

[0087] In one embodiment, Figure 5 As shown, based on the state data of multiple obstacles and the current trajectory estimation, multiple residual vectors are obtained, including:

[0088] S812, performing a matrix transformation on the current trajectory estimate to obtain a transformed trajectory estimate;

[0089] S814 , subtracting the state data of each obstacle from the transformed trajectory estimate to obtain a plurality of residual vectors.

[0090] In this embodiment, the obstacle state data obtained by sensor detection is data in the sensor coordinate system, while the trajectory estimate is an estimated value obtained through prediction. A matrix transformation is required between the trajectory estimate and the obstacle state data for direct calculation. Specifically, the current trajectory estimate can be multiplied by a preset transformation matrix to obtain a transformed trajectory estimate. The state data of each of the multiple obstacles is then subtracted from the transformed trajectory estimate to obtain multiple residual vectors.

[0091] In the above-described embodiment, a matrix transformation is used to convert the current trajectory estimate into a transformed trajectory estimate. The state data of each of the multiple obstacles is then subtracted from the transformed trajectory estimate to obtain multiple residual vectors. This matrix transformation and subsequent subtraction of the current trajectory estimate from the obstacle state data allows for direct calculation of the transformed trajectory estimate and the obstacle state data, simplifying the calculation method and improving the accuracy of the association between obstacles and trajectory estimates.

[0092] In one embodiment, Figure 6 As shown, based on the k-th Mahalanobis distance and data association algorithm, the association relationship between the k-th multiple trajectory estimates and the multiple obstacles is determined, including:

[0093] S850, obtaining a cost matrix of the k-th estimated multiple trajectories to multiple obstacles based on the k-th Mahalanobis distance and a preset threshold;

[0094] S860: Input the cost matrix into a data association algorithm to obtain an association matrix between the k-th multiple trajectory estimates and the multiple obstacles, and determine an association relationship between the k-th multiple trajectory estimates and the multiple obstacles based on the association matrix.

[0095] In this embodiment, a cost matrix of the k-th estimation of multiple trajectories to multiple obstacles is obtained based on the k-th Mahalanobis distance and a preset threshold. Specifically, each Mahalanobis distance in the k-th Mahalanobis distance is compared with the preset threshold. For example, data with Mahalanobis distances less than the preset threshold in the k-th Mahalanobis distance are extracted and added to the k-th cost matrix of the k-th estimation of multiple trajectories to multiple obstacles. The cost matrix is used to find and measure the minimum cost path from the k-th multiple trajectory estimates to the multiple obstacles. The association matrix is used to represent the corresponding relationship between the k-th multiple trajectory estimates and the multiple obstacles. The data in the association matrix is 1, indicating that the association relationship between the trajectory estimate and the obstacle is associated, and the data in the association matrix is 0, indicating that the association relationship between the trajectory estimate and the obstacle is not associated. The cost matrix is input into the data association algorithm, and the association matrix between the k-th multiple trajectory estimates and the multiple obstacles is output. Through the association matrix, it can be determined whether each trajectory estimate in the k-th multiple trajectory estimates is associated with each obstacle in the multiple obstacles. For example, the data corresponding to the 1st row and the 3rd column in the association matrix is 1, indicating that the association relationship between the first trajectory estimate in the k-th multiple trajectory estimates and the third obstacle in the multiple obstacles is associated. For another example, the data corresponding to the 2nd row and the 1st column in the association matrix is 0, indicating that the association relationship between the second trajectory estimate in the k-th multiple trajectory estimates and the first obstacle in the multiple obstacles is not associated.

[0096] In the above embodiment, a cost matrix for the k-th multiple trajectory estimates to the multiple obstacles is obtained based on the k-th Mahalanobis distance and a preset threshold. This cost matrix is input into a data association algorithm to obtain an association matrix for the k-th multiple trajectory estimates to the multiple obstacles. Based on the association matrix, the association relationship between the k-th multiple trajectory estimates and the multiple obstacles is determined. This method of data association between trajectory estimates and obstacles based on the Mahalanobis distance and the data association algorithm can effectively eliminate interference caused by correlation between data and further improve the accuracy of the association relationship.

[0097] In one embodiment, Figure 7 As shown, the method for obtaining status data of multiple obstacles includes:

[0098] S520, obtaining status data of multiple obstacles detected by the radar and status data of multiple obstacles detected by the camera;

[0099] S540: If the vehicle operating condition is a non-stationary operating condition, longitudinally correcting the state data of each obstacle in the state data of the plurality of obstacles detected by the camera to obtain a plurality of corrected camera state data; and coordinate-converting the plurality of corrected camera state data and the state data of the plurality of obstacles detected by the radar to obtain a plurality of obstacle state data.

[0100] S560: If the vehicle operating condition is a stable operating condition, coordinate conversion is performed on the state data of the multiple obstacles detected by the radar and the state data of the multiple obstacles detected by the camera to obtain the state data of the multiple obstacles.

[0101] In this embodiment, status data of multiple obstacles detected by radar and status data of multiple obstacles detected by camera are obtained. The radar sensor is generally installed in front of the vehicle chassis. Regardless of whether the vehicle is in a stable working condition or an unsteady working condition, the vehicle position detected by the radar sensor changes little, and the status data of multiple obstacles detected by the radar have little error under different vehicle working conditions. The camera is generally installed on the front windshield of the vehicle. The vehicle body is relatively stable under stable working conditions, and the vehicle body pitches to a large extent under unsteady working conditions. The status data of multiple obstacles detected by the camera when the vehicle is in an unsteady working condition have larger errors than the data under stable working conditions. Since the status data of multiple obstacles detected by the radar and the status data of multiple obstacles detected by the camera use different coordinate systems, the data in different coordinate systems cannot be directly calculated. The status data of multiple obstacles detected by the radar and the status data of multiple obstacles detected by the camera can be converted into status data to obtain the status data of multiple obstacles. Specifically, if the vehicle working condition is a stable working condition, the status data of multiple obstacles detected by the radar and the status data of multiple obstacles detected by the camera are subjected to coordinate conversion to obtain the status data of multiple obstacles; if the vehicle working condition is a non-stationary working condition, the status data of each obstacle in the status data of multiple obstacles detected by the camera are longitudinally corrected to obtain multiple corrected camera status data, and the multiple corrected camera status data are converted into the status data of multiple obstacles. The camera status data and the status data of multiple obstacles detected by the radar are subjected to coordinate conversion to obtain multiple obstacle status data, wherein the longitudinal correction refers to the correction of the longitudinal rotation of the status data of multiple obstacles detected by the camera and the status data of multiple obstacles detected by the radar during the coordinate conversion. Since it is necessary to accurately identify the lateral, longitudinal and vertical offsets and rotations of the data detected by the camera relative to the data detected by the radar during the coordinate conversion, wherein the lateral, longitudinal and vertical offsets and lateral and vertical rotations do not change much when the vehicle is running, while the longitudinal rotation changes greatly, the longitudinal rotation of the status data of each obstacle in the status data of the multiple obstacles detected by the camera is added with a preset correction value to obtain multiple corrected camera status data.

[0102] The solution of the above-mentioned embodiment obtains status data of multiple obstacles detected by radar and multiple obstacles detected by camera. Under non-stationary vehicle conditions, the status data of multiple obstacles detected by camera is longitudinally corrected before coordinate transformation is performed with the status data of multiple obstacles detected by radar to obtain the status data of multiple obstacles. This longitudinal correction method accurately converts the camera data into radar data, thereby improving the accuracy of the status data of multiple obstacles. The multiple obstacle status data obtained by coordinate transformation of the status data of multiple obstacles detected by radar and the status data of multiple obstacles detected by camera include both radar data and camera data. Data correlation is performed based on data obtained from multiple sources, thereby further improving the accuracy of the acquired status data of multiple obstacles and the accuracy of the correlation relationship.

[0103] In one embodiment, a cost matrix of the k-th estimation of multiple trajectories to multiple obstacles is obtained based on the k-th Mahalanobis distance and a preset threshold, including: if the data in the k-th Mahalanobis distance is less than or equal to the preset threshold, adding the data in the k-th Mahalanobis distance that is less than or equal to the preset threshold to the cost matrix of the k-th estimation of multiple trajectories to multiple obstacles; if the data in the k-th Mahalanobis distance is greater than the preset threshold, adding a second preset value to the k-th cost matrix of the k-th estimation of multiple trajectories to multiple obstacles, the second preset value being much greater than the k-th Mahalanobis distance.

[0104] In this embodiment, the rows of the cost matrix of the k-th multiple trajectory estimates to multiple obstacles correspond to multiple trajectory estimates, and the columns of the cost matrix correspond to multiple, that is, the m-th row of the cost matrix is the m-th trajectory estimate in the k-th multiple trajectory estimates, m is greater than or equal to 1, and the n-th column of the cost matrix is the n-th obstacle in the k-th multiple trajectory estimates, n is greater than or equal to 1. If the data in the k-th Mahalanobis distance is less than or equal to a preset threshold, the data in the k-th Mahalanobis distance that is less than or equal to the preset threshold is added to the cost matrix of the k-th multiple trajectory estimates to multiple obstacles, and the position of the data in the cost matrix of the k-th multiple trajectory estimates to multiple obstacles is determined based on the trajectory estimates and the corresponding obstacles corresponding to the data; if the data in the k-th Mahalanobis distance is greater than the preset threshold, a second preset value is added to the cost matrix of the k-th multiple trajectory estimates to multiple obstacles, the second preset value being much greater than the k-th Mahalanobis distance, and the position of the data in the cost matrix of the k-th multiple trajectory estimates to multiple obstacles is determined based on the trajectory estimates and the corresponding obstacles corresponding to the data, and the second preset value being much greater than the k-th Mahalanobis distance.

[0105] In the above embodiment, the data in the k-th Mahalanobis distance is selected using a preset threshold. If the data in the k-th Mahalanobis distance is less than or equal to the preset threshold, the data in the k-th Mahalanobis distance that is less than or equal to the preset threshold is added to the k-th cost matrix of the multiple trajectories estimated to the multiple obstacles. If the data in the k-th Mahalanobis distance is greater than the preset threshold, a second preset value is added to the k-th cost matrix of the multiple trajectories estimated to the multiple obstacles, where the second preset value is much greater than the k-th Mahalanobis distance. This method of selecting the data in the k-th Mahalanobis distance using a preset threshold can perform preliminary screening of obstacles, filtering out data with excessively large Mahalanobis distance values, and facilitating improved accuracy of association relationships during data association.

[0106] To illustrate the method and effect of associating obstacles and trajectory estimation in this solution in detail, the following is a detailed example:

[0107] Obtain multiple obstacles detected by the radar and their corresponding obstacle identifiers, multiple obstacles detected by the camera and their corresponding obstacle identifiers, and multiple lane line data detected by the camera, and filter the multiple obstacles detected by the radar and the multiple obstacles detected by the camera based on the multiple lane line data, such as Figure 8 FIG2 is a schematic diagram of a method for fusing obstacles detected by radar and camera. Specifically, according to the longitudinal closest principle, two obstacles in front of the lane, two obstacles in the left lane, and two obstacles in the right lane are selected from multiple obstacles detected by the radar. Two obstacles in front of the lane, two obstacles in the left lane, and two obstacles in the right lane are selected from multiple obstacles detected by the camera. After screening, multiple obstacles are obtained. Based on the screened multiple obstacles, the status data of the multiple obstacles detected by the radar and the status data of the multiple obstacles detected by the camera are obtained. The vehicle operation data and the initial trajectory of the multiple obstacles are obtained. The vehicle operation data includes: vehicle acceleration, accelerator pedal angle, and brake pedal angle. Figure 9The figure shows a method for data association of obstacle and trajectory estimation under different vehicle operating conditions. If the vehicle acceleration is 0, the accelerator pedal angle is less than or equal to the first preset angle, and the brake pedal angle is less than or equal to the second preset angle, the vehicle operating condition is determined to be a stable operating condition; if the vehicle acceleration is greater than 0, the accelerator pedal angle is greater than the first preset angle, or the vehicle acceleration is less than 0 and the brake pedal angle is greater than the second preset angle, the vehicle operating condition is determined to be a non-stationary operating condition. The vehicle operating condition includes a stable operating condition or a non-stationary operating condition. If the vehicle operating condition is a non-stationary operating condition, the state data of each obstacle detected by the camera is converted into a state data of each obstacle. The data is longitudinally corrected to obtain multiple corrected camera state data; the multiple corrected camera state data and the state data of multiple obstacles detected by the radar are subjected to coordinate conversion to obtain multiple obstacle state data; if the vehicle working condition is a stable working condition, the state data of multiple obstacles detected by the radar and the state data of multiple obstacles detected by the camera are subjected to coordinate conversion to obtain multiple obstacle state data, based on the state data of multiple obstacles obtained for the first time and the initial trajectories of multiple obstacles, the first multiple trajectory estimates are obtained, and the multiple trajectory estimates for the kth time are traversed, and for the current trajectory estimate currently traversed, the current trajectory is converted to The estimation is performed through matrix transformation to obtain a transformed trajectory estimation; the state data of each obstacle in the state data of the multiple obstacles are respectively subtracted from the transformed trajectory estimation to obtain a plurality of residual vectors; if the vehicle working condition is a stable working condition, the Mahalanobis distance between the current trajectory estimation under the stable working condition and each obstacle in the multiple obstacles is obtained according to each residual vector in the multiple residual vectors and a preset covariance matrix; if the vehicle working condition is a non-stationary working condition, each residual vector in the multiple residual vectors is adjusted according to a first preset value to obtain a plurality of adjusted residual vectors; the current trajectory under the non-stationary working condition is obtained according to the multiple adjusted residual vectors and the preset covariance matrix. a Mahalanobis distance from the trajectory estimate to each of the multiple obstacles; obtaining a kth Mahalanobis distance based on the Mahalanobis distance from each trajectory estimate in the kth multiple trajectory estimates to each of the multiple obstacles; determining, based on the first Mahalanobis distance and the GNN algorithm, an association relationship between the first multiple trajectory estimates and the multiple obstacles; obtaining a kth multiple trajectory estimate based on the kth obtained state data of the multiple obstacles and the k-1th multiple trajectory estimates; and obtaining a kth Mahalanobis distance based on the vehicle operating condition, the state data of the multiple obstacles, the kth multiple trajectory estimates, and a preset covariance matrix, where k-1 is greater than or equal to 1;Based on the k-th Mahalanobis distance and a preset threshold, a cost matrix is obtained for each of the k-th multiple trajectory estimates and the multiple obstacles. This cost matrix is then input into the GNN algorithm to obtain an association matrix for each of the k-th multiple trajectory estimates and the multiple obstacles. Based on this association matrix, the associations between the k-th multiple trajectory estimates and the multiple obstacles are determined. Furthermore, the lifecycle of the trajectory estimates can be correlated, removing trajectory estimates that have not been associated with obstacles for a long time. New trajectory estimates are generated based on the unassociated obstacles to obtain associations between the new trajectory estimates and the obstacles. Obstacles associated with the trajectory estimates are then screened based on the closest-in-path vehicle (CIPV) principle. This method identifies the obstacles closest to the vehicle and provides them to downstream control software.

[0108] The above-mentioned method for associating obstacle and trajectory estimation determines the vehicle operating condition, which can include stable or non-stationary operating conditions, by using acquired vehicle operation data. The obstacle and trajectory estimation can be associated based on the vehicle operating condition, thereby improving the accuracy of the association relationship. The method for obtaining the current trajectory estimate by combining the acquired obstacle state data and the previous trajectory estimate makes full use of the acquired obstacle state data and trajectory estimate, making the trajectory estimate more accurate and facilitating improved association accuracy. The Mahalanobis distance and data association algorithm can effectively eliminate interference from correlations between data, further improving the accuracy of the association relationship.

[0109] The above-described method, apparatus, computer device, storage medium, and computer program product for associating obstacle and trajectory estimates determine the vehicle operating condition, including either a steady or unsteady operating condition, based on acquired vehicle operation data. The vehicle operating condition can be associated with the obstacle and trajectory estimate based on the vehicle operating condition, thereby improving the accuracy of the association relationship. The method of obtaining the current trajectory estimate by combining the acquired obstacle state data and the previous trajectory estimate fully utilizes the acquired obstacle state data and trajectory estimate, making the trajectory estimate more accurate and facilitating improved association accuracy. The Mahalanobis distance and data association algorithm can effectively eliminate interference from correlations between data, further improving the accuracy of the association relationship.

[0110] It should be understood that, although the steps in the flowcharts of the above embodiments are shown in sequence as indicated by the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be performed in other orders. Moreover, at least a portion of the steps in the flowcharts of the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily to be performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0111] Based on the same inventive concept, embodiments of the present application also provide a device for associating obstacles and trajectory estimates for implementing the aforementioned method for associating obstacles and trajectory estimates. The solution to the problem provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more embodiments of the device for associating obstacles and trajectory estimates provided below can be found in the limitations of the method for associating obstacles and trajectory estimates above and will not be repeated here.

[0112] In one embodiment, Figure 10 As shown, a device 900 for associating obstacles with trajectory estimation is provided, comprising: a data acquisition module 920, a working condition determination module 940, a first associating module 960 and a second associating module 980, wherein:

[0113] a data acquisition module 920 for acquiring vehicle operation data and initial trajectories of multiple obstacles;

[0114] The operating condition determination module 940 is used to determine the vehicle operating condition based on the vehicle operation data, where the vehicle operating condition includes a stable operating condition or a non-steady operating condition;

[0115] a first association module 960 configured to obtain first multiple trajectory estimates based on the first acquired state data of the multiple obstacles and the initial trajectories of the multiple obstacles, obtain first Mahalanobis distances based on the vehicle operating condition, the state data of the multiple obstacles, the first multiple trajectory estimates, and a preset covariance matrix, and determine associations between the first multiple trajectory estimates and the multiple obstacles based on the first Mahalanobis distances and a data association algorithm;

[0116] A second association module 980 is configured to obtain a kth multiple trajectory estimates based on the kth acquired state data of the multiple obstacles and the k-1th multiple trajectory estimates, and obtain a kth Mahalanobis distance based on the vehicle operating condition, the state data of the multiple obstacles, the kth multiple trajectory estimates, and a preset covariance matrix, where k-1 is greater than or equal to 1; and determine an association relationship between the kth multiple trajectory estimates and the multiple obstacles based on the kth Mahalanobis distance and a data association algorithm.

[0117] In one embodiment, the vehicle operation data includes: vehicle acceleration, accelerator pedal angle and brake pedal angle. According to the vehicle operation data, the vehicle operating condition is determined. The operating condition determination module 940 is also used to determine that the vehicle operating condition is a stable operating condition if the vehicle acceleration is 0, the accelerator pedal angle is less than or equal to a first preset angle and the brake pedal angle is less than or equal to a second preset angle; if the vehicle acceleration is greater than 0, the accelerator pedal angle is greater than the first preset angle, or the vehicle acceleration is less than 0 and the brake pedal angle is greater than the second preset angle, then the vehicle operating condition is determined to be a non-stable operating condition.

[0118] In one embodiment, in terms of obtaining the kth Mahalanobis distance based on the vehicle operating condition, the state data of multiple obstacles, the kth multiple trajectory estimates, and a preset covariance matrix, the second association module 980 is further configured to traverse the kth multiple trajectory estimates, and for a current trajectory estimate currently traversed, obtain multiple residual vectors based on the state data of the multiple obstacles and the current trajectory estimate; if the vehicle operating condition is a stationary condition, obtain the Mahalanobis distance between the current trajectory estimate and each of the multiple obstacles under the stationary condition based on each residual vector in the multiple residual vectors and the preset covariance matrix; if the vehicle operating condition is a non-stationary condition, adjust each of the multiple residual vectors based on a first preset value to obtain multiple adjusted residual vectors; obtain the Mahalanobis distance between the current trajectory estimate and each of the multiple obstacles under the non-stationary condition based on the multiple adjusted residual vectors and the preset covariance matrix; and obtain the kth Mahalanobis distance based on the Mahalanobis distance between each trajectory estimate in the kth multiple trajectory estimates and each of the multiple obstacles.

[0119] In one embodiment, in terms of obtaining multiple residual vectors based on the state data of multiple obstacles and the current trajectory estimate, the second association module 980 is further used to perform a matrix transformation on the current trajectory estimate to obtain a transformed trajectory estimate; and to subtract the state data of each obstacle in the state data of the multiple obstacles from the transformed trajectory estimate to obtain multiple residual vectors.

[0120] In one embodiment, in determining the association relationship between the k-th multiple trajectory estimates and the multiple obstacles based on the k-th Mahalanobis distance and the data association algorithm, the second association module 980 is further configured to obtain a cost matrix for the k-th multiple trajectory estimates to the multiple obstacles based on the k-th Mahalanobis distance and a preset threshold; input the cost matrix into the data association algorithm to obtain an association matrix for the k-th multiple trajectory estimates to the multiple obstacles; and determine the association relationship between the k-th multiple trajectory estimates and the multiple obstacles based on the association matrix.

[0121] In one embodiment, with respect to obtaining status data of multiple obstacles, the device 900 for associating obstacles with trajectory estimation is further configured to obtain status data of multiple obstacles detected by radar and status data of multiple obstacles detected by camera; if the vehicle operating condition is a non-stationary operating condition, the status data of each obstacle in the status data of the multiple obstacles detected by camera is longitudinally corrected to obtain multiple corrected camera status data; the multiple corrected camera status data and the status data of the multiple obstacles detected by radar are coordinate-converted to obtain multiple obstacle status data; if the vehicle operating condition is a stationary operating condition, the status data of the multiple obstacles detected by radar and the status data of the multiple obstacles detected by camera are coordinate-converted to obtain multiple obstacle status data.

[0122] Each module in the aforementioned apparatus for associating obstacle and trajectory estimation may be implemented in whole or in part via software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in the form of hardware, or may be stored in a memory in the computer device in the form of software, so that the processor can call and execute the corresponding operations of each module.

[0123] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 11As shown. The computer device includes a processor, a memory, an input / output (I / O) interface, and a communication interface. The processor, memory, and I / O interface are connected via a system bus, and the communication interface is connected to the system bus via the I / O interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store vehicle operation data, initial trajectories of multiple obstacles, a first multiple trajectory estimate, a k-th multiple trajectory estimate, a first Mahalanobis distance, a k-th Mahalanobis distance, an association between the first multiple trajectory estimate and multiple obstacles, and an association between the k-th multiple trajectory estimate and multiple obstacles. The I / O interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When executed by the processor, the computer program implements a method for associating obstacles with trajectory estimates.

[0124] Those skilled in the art will understand that Figure 11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0125] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0126] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0127] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0128] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0129] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0130] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0131] The above embodiments merely illustrate several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A method for associating obstacles and trajectory estimates, characterized in that The method comprises: Acquiring vehicle operation data and initial trajectories of multiple obstacles; the initial trajectories are state data of the obstacles initially obtained and stored locally; Determining a vehicle operating condition based on the vehicle operating data, wherein the vehicle operating condition includes a stable operating condition or a non-steady operating condition; Obtaining a first plurality of trajectory estimates based on the state data of the plurality of obstacles obtained for the first time and the initial trajectories of the plurality of obstacles, and obtaining a first Mahalanobis distance based on the vehicle operating condition, the state data of the plurality of obstacles, the first plurality of trajectory estimates, and a preset covariance matrix; determining an association relationship between the first plurality of trajectory estimates and the plurality of obstacles based on the first Mahalanobis distance and a data association algorithm; the first Mahalanobis distance being the Mahalanobis distance from each trajectory estimate in the first plurality of trajectory estimates to each of the plurality of obstacles obtained for the first time; the trajectory estimate being a predicted value of the obstacle state data; Obtaining a kth multiple trajectory estimates based on the state data of the multiple obstacles obtained at a kth time and the multiple trajectory estimates obtained at a k-1th time, and obtaining a kth Mahalanobis distance based on the vehicle operating condition, the state data of the multiple obstacles, the multiple trajectory estimates obtained at the kth time, and a preset covariance matrix, where k-1 is greater than or equal to 1; determining an association relationship between the multiple trajectory estimates obtained at the kth time and the multiple obstacles based on the kth Mahalanobis distance and the data association algorithm; the kth Mahalanobis distance being a Mahalanobis distance from each trajectory estimate in the multiple trajectory estimates obtained at the kth time to each obstacle in the multiple obstacles obtained at the kth time; The step of obtaining the kth Mahalanobis distance based on the vehicle operating condition, state data of multiple obstacles, multiple trajectory estimates for the kth time, and a preset covariance matrix includes: Traversing the k-th plurality of trajectory estimates, and obtaining, for a current trajectory estimate currently traversed, a plurality of residual vectors based on the state data of the plurality of obstacles and the current trajectory estimate; If the vehicle operating condition is the stationary operating condition, obtaining, according to each residual vector in the plurality of residual vectors and the preset covariance matrix, a Mahalanobis distance between the current trajectory estimate and each of the plurality of obstacles under the stationary operating condition; If the vehicle operating condition is the non-stationary operating condition, adjusting each of the plurality of residual vectors according to a first preset value to obtain a plurality of adjusted residual vectors; and obtaining a Mahalanobis distance between the current trajectory estimate and each of the plurality of obstacles under the non-stationary operating condition based on the plurality of adjusted residual vectors and the preset covariance matrix; A k-th Mahalanobis distance is obtained according to the Mahalanobis distance between each trajectory estimation in the k-th plurality of trajectory estimations and each obstacle in the plurality of obstacles.

2. The method according to claim 1, characterized in that The vehicle operation data includes: vehicle acceleration, accelerator pedal angle and brake pedal angle; Determining the vehicle operating condition based on the vehicle operating data includes: If the vehicle acceleration is 0, the accelerator pedal angle is less than or equal to a first preset angle, and the brake pedal angle is less than or equal to a second preset angle, then determining that the vehicle operating condition is a stable operating condition; If the vehicle acceleration is greater than 0 and the accelerator pedal angle is greater than the first preset angle, or if the vehicle acceleration is less than 0 and the brake pedal angle is greater than the second preset angle, it is determined that the vehicle operating condition is a non-stationary operating condition.

3. The method according to claim 1, characterized in that The obtaining of a plurality of residual vectors according to the state data of the plurality of obstacles and the current trajectory estimation comprises: Transforming the current trajectory estimate through a matrix to obtain a transformed trajectory estimate; Subtracting the transformed trajectory estimate from the state data of each obstacle in the state data of the plurality of obstacles to obtain a plurality of residual vectors.

4. The method according to any one of claims 1 to 3, characterized in that The determining, based on the k-th Mahalanobis distance and the data association algorithm, an association relationship between the k-th plurality of trajectory estimates and the plurality of obstacles comprises: Obtaining a cost matrix of the k-th multiple trajectory estimates to the multiple obstacles according to the k-th Mahalanobis distance and a preset threshold; The cost matrix is input into the data association algorithm to obtain an association matrix between the k-th multiple trajectory estimates and the multiple obstacles, and an association relationship between the k-th multiple trajectory estimates and the multiple obstacles is determined based on the association matrix.

5. The method according to claim 1, wherein The method for acquiring the status data of the plurality of obstacles includes: Obtain status data of multiple obstacles detected by radar and status data of multiple obstacles detected by camera; If the vehicle operating condition is the non-stationary operating condition, performing longitudinal correction on the state data of each obstacle in the state data of the plurality of obstacles detected by the camera to obtain a plurality of corrected camera state data; performing coordinate conversion on the plurality of corrected camera state data and the state data of the plurality of obstacles detected by the radar to obtain the plurality of obstacle state data; If the vehicle operating condition is the stable operating condition, the state data of the multiple obstacles detected by the radar and the state data of the multiple obstacles detected by the camera are subjected to coordinate conversion to obtain the state data of the multiple obstacles.

6. A device for associating obstacles and trajectory estimates, characterized in that The device comprises: A data acquisition module is used to acquire vehicle operation data and initial trajectories of multiple obstacles; the initial trajectories are state data of the obstacles initially acquired and stored locally; an operating condition determination module, configured to determine a vehicle operating condition based on the vehicle operation data, wherein the vehicle operating condition includes a stable operating condition or a non-steady operating condition; a first association module configured to obtain a first plurality of trajectory estimates based on the state data of the plurality of obstacles obtained for the first time and the initial trajectories of the plurality of obstacles, and to obtain a first Mahalanobis distance based on the vehicle operating condition, the state data of the plurality of obstacles, the first plurality of trajectory estimates, and a preset covariance matrix; and determine an association relationship between the first plurality of trajectory estimates and the plurality of obstacles based on the first Mahalanobis distance and a data association algorithm; the first Mahalanobis distance being a Mahalanobis distance from each trajectory estimate in the first plurality of trajectory estimates to each of the plurality of obstacles obtained for the first time; and the trajectory estimate being a predicted value of the state data of the obstacle; a second association module, configured to obtain a kth plurality of trajectory estimates based on the state data of the plurality of obstacles obtained at a kth time and the plurality of trajectory estimates obtained at a k-1th time, and obtain a kth Mahalanobis distance based on the vehicle operating condition, the state data of the plurality of obstacles, the kth plurality of trajectory estimates, and a preset covariance matrix, where k-1 is greater than or equal to 1; and determine an association relationship between the kth plurality of trajectory estimates and the plurality of obstacles based on the kth Mahalanobis distance and the data association algorithm; the kth Mahalanobis distance being a Mahalanobis distance from each trajectory estimate in the kth plurality of trajectory estimates to each of the plurality of obstacles obtained at the kth time; The second associating module is further configured to traverse the k-th multiple trajectory estimates, and for a current trajectory estimate currently traversed, obtain multiple residual vectors based on the state data of the multiple obstacles and the current trajectory estimate; if the vehicle operating condition is the stationary operating condition, obtain a Mahalanobis distance between the current trajectory estimate and each of the multiple obstacles under the stationary operating condition based on each residual vector in the multiple residual vectors and the preset covariance matrix; if the vehicle operating condition is the non-stationary operating condition, adjust each of the multiple residual vectors based on a first preset value to obtain multiple adjusted residual vectors; obtain a Mahalanobis distance between the current trajectory estimate and each of the multiple obstacles under the non-stationary operating condition based on the multiple adjusted residual vectors and the preset covariance matrix; and obtain a k-th Mahalanobis distance based on the Mahalanobis distance between each trajectory estimate in the k-th multiple trajectory estimates and each of the multiple obstacles.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

9. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images