Data processing method, apparatus, device, storage medium, and program product

Abstract

The application relates to a data processing method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: for a current frame, determining a rotation vector and a translation matrix corresponding to the current frame; based on motion information of a vehicle in a previous frame, motion information of at least one obstacle in point cloud data collected in the previous frame, and the rotation vector and the translation matrix corresponding to the current frame, obtaining predicted position information of the obstacle appearing in the previous frame in the current frame; based on actual position information of the obstacle appearing in the current frame and the predicted position information of the obstacle appearing in the previous frame in the current frame, determining whether there is an unattended obstacle among the obstacles appearing in the previous frame; and in the case of existence, deleting point cloud data corresponding to the unattended obstacle in the point cloud data collected in the previous frame. The method can timely eliminate the point cloud data of the unattended obstacle, and reduces the processing amount of data in the perception fusion system.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data processing method, apparatus, computer equipment, storage medium, and computer program product. Background Technology

[0002] In traditional technology, the data acquisition device in the vehicle collects point cloud data of obstacles and then directly transmits the collected point cloud data to the vehicle's perception fusion system for fusion processing.

[0003] However, the amount of data that a vehicle's perception fusion system can process is limited, resulting in low efficiency in processing point cloud data. Summary of the Invention

[0004] Therefore, it is necessary to provide a data processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve data processing efficiency in response to the above-mentioned technical problems.

[0005] Firstly, this application provides a data processing method. Applied to a vehicle, the vehicle includes an onboard 4D millimeter-wave radar, which is used to collect point cloud data during autonomous driving. The method includes:

[0006] For the current frame, the corresponding rotation vector and translation matrix are determined based on the vehicle's position and heading angle information in the current frame, the vehicle's position and heading angle information in the previous frame, and the vehicle's heading angle information in the initial frame.

[0007] Obtain the motion information of the vehicle in the previous frame, and the motion information of at least one obstacle in the point cloud data collected in the previous frame;

[0008] Based on the vehicle's motion information in the previous frame, the motion information of at least one obstacle in the point cloud data collected in the previous frame, and the corresponding rotation vector and translation matrix in the current frame, the predicted position information of the obstacle that appeared in the previous frame in the current frame is obtained.

[0009] Obtain the actual location information of at least one obstacle in the point cloud data collected in the current frame;

[0010] Based on the actual location information of obstacles appearing in the current frame and the predicted location information of obstacles appearing in the previous frame in the current frame, determine whether there are any non-interested obstacles among the obstacles appearing in the previous frame.

[0011] If the obstacle exists, delete the corresponding point cloud data of the non-interested obstacle from the point cloud data collected in the previous frame.

[0012] In one embodiment, the first relative heading angle information and the second relative heading angle information of the vehicle are obtained based on the heading angle information of the vehicle in the initial frame, the heading angle information in the previous frame and the heading angle information in the current frame.

[0013] Based on the vehicle's position information in the current frame and its position information in the previous frame, obtain the vehicle's relative position information;

[0014] Based on the first relative heading angle information, obtain the corresponding rotation vector for the current frame;

[0015] Based on the second relative heading angle information and the relative position information, the translation matrix corresponding to the current frame is obtained.

[0016] In one embodiment, obtaining the predicted position information of the obstacle appearing in the previous frame in the current frame based on the vehicle's motion information in the previous frame, the motion information of at least one obstacle in the point cloud data collected in the previous frame, and the corresponding rotation vector and translation matrix in the current frame includes:

[0017] Based on the vehicle's motion information in the previous frame, the motion information of at least one obstacle in the point cloud data collected in the previous frame, and the corresponding rotation vector in the current frame, the absolute velocity and absolute acceleration of the obstacle in the previous frame in the vehicle coordinate system of the current frame are obtained; the vehicle coordinate system is constructed based on the vehicle's position information in each frame, and there is a corresponding vehicle coordinate system for each frame.

[0018] Based on the absolute velocity and absolute acceleration of the obstacle in the previous frame in the vehicle coordinate system of the current frame, and the corresponding rotation vector and translation matrix of the current frame, the predicted position information of the obstacle that appeared in the previous frame in the current frame is obtained.

[0019] In one embodiment, the vehicle's motion information in the previous frame includes the vehicle's velocity and acceleration in the vehicle's coordinate system of the previous frame; the motion information of at least one obstacle in the point cloud data collected in the previous frame includes the absolute velocity and absolute acceleration of the obstacle in the vehicle's coordinate system of the previous frame; obtaining the absolute velocity and absolute acceleration of the obstacle in the vehicle's coordinate system of the current frame based on the vehicle's motion information in the previous frame, the motion information of at least one obstacle in the point cloud data collected in the previous frame, and the corresponding rotation vector of the current frame includes:

[0020] Based on the absolute velocity of the obstacle in the previous frame in the vehicle coordinate system of the previous frame, the absolute velocity of the vehicle in the vehicle coordinate system of the previous frame, and the corresponding rotation vector in the current frame, calculate the absolute velocity of the obstacle that appeared in the previous frame in the vehicle coordinate system of the current frame.

[0021] Based on the absolute acceleration of the obstacle in the previous frame in the vehicle coordinate system of the previous frame, the absolute acceleration of the vehicle in the vehicle coordinate system of the previous frame, and the corresponding rotation vector in the current frame, calculate the absolute acceleration of each obstacle in the previous frame in the vehicle coordinate system of the current frame.

[0022] In one embodiment, the non-interested obstacle includes invalid obstacles. The step of determining whether there are non-interested obstacles among the obstacles appearing in the previous frame, based on the actual position information of obstacles appearing in the current frame and the predicted position information of obstacles appearing in the previous frame in the current frame, includes:

[0023] Based on the actual location information of the obstacles appearing in the current frame, the predicted location information of the obstacles appearing in the previous frame in the current frame, and the preset similarity calculation formula, calculate multiple first similarities for each obstacle appearing in the previous frame.

[0024] In the case where there are multiple instances where the first similarity is not greater than the first preset threshold, the predicted position information of the first target obstacle in the next frame is obtained; the first target obstacle is the obstacle that appeared in the previous frame where the first similarity is not greater than the first preset threshold.

[0025] Obtain the actual location information of at least one obstacle in the point cloud data collected in the next frame;

[0026] Based on the predicted position information of the first target obstacle in the next frame, the actual position information of all obstacles in the point cloud data collected in the next frame, and the preset similarity calculation formula, calculate the corresponding multiple second similarities.

[0027] If there is a second similarity greater than a second preset threshold, it is determined that the second target obstacle exists in at least one obstacle in the point cloud data collected in the next frame, and the second target obstacle is the first target obstacle with a second similarity greater than the second preset threshold;

[0028] If multiple second similarities are not greater than the second preset threshold, the third target obstacle is determined to be an invalid obstacle; the third target obstacle is a first target obstacle whose second similarity is not greater than the second preset threshold.

[0029] In one embodiment, the non-interested obstacles include non-dangerous obstacles. The step of determining whether there are non-interested obstacles among the obstacles appearing in the previous frame, based on the actual position information of obstacles appearing in the current frame and the predicted position information of obstacles appearing in the previous frame in the current frame, further includes:

[0030] If a first similarity greater than a first preset threshold exists, it is determined that the fourth target obstacle exists in at least one obstacle in the point cloud data collected in the current frame, wherein the fourth target obstacle is an obstacle that appeared in the previous frame where the first similarity is greater than the first preset threshold.

[0031] If it is determined that the fourth target obstacle exists in at least one obstacle in the point cloud data collected in the current frame, the first lateral distance and the second lateral distance between the fourth target obstacle and the vehicle are obtained; the first lateral distance is the lateral distance between the fourth target obstacle and the vehicle in the previous frame; the second lateral distance is the lateral distance between the fourth target obstacle and the vehicle in the current frame.

[0032] If both the first lateral distance and the second lateral distance are greater than the distance threshold, the fourth target obstacle is determined to be a non-threatening obstacle.

[0033] In one embodiment, obtaining the predicted location information of the first target obstacle in the next frame includes:

[0034] Obtain the vehicle's motion information in the current frame;

[0035] Obtain the rotation vector and translation matrix corresponding to the next frame;

[0036] Based on the motion information of the first target obstacle in the previous frame, the motion information of the vehicle in the current frame, and the corresponding rotation vector in the next frame, the absolute acceleration and predicted absolute velocity of the first target obstacle in the vehicle coordinate system in the next frame are obtained.

[0037] Based on the absolute acceleration and predicted absolute velocity of the first target obstacle in the vehicle coordinate system of the next frame, and the corresponding rotation vector and translation matrix of the next frame, the predicted position information of the first target obstacle in the next frame is predicted.

[0038] Secondly, this application also provides a data processing apparatus. The apparatus includes:

[0039] The rotation vector and translation matrix determination module is used to determine the corresponding rotation vector and translation matrix for the current frame based on the vehicle's position information and heading angle information in the current frame, the vehicle's position information and heading angle information in the previous frame, and the vehicle's heading angle information in the initial frame.

[0040] The motion information acquisition module is used to acquire the motion information of the vehicle in the previous frame, as well as the motion information of at least one obstacle in the point cloud data collected in the previous frame.

[0041] The predicted position information acquisition module is used to acquire the predicted position information of the obstacle that appeared in the previous frame in the current frame based on the vehicle's motion information in the previous frame, the motion information of at least one obstacle in the point cloud data collected in the previous frame, and the corresponding rotation vector and translation matrix in the current frame.

[0042] The current frame actual position information acquisition module is used to acquire the actual position information of at least one obstacle in the point cloud data collected in the current frame;

[0043] The non-interest obstacle determination module is used to determine whether there are non-interest obstacles among the obstacles that appeared in the previous frame, based on the actual position information of the obstacles that appeared in the current frame and the predicted position information of the obstacles that appeared in the previous frame in the current frame.

[0044] The point cloud data deletion module is used to delete the point cloud data corresponding to non-interested obstacles from the point cloud data collected in the previous frame, if they exist.

[0045] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described above.

[0046] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the above-described method.

[0047] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the above-described method.

[0048] The aforementioned data processing method, apparatus, computer equipment, storage medium, and computer program product, for the current frame, determine the corresponding rotation vector and translation matrix based on the vehicle's position and heading angle information in the current frame, the vehicle's position and heading angle information in the previous frame, and the vehicle's heading angle information in the initial frame; acquire the vehicle's motion information in the previous frame and the motion information of at least one obstacle in the point cloud data collected in the previous frame; based on the vehicle's motion information in the previous frame, the motion information of at least one obstacle in the point cloud data collected in the previous frame, and the corresponding rotation vector and translation matrix in the current frame, acquire the predicted position information of the obstacle appearing in the previous frame in the current frame; acquire the actual position information of at least one obstacle in the point cloud data collected in the current frame; based on the actual position information of the obstacle appearing in the current frame and the predicted position information of the obstacle appearing in the previous frame in the current frame, determine whether there is a non-interest obstacle among the obstacles appearing in the previous frame; if so, delete the point cloud data corresponding to the non-interest obstacle from the point cloud data collected in the previous frame. Compared to traditional methods, this approach uses vehicle position information, heading angle information, motion information, and motion information of at least one obstacle in the point cloud data collected in the previous frame to predict the position information of obstacles appearing in the previous frame in the current frame. Based on the predicted position information of obstacles appearing in the previous frame in the current frame and the actual position information of at least one obstacle in the point cloud data collected in the current frame, it determines whether there are any non-interested obstacles among the obstacles appearing in the previous frame, and promptly removes the point cloud data of non-interested obstacles. This reduces the amount of data processing in the perception fusion system and improves the data processing efficiency of the perception fusion system. Attached Figure Description

[0049] Figure 1 This is a flowchart illustrating a data processing method in one embodiment;

[0050] Figure 2 This is a schematic diagram of the initial coordinate system and the vehicle body coordinate system in one embodiment;

[0051] Figure 3 This is a schematic diagram illustrating the relationship between the vehicle body coordinate systems of adjacent frames in one embodiment;

[0052] Figure 4 This is a flowchart illustrating the process for determining a non-interested obstacle in one embodiment;

[0053] Figure 5 This is a flowchart illustrating the determination process for another type of non-interested obstacle in one embodiment;

[0054] Figure 6 This is a flowchart illustrating the data processing method in another embodiment;

[0055] Figure 7 This is a flowchart illustrating the data processing method in yet another embodiment;

[0056] Figure 8 This is a structural block diagram of a data processing device in one embodiment;

[0057] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0059] In one embodiment, such as Figure 1 As shown, a data processing method is provided for a vehicle, the vehicle including an onboard 4D millimeter-wave radar, the onboard 4D millimeter-wave radar being used to collect point cloud data during the vehicle's autonomous driving process, including the following steps:

[0060] Step 102: For the current frame, determine the corresponding rotation vector and translation matrix based on the vehicle's position information and heading angle information in the current frame, the vehicle's position information and heading angle information in the previous frame, and the vehicle's heading angle information in the initial frame.

[0061] The vehicle's position information can be the vehicle's location information during autonomous driving on the road, and can be recorded using an initial coordinate system. The initial coordinate system can be a coordinate system constructed based on the vehicle's initial position.

[0062] The vehicle's heading angle information can include the heading angle information of the vehicle during autonomous driving on the road. The heading angle of the vehicle in each frame is the angle between the center of mass velocity of the corresponding frame and the X-axis in the initial coordinate system. Specifically, in the initial coordinate system, the direction directly in front of the vehicle is the positive X-axis direction, and the direction in which the X-axis rotates 90 degrees counterclockwise with the ground as the rotation plane is the positive Y-axis direction.

[0063] For example, for the current frame, the corresponding rotation vector and translation matrix can be determined based on the vehicle's position and heading angle information in the current frame, the vehicle's position and heading angle information in the previous frame, and the vehicle's heading angle information in the initial frame.

[0064] Step 104: Obtain the motion information of the vehicle in the previous frame, and the motion information of at least one obstacle in the point cloud data collected in the previous frame.

[0065] The vehicle includes an onboard 4D millimeter-wave radar. Point cloud data is collected by this radar during autonomous driving, including obstacle information. Typically, point cloud data can include motion information of many obstacles, recorded during the vehicle's autonomous driving process. The onboard 4D millimeter-wave radar can collect point cloud data in each time frame, and this data can include motion information of at least one obstacle.

[0066] In practice, vehicle-mounted 4D millimeter-wave radar can accurately detect the position and speed of obstacles. Obstacles can be people or vehicles. They can be static or dynamic. Specifically, during autonomous driving, obstacles can be vehicles, people, streetlights, etc., in front of the vehicle.

[0067] Step 106: Based on the vehicle's motion information in the previous frame, the motion information of at least one obstacle in the point cloud data collected in the previous frame, and the corresponding rotation vector and translation matrix in the current frame, obtain the predicted position information of the obstacle that appeared in the previous frame in the current frame.

[0068] Among them, the predicted position information of the obstacle in the current frame can be the predicted position information of the obstacle in the current time frame.

[0069] In practice, if the current time frame is set as frame k, the position information of the corresponding obstacle in frame k can be predicted by using the motion and position information of the obstacle in frame k-1.

[0070] For example, the predicted position information of an obstacle appearing in the previous frame in the current frame can be predicted using the vehicle's motion information in the previous frame, the motion information of at least one obstacle in the point cloud data collected in the previous frame, and the corresponding rotation vector and translation matrix in the current frame.

[0071] Step 108: Obtain the actual location information of at least one obstacle in the point cloud data collected in the current frame.

[0072] For example, an onboard 4D millimeter-wave radar can acquire the actual location information of at least one obstacle in the point cloud data collected in the current frame. Specifically, an obstacle in the point cloud data collected in the previous frame may not be among the obstacles collected in the point cloud data collected in the current frame.

[0073] Step 110: Based on the actual position information of the obstacles appearing in the current frame and the predicted position information of the obstacles appearing in the previous frame in the current frame, determine whether there are any non-interested obstacles among the obstacles appearing in the previous frame.

[0074] Among them, non-interest obstacles can be useless obstacles, such as obstacles that do not affect driving.

[0075] For example, whether there are non-interested obstacles among the obstacles that appeared in the previous frame can be determined based on the actual position information of the obstacles that appeared in the current frame and the predicted position information of the obstacles that appeared in the previous frame in the current frame.

[0076] Step 112: If the obstacle exists, delete the point cloud data corresponding to the non-interested obstacle from the point cloud data collected in the previous frame.

[0077] In practice, the vehicle includes a perception fusion module. After deleting non-interested obstacles from the point cloud data collected in the previous frame, the processed data can be input into the perception fusion module for data fusion.

[0078] In the above data processing method, for the current frame, based on the vehicle's position and heading angle information in the current frame, the vehicle's position and heading angle information in the previous frame, and the vehicle's heading angle information in the initial frame, the corresponding rotation vector and translation matrix for the current frame are determined; the vehicle's motion information in the previous frame and the motion information of at least one obstacle in the point cloud data collected in the previous frame are obtained; based on the vehicle's motion information in the previous frame, the motion information of at least one obstacle in the point cloud data collected in the previous frame, and the corresponding rotation vector and translation matrix for the current frame, the predicted position information of the obstacle appearing in the previous frame in the current frame is obtained; the actual position information of at least one obstacle in the point cloud data collected in the current frame is obtained; based on the actual position information of the obstacle appearing in the current frame and the predicted position information of the obstacle appearing in the previous frame in the current frame, it is determined whether there are any non-interested obstacles among the obstacles appearing in the previous frame; if so, the point cloud data corresponding to the non-interested obstacle is deleted from the point cloud data collected in the previous frame. Compared to traditional methods, this approach uses vehicle position information, heading angle information, motion information, and motion information of at least one obstacle in the point cloud data collected in the previous frame to predict the position information of obstacles appearing in the previous frame in the current frame. Based on the predicted position information of obstacles appearing in the previous frame in the current frame and the actual position information of at least one obstacle in the point cloud data collected in the current frame, it determines whether there are any non-interested obstacles among the obstacles appearing in the previous frame, and promptly removes the point cloud data of non-interested obstacles. This reduces the amount of data processing in the perception fusion system and improves the data processing efficiency of the perception fusion system.

[0079] In one embodiment, step 102 includes:

[0080] Step 1022: Based on the vehicle's heading angle information in the initial frame, the heading angle information in the previous frame, and the heading angle information in the current frame, obtain the vehicle's first relative heading angle information and second relative heading angle information.

[0081] The heading angle information can be the angle between the vehicle's center of mass velocity in the corresponding frame and the X-axis in the initial coordinate system. In the initial coordinate system, the direction directly in front of the vehicle is the positive X-axis, and the direction in which the X-axis rotates 90 degrees counterclockwise around the ground is the positive Y-axis. Specifically, the vehicle's heading angle information in the initial frame is the angle between its center of mass velocity in the initial frame and the X-axis in the initial coordinate system. The vehicle's heading angle information in the previous frame is the angle between its center of mass velocity in the previous frame and the X-axis in the initial coordinate system. The vehicle's heading angle information in the current frame is the angle between its center of mass velocity in the current frame and the X-axis in the initial coordinate system.

[0082] The first relative heading angle information is the relative angle information between the heading angle information of the current frame and the heading angle information of the previous frame; the second relative heading angle information is the relative angle information between the heading angle information of the current frame and the heading angle information of the initial frame.

[0083] For example, a first relative heading angle is determined based on the heading angle information of the current frame and the heading angle information of the previous frame; a second relative heading angle is determined based on the heading angle information of the current frame and the heading angle information of the initial frame.

[0084] Step 1024: Obtain the relative position information of the vehicle based on the vehicle's position information in the current frame and its position information in the previous frame.

[0085] The relative position information refers to the vehicle's position in the current frame relative to its position in the previous frame. The vehicle's position information is determined based on the initial coordinate system.

[0086] Step 1026: Based on the first relative heading angle information, obtain the rotation vector corresponding to the current frame.

[0087] Among them, the angle information corresponding to the rotation vector of the current frame is the first relative heading angle information.

[0088] For example, if the first relative heading angle information is θ(k), then the corresponding rotation vector for the current frame is:

[0089]

[0090] Where k is the current frame, θ(k) is the relative angle information between the heading angle information of the current frame and the heading angle information of the previous frame, i.e., the first relative heading angle information, and R(k) is the corresponding rotation vector of the current frame.

[0091] Step 1028: Based on the second relative heading angle information and the relative position information, obtain the translation matrix corresponding to the current frame.

[0092] The translation matrix corresponding to the current frame can be expressed as:

[0093]

[0094] Where k is the current frame, and α(k) is the second relative heading angle information, specifically the relative angle information of the current frame's heading angle information with respect to the initial heading angle information; (P x (k), P y (k) represents the relative position information, specifically the coordinate information of the vehicle's position in the current frame relative to the vehicle's position in the previous frame in the initial coordinate system; T(k) is the corresponding translation matrix for the current frame.

[0095] In the above embodiments, the rotation vector and translation vector of the current frame are obtained by using the vehicle's first relative heading angle information, second relative heading angle information, and relative position information. This allows for a better determination of whether there are non-interested obstacles in the point cloud data collected in the previous frame, and timely removal of point cloud data containing non-interested obstacles. This reduces the amount of data processing in the perception fusion system and improves the data processing efficiency of the perception fusion system.

[0096] In one embodiment, step 106 includes:

[0097] Step 1062: Based on the vehicle's motion information in the previous frame, the motion information of at least one obstacle in the point cloud data collected in the previous frame, and the corresponding rotation vector in the current frame, obtain the absolute velocity and absolute acceleration of the obstacle in the previous frame in the vehicle coordinate system of the current frame; the vehicle coordinate system is constructed based on the vehicle's position information in each frame, and there is a corresponding vehicle coordinate system for each frame.

[0098] The vehicle coordinate system is a two-dimensional coordinate system established with the vehicle's position at the corresponding time frame as its origin. Specifically, the initial coordinate system is the coordinate system established when the vehicle starts, a two-dimensional coordinate system with the vehicle's starting position as its origin. During the vehicle's autonomous driving process, a vehicle coordinate system exists for each corresponding frame.

[0099] Specifically, refer to Figure 2 The diagram illustrates a schematic of the initial coordinate system and the vehicle body coordinate system in one embodiment. In the initial coordinate system, the front of the vehicle is defined as the positive X-axis; the positive Y-axis is defined as the direction in which the X-axis rotates 90 degrees counterclockwise around the ground plane. In the vehicle body coordinate system, the front of the vehicle is defined as the positive x-axis; the positive y-axis is defined as the direction in which the x-axis rotates 90 degrees counterclockwise around the ground plane.

[0100] The vehicle's motion information can be the vehicle's motion information in the vehicle's coordinate system of the corresponding frame. Specifically, the vehicle's motion information in the previous frame is the vehicle's motion information in the vehicle's coordinate system of the previous frame.

[0101] The motion information of at least one obstacle in the point cloud data acquired in the previous frame is the motion information of at least one obstacle in the vehicle coordinate system of the previous frame in the point cloud data acquired in the previous frame.

[0102] The rotation vector corresponding to the current frame is the rotation vector of the vehicle body coordinate system obtained by transforming the vehicle body coordinate system of the previous frame into the vehicle body coordinate system of the current frame.

[0103] For example, refer to Figure 3 This diagram illustrates the relationship between the vehicle coordinate systems of adjacent frames, including the vehicle coordinate system of frame (k-1) and frame (k). The x-axis of the vehicle coordinate system in frame (k-1) is x... (k-1) The y-axis is y (k-1) The origin is O (k-1) The x-axis of the vehicle body coordinate system in the k-th frame is x k The y-axis is y k The origin is O k .

[0104] Specifically, the vehicle coordinate system of the k-th frame is translated so that its origin coincides with the origin of the vehicle coordinate system of the (k-1)-th frame, resulting in the x-axis of the coordinate system being x′. k The y-axis is y′ k Where θ represents the angular change of the vehicle's body coordinate system in frame k relative to the vehicle coordinate system in frame (k-1); (P x (k), P y (k) represents the relative position information of the vehicle in the initial coordinate system at frame k relative to frame k-1. Figure 3 In the diagram, the initial coordinate system is not shown.

[0105] Step 1064: Based on the absolute velocity and absolute acceleration of the obstacle in the previous frame in the vehicle coordinate system of the current frame, and the corresponding rotation vector and translation matrix of the current frame, obtain the predicted position information of the obstacle that appeared in the previous frame in the current frame.

[0106] Specifically, the calculation process of transforming the coordinate information of the obstacle acquired in the previous frame in the vehicle coordinate system to the coordinate information in the current frame in the vehicle coordinate system can be as follows:

[0107]

[0108] Where k represents the current frame; k-1 represents the previous frame;

[0109] The x-coordinate information of the position of obstacle i in the vehicle coordinate system in the k-1th frame;

[0110] The vertical coordinate information of obstacle i in the vehicle coordinate system in the k-1th frame;

[0111] The x-coordinate information of the position of obstacle i in the vehicle coordinate system corresponding to the k-1 frame is collected in the k-1 frame;

[0112] P yi (k-1) represents the vertical coordinate of obstacle i in the vehicle coordinate system corresponding to the k-1 frame, which is the position information of obstacle i collected in the k-1 frame.

[0113] R(k) is the rotation vector corresponding to the current frame; T(k) is the translation matrix corresponding to the current frame.

[0114] Specifically, the process of obtaining the predicted position information of an obstacle that appeared in the previous frame in the current frame can be as follows:

[0115] Assuming obstacle i undergoes uniform acceleration between frame k-1 and frame k, the predicted position information of the obstacle appearing in frame k-1 in frame k is as follows:

[0116]

[0117] Where k is the current frame; k-1 is the previous frame; t(k) is the time corresponding to the current frame; t(k-1) is the time corresponding to the previous frame;

[0118] v′ xiabs (k-1) represents the x-coordinate of the absolute velocity of obstacle i in the vehicle coordinate system of the current frame, acquired in the previous frame; v′ yiabs (k-1) represents the vertical coordinate information of the absolute velocity of obstacle i in the vehicle coordinate system of the current frame, which was collected in the previous frame.

[0119] a′ xiabs (k-1) represents the x-coordinate information of the absolute acceleration of obstacle i in the vehicle coordinate system of the current frame, acquired in the previous frame; a′ yiabs (k-1) represents the vertical coordinate information of the absolute acceleration of obstacle i in the vehicle coordinate system of the current frame, which was collected in the previous frame.

[0120] The x-coordinate information of the position of obstacle i in the vehicle coordinate system in the k-1th frame;

[0121] The vertical coordinate information of obstacle i in the vehicle coordinate system in the k-1th frame;

[0122] The x-coordinate information of the position information of obstacle i in the vehicle coordinate system of the k-1th frame, which is collected for the prediction of the k-1th frame;

[0123] The vertical coordinate information of obstacle i in the vehicle coordinate system of the k-th frame, which is the position information of obstacle i collected in the k-1th frame for prediction.

[0124] In the above embodiments, by using the vehicle's motion information in the previous frame, the motion information of at least one obstacle in the point cloud data collected in the previous frame, and the corresponding rotation vector and translation matrix in the current frame, the position information of obstacles appearing in the previous frame in the current frame is predicted. Based on the predicted position information of obstacles appearing in the previous frame in the current frame and the actual position information of at least one obstacle in the point cloud data collected in the current frame, it is determined whether there are any non-interested obstacles among the obstacles appearing in the previous frame. This allows for the timely removal of point cloud data containing non-interested obstacles, reducing the amount of data processing in the perception fusion system and improving its data processing efficiency. Furthermore, the vehicle coordinate system is switched in real time to make the point cloud data more consistent with the real-time requirements of autonomous driving, improving the real-time performance of the point cloud data.

[0125] In one embodiment, the vehicle's motion information in the previous frame includes the vehicle's velocity and acceleration in the vehicle's coordinate system in the previous frame; the motion information of at least one obstacle in the point cloud data acquired in the previous frame includes the obstacle's velocity and acceleration in the vehicle's coordinate system in the previous frame; step 1062 includes:

[0126] Based on the speed of the obstacle in the previous frame in the vehicle coordinate system, the speed of the vehicle in the previous frame in the vehicle coordinate system, and the corresponding rotation vector in the current frame, calculate the absolute speed of the obstacle that appeared in the previous frame in the vehicle coordinate system in the current frame.

[0127] Based on the acceleration of obstacles appearing in the previous frame in the vehicle's coordinate system in the previous frame, the acceleration of the vehicle in the vehicle's coordinate system in the previous frame, and the corresponding rotation vector in the current frame, calculate the absolute acceleration of each obstacle appearing in the previous frame in the vehicle's coordinate system in the current frame.

[0128] The absolute velocity of an obstacle appearing in the previous frame in the vehicle's coordinate system in the current frame can be calculated using the obstacle's velocity in the previous frame's vehicle coordinate system, the vehicle's velocity in the previous frame's vehicle coordinate system, and the corresponding rotation vector in the current frame. Specifically, the calculation process for the absolute velocity of an obstacle appearing in the previous frame in the vehicle's coordinate system in the current frame can be as follows:

[0129] 1) Calculate the absolute velocity of the obstacle that appeared in the previous frame in the vehicle coordinate system of the previous frame.

[0130]

[0131] Among them, v xiabs (k-1) represents the x-coordinate of the absolute velocity of obstacle i in the vehicle coordinate system of the previous frame; v yiabs (k-1) represents the vertical coordinate information of the absolute velocity of obstacle i in the vehicle coordinate system of the previous frame; v xi (k-1) represents the x-coordinate of the velocity of obstacle i in the vehicle coordinate system of the previous frame; v yi (k-1) represents the vertical coordinate of the velocity of obstacle i in the vehicle coordinate system of the previous frame; v(k-1) represents the horizontal coordinate of the velocity of the vehicle in the vehicle coordinate system of the previous frame.

[0132] Specifically, the vehicle coordinate system of each frame takes the current vehicle as the origin, and the front of the vehicle in the vehicle coordinate system is the positive x-axis; the direction of the x-axis rotating 90 degrees counterclockwise with the ground as the rotation plane is the positive y-axis. Therefore, the vertical coordinate information of the collected vehicle acceleration in the corresponding frame is 0.

[0133] 2) Calculate the absolute velocity of the obstacle that appeared in the previous frame in the vehicle coordinate system of the current frame based on the corresponding rotation vector of the current frame.

[0134]

[0135] Where R(k) is the rotation vector corresponding to the current frame; v xiabs (k-1) represents the x-coordinate of the absolute velocity of obstacle i in the vehicle coordinate system of the previous frame; v yiabs (k-1) represents the vertical coordinate information of the absolute velocity of obstacle i in the vehicle coordinate system of the previous frame; v′ xiabs (k-1) represents the x-coordinate of the absolute velocity of obstacle i in the vehicle coordinate system of the current frame, acquired in the previous frame; v′ yiabs (k-1) represents the vertical coordinate information of the absolute velocity of obstacle i in the vehicle coordinate system of the current frame, which was collected in the previous frame.

[0136] Specifically, the calculation process for the absolute acceleration of each obstacle appearing in the previous frame in the vehicle coordinate system of the current frame can be as follows:

[0137] 1) Calculate the absolute acceleration of the obstacle that appeared in the previous frame in the vehicle coordinate system of the previous frame.

[0138]

[0139] Among them, a xiabs (k-1) represents the x-coordinate information of the absolute acceleration of obstacle i in the vehicle coordinate system of the previous frame; a yiabs (k-1) represents the ordinate information of the absolute acceleration of obstacle i in the vehicle coordinate system of the previous frame; a xi (k-1) represents the x-coordinate information of the acceleration of obstacle i in the vehicle coordinate system of the previous frame; a yi (k-1) represents the vertical coordinate information of the acceleration of obstacle i in the vehicle coordinate system of the previous frame; a(k-1) represents the horizontal coordinate information of the acceleration of the vehicle in the vehicle coordinate system of the previous frame.

[0140] Similarly, the vertical coordinate information of the collected vehicle acceleration in the corresponding frame is 0.

[0141] 2) Based on the rotation vector of the current frame, calculate the absolute acceleration of the obstacle that appeared in the previous frame in the vehicle coordinate system of the current frame.

[0142]

[0143] Where R(k) is the rotation vector corresponding to the current frame; a xiabs (k-1) represents the abscissa information of the absolute velocity of obstacle i in the vehicle coordinate system of the previous frame; a yiabs (k-1) represents the vertical coordinate information of the absolute velocity of obstacle i in the vehicle coordinate system of the previous frame; a′ xiabs (k-1) represents the abscissa information of the absolute velocity of obstacle i in the vehicle coordinate system of the current frame, acquired in the previous frame; a′ yiabs (k-1) represents the vertical coordinate information of the absolute velocity of obstacle i in the vehicle coordinate system of the current frame, which was collected in the previous frame.

[0144] In the above embodiments, based on the acceleration and velocity of obstacles in the vehicle coordinate system of the previous frame, the acceleration and velocity of the vehicle in the vehicle coordinate system of the previous frame, and the corresponding rotation vector in the current frame, the absolute acceleration and absolute velocity of each obstacle appearing in the previous frame in the vehicle coordinate system of the current frame are calculated. Based on the absolute acceleration and absolute velocity of each obstacle appearing in the previous frame in the vehicle coordinate system of the current frame, the position information of obstacles appearing in the previous frame in the current frame is predicted to determine whether there are any non-interested obstacles among the obstacles appearing in the previous frame. This allows for the timely removal of point cloud data containing non-interested obstacles, reducing the amount of data processing in the perception fusion system and improving the data processing efficiency of the perception fusion system.

[0145] In one embodiment, reference Figure 4 The diagram illustrates a flowchart of a process for determining a non-interested obstacle in one embodiment. The non-interested obstacle includes an invalid obstacle. Step 110 includes:

[0146] Step 402: Based on the actual location information of the obstacles appearing in the current frame, the predicted location information of the obstacles appearing in the previous frame in the current frame, and the preset similarity calculation formula, calculate multiple first similarities for each obstacle appearing in the previous frame.

[0147] The preset similarity calculation formula can be:

[0148]

[0149] Where simi represents the similarity, p′ xi (k) represents the x-coordinate of the position of obstacle i in the vehicle coordinate system in the k-th frame, which is the predicted position of obstacle i in the (k-1)-th frame; i is the obstacle in the (k-1)-th frame; j is the obstacle in the k-th frame.

[0150] p′ yi (k) represents the ordinate of the position of obstacle i in the vehicle coordinate system in the k-th frame, which is the predicted position information of obstacle i in the k-1th frame.

[0151] p xj (k) represents the x-coordinate of the actual position of obstacle j in the vehicle coordinate system of the current frame in the k-th frame;

[0152] p yj (k) represents the ordinate of the actual position of obstacle j in the vehicle coordinate system in the current frame.

[0153] In practice, for an obstacle i collected in the previous frame, the actual position information of each obstacle collected in the current frame needs to be calculated sequentially to obtain multiple first similarities. Specifically, for an obstacle collected in frame k-1, the number of corresponding first similarities is equal to the number of obstacles collected in frame k.

[0154] Step 404: In the case that there are multiple first similarities that are not greater than the first preset threshold, obtain the predicted position information of the first target obstacle in the next frame; the first target obstacle is the obstacle that appeared in the previous frame where the first similarities are not greater than the first preset threshold.

[0155] The first target obstacle is any of the obstacles appearing in the previous frame whose first similarity is no greater than a first preset threshold. The predicted position information of the first target obstacle in the next frame can be predicted from the motion information of the first target obstacle in the previous frame.

[0156] Specifically, the first preset threshold can be set based on an empirical value; for example, the first preset threshold can be 0.7.

[0157] For example, if multiple obstacles captured in the previous frame have multiple first similarities that are all less than a first preset threshold, the predicted position information of the first target obstacle in the next frame is obtained. Specifically, there may be one or more first target obstacles among the obstacles captured in the previous frame.

[0158] Step 406: Obtain the actual location information of at least one obstacle in the point cloud data collected in the next frame.

[0159] The vehicle-mounted 4D millimeter-wave radar can acquire the actual location information of at least one obstacle in the point cloud data collected in the next frame. Specifically, an obstacle in the point cloud data collected in the current frame may not be an obstacle in the point cloud data collected in the next frame.

[0160] Step 408: Based on the predicted position information of the first target obstacle in the next frame, the actual position information of all obstacles in the point cloud data collected in the next frame, and the preset similarity calculation formula, calculate the corresponding multiple second similarities.

[0161] In practice, for an obstacle i collected in the current frame, the actual position information of each obstacle collected in the next frame needs to be calculated sequentially to obtain multiple second similarities. Specifically, for an obstacle collected in frame k, the number of corresponding second similarities is equal to the number of obstacles collected in frame k+1.

[0162] Specifically, the calculation method for the second similarity can refer to the calculation process for the first similarity, which will not be elaborated here.

[0163] Step 410: If there is a second similarity greater than the second preset threshold, determine that the second target obstacle exists in at least one obstacle in the point cloud data collected in the next frame, and the second target obstacle is the first target obstacle with the second similarity greater than the second preset threshold.

[0164] The second preset threshold can be an empirical value, or it can be equal to the first preset threshold. The second target obstacle is an obstacle among the first target obstacles whose second similarity is greater than the second preset threshold.

[0165] For example, if an obstacle in the first target obstacle has a second similarity greater than a second preset threshold, it is determined that the obstacle exists in at least one obstacle in the point cloud data collected in the next frame.

[0166] Step 412: If there are multiple instances where the second similarity is not greater than the second preset threshold, the third target obstacle is determined to be an invalid obstacle; the third target obstacle is a first target obstacle where the second similarity is not greater than the second preset threshold.

[0167] The third target obstacle can be any of the first target obstacles whose second similarity is not greater than a second preset threshold.

[0168] In the above embodiments, multiple first similarities corresponding to each obstacle appearing in the previous frame are initially obtained through a preset similarity calculation formula, thereby filtering out obstacles appearing in the previous frame whose first similarities are not greater than a first preset threshold. The position information of the next frame is predicted again, so as to determine whether there are invalid obstacles among the first target obstacles based on the predicted position information of the next frame and the preset similarity calculation formula, thereby timely removing the point cloud data of invalid obstacles in the previous frame.

[0169] In one embodiment, step 404 includes:

[0170] Step 4042: Obtain the vehicle's motion information in the current frame.

[0171] The vehicle's motion information in the current frame can include the vehicle's velocity and acceleration in the vehicle's coordinate system in the current frame.

[0172] Step 4044: Obtain the rotation vector and translation matrix of the next frame.

[0173] Specifically, the process of obtaining the rotation vector and translation matrix of the next frame can be referred to in steps 1022 to 1028, which are for obtaining the rotation vector and translation matrix of the current frame. This invention will not elaborate on this process here.

[0174] Step 4046: Based on the motion information of the first target obstacle in the previous frame, the motion information of the vehicle in the current frame, and the corresponding rotation vector in the next frame, obtain the absolute acceleration and predicted absolute velocity of the first target obstacle in the vehicle coordinate system of the next frame.

[0175] In practice, the first target obstacle can be set to undergo uniform acceleration between the previous and next frames to obtain its absolute acceleration and predicted absolute velocity in the vehicle's coordinate system in the next frame. The specific steps are as follows:

[0176] 1) Based on the velocity and acceleration of the first target obstacle in the vehicle coordinate system of the previous frame, calculate the predicted velocity of the first target obstacle in the current frame.

[0177] Specifically, through the velocity formula for uniform acceleration (V) k =V k-1 +a k t) is calculated.

[0178] Where k is the current frame; k-1 is the previous frame; V k For the predicted velocity, V is the predicted velocity in the vehicle coordinate system of the previous frame for the current frame; k-1 This provides the velocity information of the first target obstacle in the previous frame.

[0179] 2) Using the corresponding rotation vector of the current frame, the acceleration in the vehicle coordinate system of the previous frame and the predicted velocity of the first target obstacle in the current frame are transformed to the vehicle coordinate system of the current frame, so as to obtain the acceleration and predicted velocity of the first target obstacle in the vehicle coordinate system of the current frame.

[0180] 3) Based on the predicted velocity and acceleration of the first target obstacle in the vehicle coordinate system of the current frame, and the velocity and acceleration of the vehicle in the current frame, calculate the absolute acceleration and predicted absolute velocity of the first target obstacle in the vehicle coordinate system of the current frame.

[0181] 4) Using the corresponding rotation vector in the next frame, the absolute acceleration and predicted absolute velocity of the first target obstacle in the current frame's vehicle coordinate system are transformed to the vehicle coordinate system in the next frame, so as to obtain the absolute acceleration and predicted absolute velocity of the first target obstacle in the vehicle coordinate system in the next frame.

[0182] Specifically, the coordinate transformation through rotation vector and the solution of the absolute velocity and absolute acceleration of the obstacle can be found in the relevant content of step 1062, which will not be elaborated here.

[0183] Step 4048: Based on the absolute acceleration and predicted absolute velocity of the first target obstacle in the vehicle coordinate system of the next frame, and the corresponding rotation vector and translation matrix of the next frame, predict the predicted position information of the first target obstacle in the next frame.

[0184] Specifically, how to obtain the predicted position information of the first target obstacle in the next frame can be found in the relevant content of step 1064, which will not be elaborated here.

[0185] In the above embodiments, the position information of obstacles appearing in the previous frame in the next frame is predicted by using the vehicle's motion information in the current frame, the motion information of at least one obstacle in the point cloud data collected in the current frame, and the corresponding rotation vector and translation matrix in the next frame.

[0186] In one embodiment, reference Figure 5 The diagram illustrates a flowchart of a process for determining another type of non-concerned obstacle in one embodiment. The non-concerned obstacle includes a non-dangerous obstacle. Step 110 includes:

[0187] Step 502: If there is a first similarity greater than a first preset threshold, determine that the fourth target obstacle exists in at least one obstacle in the point cloud data collected in the current frame, wherein the fourth target obstacle is an obstacle that appeared in the previous frame with a first similarity greater than the first preset threshold.

[0188] The fourth target obstacle is an obstacle that appeared in the previous frame and has a first similarity greater than a first preset threshold. In practice, if at least one of the multiple first similarities corresponding to obstacles that appeared in the previous frame is greater than the first preset threshold, it means that at least one obstacle in the point cloud data collected in the current frame is an obstacle that appeared in the previous frame.

[0189] Step 504: If it is determined that the fourth target obstacle exists in at least one obstacle in the point cloud data collected in the current frame, obtain the first lateral distance and the second lateral distance between the fourth target obstacle and the vehicle; the first lateral distance is the lateral distance between the fourth target obstacle and the vehicle in the previous frame; the second lateral distance is the lateral distance between the fourth target obstacle and the vehicle in the current frame.

[0190] The lateral distance is the distance between the side of the vehicle and the obstacle in the vehicle coordinate system. Specifically, in the current frame's vehicle coordinate system, the lateral distance is the absolute value of the obstacle's vertical coordinate (y-axis). In the vehicle coordinate system, the front of the vehicle is the positive x-axis; the positive y-axis is the direction in which the x-axis rotates 90 degrees counterclockwise around the ground plane.

[0191] Step 506: If both the first lateral distance and the second lateral distance are greater than the distance threshold, the fourth target obstacle is determined to be a non-threatening obstacle.

[0192] Specifically, the distance threshold can be set based on actual conditions. For example, it can be set based on the width of the lane.

[0193] In practice, if the absolute value of the lateral position of an obstacle is outside the adjacent lane for two consecutive frames, it indicates that the obstacle is a non-threatening obstacle.

[0194] For example, if |p yi (k-1)| and |p yi If all (k) are greater than the distance threshold, then obstacle i can be determined to be a non-threatening obstacle.

[0195] In the above embodiments, by using a preset similarity calculation formula, multiple first similarities are initially obtained for each obstacle that appears in the previous frame, thereby filtering out obstacles that appear in the previous frame with a first similarity greater than a first preset threshold, and obtaining the first lateral distance and the second lateral distance between the obstacle and the vehicle in the previous frame and the current frame, thereby filtering out non-threatening obstacles when both the first lateral distance and the second lateral distance are greater than the distance threshold.

[0196] To better understand the data processing process, an example will be provided. (See reference...) Figure 6 The diagram illustrates a data processing method.

[0197] Step 602: Convert the point cloud information of frame k-1 to the vehicle coordinate system of frame k.

[0198] Specifically, the point cloud information can be the point cloud data of the obstacle collected in the (k-1)th frame, which may include, but is not limited to, the obstacle's position information, velocity information, and acceleration information.

[0199] Step 604: Traverse all point cloud information collected in frame k-1.

[0200] Step 606: Determine whether the point cloud information collected in frame k matches the point cloud information collected in frame (k-1). If it matches, proceed to step 610; otherwise, proceed to step 608.

[0201] Step 608: Determine whether the point cloud information collected in frame (k-1) matches the point cloud information collected in frame (k-1). If it matches, proceed to step 610. If it does not match, proceed to step 614.

[0202] Step 610: Determine if there is no dangerous obstacle. If there is no dangerous obstacle, proceed to step 612; if there is no dangerous obstacle, proceed to step 614.

[0203] Specifically, it determines whether the obstacles in the point cloud information are non-dangerous obstacles.

[0204] Step 612: Retain the point cloud information. Then proceed to step 616.

[0205] Step 614: Delete the point cloud information. Then proceed to step 616.

[0206] Step 616: Determine whether all point cloud information collected in frame k-1 has been traversed. If the traversal is complete, proceed to step 618; otherwise, proceed to step 604.

[0207] Step 618: Output the point cloud information of the (k-1)th frame.

[0208] To better understand the data processing process, an example will be provided. (See reference...) Figure 7 This shows a flowchart of yet another data processing method.

[0209] Step 702: Based on the vehicle's heading angle information in the initial frame, the heading angle information in the previous frame, and the heading angle information in the current frame, obtain the vehicle's first relative heading angle information and second relative heading angle information.

[0210] Step 704: Obtain the relative position information of the vehicle based on the vehicle's position information in the current frame and its position information in the previous frame.

[0211] Step 706: Based on the first relative heading angle information, obtain the rotation vector corresponding to the current frame; based on the second relative heading angle information and relative position information, obtain the translation matrix corresponding to the current frame.

[0212] Step 708: Calculate the absolute velocity of the obstacle that appeared in the previous frame in the vehicle coordinate system of the current frame based on the velocity of the obstacle in the previous frame in the vehicle coordinate system of the previous frame, the velocity of the vehicle in the vehicle coordinate system of the previous frame, and the corresponding rotation vector in the current frame.

[0213] Step 710: Calculate the absolute acceleration of each obstacle in the previous frame in the vehicle coordinate system of the current frame based on the acceleration of the obstacle in the previous frame in the vehicle coordinate system of the previous frame, the acceleration of the vehicle in the vehicle coordinate system of the previous frame, and the corresponding rotation vector in the current frame.

[0214] Step 712: Based on the absolute velocity and absolute acceleration of the obstacle in the previous frame in the vehicle coordinate system of the current frame, and the corresponding rotation vector and translation matrix of the current frame, obtain the predicted position information of the obstacle that appeared in the previous frame in the current frame.

[0215] Step 714: Based on the actual location information of the obstacles appearing in the current frame, the predicted location information of the obstacles appearing in the previous frame in the current frame, and the preset similarity calculation formula, calculate multiple first similarities for each obstacle appearing in the previous frame.

[0216] Step 716: If there are multiple first similarities that are not greater than the first preset threshold, obtain the predicted position information of the first target obstacle in the next frame.

[0217] Step 718: Based on the predicted position information of the first target obstacle in the next frame, the actual position information of all obstacles in the point cloud data collected in the next frame, and the preset similarity calculation formula, calculate the corresponding multiple second similarities.

[0218] Step 720: If there are multiple instances where the second similarity is not greater than the second preset threshold, the third target obstacle is determined to be an invalid obstacle.

[0219] Step 722: If the first similarity is greater than the first preset threshold, determine that the fourth target obstacle exists in at least one obstacle in the point cloud data collected in the current frame, and obtain the first lateral distance and the second lateral distance between the fourth target obstacle and the vehicle.

[0220] Step 724: If both the first lateral distance and the second lateral distance are greater than the distance threshold, the fourth target obstacle is determined to be a non-threatening obstacle.

[0221] In this embodiment, for the current frame, based on the vehicle's position and heading angle information in the current frame, the vehicle's position and heading angle information in the previous frame, and the vehicle's heading angle information in the initial frame, the corresponding rotation vector and translation matrix for the current frame are determined; the vehicle's motion information in the previous frame and the motion information of at least one obstacle in the point cloud data collected in the previous frame are obtained; based on the vehicle's motion information in the previous frame, the motion information of at least one obstacle in the point cloud data collected in the previous frame, and the corresponding rotation vector and translation matrix for the current frame, the predicted position information of the obstacle appearing in the previous frame in the current frame is obtained; the actual position information of at least one obstacle in the point cloud data collected in the current frame is obtained; based on the actual position information of the obstacle appearing in the current frame and the predicted position information of the obstacle appearing in the previous frame in the current frame, it is determined whether there are any non-interested obstacles among the obstacles appearing in the previous frame; if so, the point cloud data corresponding to the non-interested obstacle is deleted from the point cloud data collected in the previous frame. Compared to traditional methods, this approach uses vehicle position information, heading angle information, motion information, and motion information of at least one obstacle in the point cloud data collected in the previous frame to predict the position information of obstacles appearing in the previous frame in the current frame. Based on the predicted position information of obstacles appearing in the previous frame in the current frame and the actual position information of at least one obstacle in the point cloud data collected in the current frame, it determines whether there are any non-interested obstacles among the obstacles appearing in the previous frame, and promptly removes the point cloud data of non-interested obstacles. This reduces the amount of data processing in the perception fusion system and improves the data processing efficiency of the perception fusion system.

[0222] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0223] Based on the same inventive concept, this application also provides a data processing apparatus for implementing the data processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more data processing apparatus embodiments provided below can be found in the limitations of the data processing method described above, and will not be repeated here.

[0224] In one embodiment, such as Figure 8 As shown, a data processing device is provided, including: a rotation vector and translation matrix determination module 802, a motion information acquisition module 804, a predicted position information acquisition module 806, a current frame actual position information acquisition module 808, a non-interested obstacle determination module 810, and a point cloud data deletion module 812, wherein:

[0225] The rotation vector and translation matrix determination module 802 is used to determine the corresponding rotation vector and translation matrix for the current frame based on the vehicle's position information and heading angle information in the current frame, the vehicle's position information and heading angle information in the previous frame, and the vehicle's heading angle information in the initial frame.

[0226] The motion information acquisition module 804 is used to acquire the motion information of the vehicle in the previous frame and the motion information of at least one obstacle in the point cloud data collected in the previous frame.

[0227] The predicted position information acquisition module 806 is used to acquire the predicted position information of the obstacle that appeared in the previous frame in the current frame based on the vehicle's motion information in the previous frame, the motion information of at least one obstacle in the point cloud data collected in the previous frame, and the corresponding rotation vector and translation matrix in the current frame.

[0228] The current frame actual position information acquisition module 808 is used to acquire the actual position information of at least one obstacle in the point cloud data collected in the current frame;

[0229] The non-interest obstacle determination module 810 is used to determine whether there are non-interest obstacles among the obstacles that appeared in the previous frame, based on the actual position information of the obstacles that appeared in the current frame and the predicted position information of the obstacles that appeared in the previous frame in the current frame.

[0230] The point cloud data deletion module 812 is used to delete the point cloud data corresponding to non-interested obstacles from the point cloud data collected in the previous frame, if they exist.

[0231] In some embodiments, the rotation vector and translation matrix determination module 802 includes:

[0232] The relative heading angle acquisition submodule is used to acquire the first relative heading angle information and the second relative heading angle information of the vehicle based on the heading angle information of the vehicle in the initial frame, the heading angle information in the previous frame and the heading angle information in the current frame.

[0233] The relative position information acquisition submodule is used to acquire the relative position information of the vehicle based on the vehicle's position information in the current frame and its position information in the previous frame.

[0234] The current frame rotation vector acquisition submodule is used to acquire the corresponding rotation vector of the current frame based on the first relative heading angle information;

[0235] The current frame translation matrix acquisition submodule is used to acquire the corresponding translation matrix of the current frame based on the second relative heading angle information and the relative position information.

[0236] In some embodiments, the predicted location information acquisition module 806 includes:

[0237] The current frame absolute velocity and acceleration acquisition submodule is used to acquire the absolute velocity and absolute acceleration of the obstacle in the previous frame in the vehicle coordinate system of the current frame based on the vehicle's motion information in the previous frame, the motion information of at least one obstacle in the point cloud data collected in the previous frame, and the corresponding rotation vector in the current frame; the vehicle coordinate system is constructed based on the vehicle's position information in each frame, and there is a corresponding vehicle coordinate system for each frame.

[0238] The current frame prediction position information acquisition submodule obtains the prediction position information of the obstacle that appeared in the previous frame in the current frame based on the absolute velocity and absolute acceleration of the obstacle in the vehicle coordinate system of the previous frame, the corresponding rotation vector and translation matrix of the current frame.

[0239] In some embodiments, the vehicle's motion information in the previous frame includes the vehicle's velocity and acceleration in the vehicle's coordinate system of the previous frame; the motion information of at least one obstacle in the point cloud data acquired in the previous frame includes the absolute velocity and absolute acceleration of the obstacle in the vehicle's coordinate system of the previous frame; the current frame absolute velocity and acceleration acquisition submodule includes:

[0240] The current frame absolute velocity acquisition unit is used to calculate the absolute velocity of the obstacle that appeared in the previous frame in the vehicle coordinate system of the current frame based on the absolute velocity of the obstacle in the previous frame in the vehicle coordinate system of the previous frame, the absolute velocity of the vehicle in the vehicle coordinate system of the previous frame, and the corresponding rotation vector in the current frame.

[0241] The current frame absolute acceleration acquisition unit is used to calculate the absolute acceleration of each obstacle that appeared in the previous frame in the vehicle coordinate system of the current frame based on the absolute acceleration of the obstacle in the previous frame in the vehicle coordinate system of the previous frame, the absolute acceleration of the vehicle in the vehicle coordinate system of the previous frame, and the corresponding rotation vector in the current frame.

[0242] In some embodiments, the non-interested obstacle includes an invalid obstacle, and the non-interested obstacle determination module includes:

[0243] The first similarity calculation submodule is used to calculate multiple first similarities for each obstacle that appeared in the previous frame based on the actual location information of the obstacle that appeared in the current frame, the predicted location information of the obstacle that appeared in the previous frame in the current frame, and a preset similarity calculation formula.

[0244] The next frame prediction location information acquisition submodule is used to acquire the predicted location information of the first target obstacle in the next frame when there are multiple corresponding first similarities that are not greater than the first preset threshold; the first target obstacle is the obstacle that appeared in the previous frame when the first similarities are not greater than the first preset threshold.

[0245] The next frame position information acquisition submodule is used to acquire the actual position information of at least one obstacle in the point cloud data collected in the next frame;

[0246] The second similarity calculation submodule is used to calculate multiple corresponding second similarities based on the predicted position information of the first target obstacle in the next frame, the actual position information of all obstacles in the point cloud data collected in the next frame, and the preset similarity calculation formula.

[0247] The second target obstacle determination module is used to determine that a second target obstacle exists in at least one obstacle in the point cloud data collected in the next frame when there is a second similarity greater than a second preset threshold. The second target obstacle is a first target obstacle with a second similarity greater than a second preset threshold.

[0248] The third target obstacle determination module is used to determine the third target obstacle as an invalid obstacle when there are multiple corresponding second similarities that are not greater than the second preset threshold; the third target obstacle is a first target obstacle whose second similarity is not greater than the second preset threshold.

[0249] In some embodiments, the non-interested obstacle includes a non-dangerous obstacle, and the non-interested obstacle determination module further includes:

[0250] The fourth target obstacle determination submodule is used to determine that a fourth target obstacle exists in at least one obstacle in the point cloud data collected in the current frame when there is a first similarity greater than a first preset threshold. The fourth target obstacle is an obstacle that appeared in the previous frame when the first similarity is greater than the first preset threshold.

[0251] The lateral distance acquisition submodule is used to acquire a first lateral distance and a second lateral distance between the fourth target obstacle and the vehicle when it is determined that the fourth target obstacle exists in at least one obstacle in the point cloud data acquired in the current frame; the first lateral distance is the lateral distance between the fourth target obstacle and the vehicle in the previous frame; the second lateral distance is the lateral distance between the fourth target obstacle and the vehicle in the current frame.

[0252] The non-threatening obstacle determination submodule is used to determine the fourth target obstacle as a non-threatening obstacle when both the first lateral distance and the second lateral distance are greater than a distance threshold.

[0253] In some embodiments, the next frame predicted location information acquisition submodule includes:

[0254] The current frame motion information acquisition unit is used to acquire the vehicle's motion information in the current frame;

[0255] The next frame rotation vector and translation matrix acquisition unit is used to acquire the corresponding rotation vector and translation matrix of the next frame.

[0256] The absolute velocity and absolute acceleration acquisition unit is used to acquire the absolute acceleration and predicted absolute velocity of the first target obstacle in the vehicle coordinate system of the next frame based on the motion information of the first target obstacle in the previous frame, the motion information of the vehicle in the current frame, and the corresponding rotation vector in the next frame.

[0257] The predicted position information unit is used to predict the predicted position information of the first target obstacle in the next frame based on the absolute acceleration and predicted absolute velocity of the first target obstacle in the vehicle coordinate system of the next frame, the corresponding rotation vector and translation matrix of the next frame.

[0258] Each module in the aforementioned data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0259] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 9 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a data processing method.

[0260] The display unit of this computer device is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of this computer device can be a touch layer covering the display screen, or buttons, a trackball, or a touchpad set on the casing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0261] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0262] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0263] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0264] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0265] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0266] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0267] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0268] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A data processing method, characterized in that, The method is applied to a vehicle, the vehicle including an onboard 4D millimeter-wave radar, the onboard 4D millimeter-wave radar being used to collect point cloud data during autonomous driving; the method includes: For the current frame, the corresponding rotation vector and translation matrix are determined based on the vehicle's position and heading angle information in the current frame, the vehicle's position and heading angle information in the previous frame, and the vehicle's heading angle information in the initial frame. Obtain the motion information of the vehicle in the previous frame, and the motion information of at least one obstacle in the point cloud data collected in the previous frame; Based on the vehicle's motion information in the previous frame, the motion information of at least one obstacle in the point cloud data collected in the previous frame, and the corresponding rotation vector and translation matrix in the current frame, the predicted position information of the obstacle that appeared in the previous frame in the current frame is obtained. Obtain the actual location information of at least one obstacle in the point cloud data collected in the current frame; Based on the actual location information of obstacles appearing in the current frame and the predicted location information of obstacles appearing in the previous frame in the current frame, determine whether there are any non-interested obstacles among the obstacles appearing in the previous frame. If the obstacle exists, delete the corresponding point cloud data of the non-interested obstacle from the point cloud data collected in the previous frame.

2. The method according to claim 1, characterized in that, The step of determining the corresponding rotation vector and translation matrix for the current frame based on the vehicle's position and heading angle information in the current frame, the vehicle's position and heading angle information in the previous frame, and the vehicle's heading angle information in the initial frame includes: Based on the vehicle's heading angle information in the initial frame, the heading angle information in the previous frame, and the heading angle information in the current frame, the vehicle's first relative heading angle information and second relative heading angle information are obtained. Based on the vehicle's position information in the current frame and its position information in the previous frame, obtain the vehicle's relative position information; Based on the first relative heading angle information, obtain the corresponding rotation vector for the current frame; Based on the second relative heading angle information and the relative position information, the translation matrix corresponding to the current frame is obtained.

3. The method according to claim 1, characterized in that, The step of obtaining the predicted position information of the obstacle appearing in the previous frame in the current frame based on the vehicle's motion information in the previous frame, the motion information of at least one obstacle in the point cloud data collected in the previous frame, and the corresponding rotation vector and translation matrix in the current frame includes: Based on the vehicle's motion information in the previous frame, the motion information of at least one obstacle in the point cloud data collected in the previous frame, and the corresponding rotation vector in the current frame, the absolute velocity and absolute acceleration of the obstacle in the previous frame in the vehicle coordinate system of the current frame are obtained; the vehicle coordinate system is constructed based on the vehicle's position information in each frame, and there is a corresponding vehicle coordinate system for each frame. Based on the absolute velocity and absolute acceleration of the obstacle in the previous frame in the vehicle coordinate system of the current frame, and the corresponding rotation vector and translation matrix of the current frame, the predicted position information of the obstacle that appeared in the previous frame in the current frame is obtained.

4. The method according to claim 3, characterized in that, The vehicle's motion information in the previous frame includes the vehicle's velocity and acceleration in the vehicle's coordinate system in the previous frame; the motion information of at least one obstacle in the point cloud data collected in the previous frame includes the obstacle's velocity and acceleration in the vehicle's coordinate system in the previous frame; obtaining the absolute velocity and absolute acceleration of the obstacle in the vehicle's coordinate system in the current frame based on the vehicle's motion information in the previous frame, the motion information of at least one obstacle in the point cloud data collected in the previous frame, and the corresponding rotation vector in the current frame includes: Based on the speed of the obstacle in the previous frame in the vehicle coordinate system, the speed of the vehicle in the previous frame in the vehicle coordinate system, and the corresponding rotation vector in the current frame, calculate the absolute speed of the obstacle that appeared in the previous frame in the vehicle coordinate system in the current frame. Based on the acceleration of obstacles appearing in the previous frame in the vehicle's coordinate system in the previous frame, the acceleration of the vehicle in the vehicle's coordinate system in the previous frame, and the corresponding rotation vector in the current frame, calculate the absolute acceleration of each obstacle appearing in the previous frame in the vehicle's coordinate system in the current frame.

5. The method according to claim 1, characterized in that, The non-interested obstacles include invalid obstacles. The step of determining whether there are non-interested obstacles among the obstacles appearing in the previous frame, based on the actual position information of obstacles appearing in the current frame and the predicted position information of obstacles appearing in the previous frame in the current frame, includes: Based on the actual location information of the obstacles appearing in the current frame, the predicted location information of the obstacles appearing in the previous frame in the current frame, and the preset similarity calculation formula, calculate multiple first similarities for each obstacle appearing in the previous frame. In the case where there are multiple instances where the first similarity is not greater than the first preset threshold, the predicted position information of the first target obstacle in the next frame is obtained; the first target obstacle is the obstacle that appeared in the previous frame where the first similarity is not greater than the first preset threshold. Obtain the actual location information of at least one obstacle in the point cloud data collected in the next frame; Based on the predicted position information of the first target obstacle in the next frame, the actual position information of all obstacles in the point cloud data collected in the next frame, and the preset similarity calculation formula, calculate the corresponding multiple second similarities. If there is a second similarity greater than a second preset threshold, it is determined that the second target obstacle exists in at least one obstacle in the point cloud data collected in the next frame, and the second target obstacle is the first target obstacle with a second similarity greater than the second preset threshold; If there are multiple instances where the second similarity is not greater than the second preset threshold, the third target obstacle is determined to be an invalid obstacle; the third target obstacle is the first target obstacle where the second similarity is not greater than the second preset threshold.

6. The method according to claim 5, characterized in that, The non-interested obstacles include obstacles that pose no danger. The step of determining whether there are non-interested obstacles among the obstacles appearing in the previous frame, based on the actual position information of obstacles appearing in the current frame and the predicted position information of obstacles appearing in the previous frame in the current frame, further includes: If a first similarity greater than a first preset threshold exists, it is determined that the fourth target obstacle exists in at least one obstacle in the point cloud data collected in the current frame, wherein the fourth target obstacle is an obstacle that appeared in the previous frame where the first similarity is greater than the first preset threshold. If it is determined that the fourth target obstacle exists in at least one obstacle in the point cloud data collected in the current frame, the first lateral distance and the second lateral distance between the fourth target obstacle and the vehicle are obtained; the first lateral distance is the lateral distance between the fourth target obstacle and the vehicle in the previous frame; the second lateral distance is the lateral distance between the fourth target obstacle and the vehicle in the current frame. If both the first lateral distance and the second lateral distance are greater than the distance threshold, the fourth target obstacle is determined to be a non-threatening obstacle.

7. The method according to claim 5, characterized in that, The step of obtaining the predicted position information of the first target obstacle in the next frame includes: Obtain the vehicle's motion information in the current frame; Obtain the rotation vector and translation matrix corresponding to the next frame; Based on the motion information of the first target obstacle in the previous frame, the motion information of the vehicle in the current frame, and the corresponding rotation vector in the next frame, the absolute acceleration and predicted absolute velocity of the first target obstacle in the vehicle coordinate system in the next frame are obtained. Based on the absolute acceleration and predicted absolute velocity of the first target obstacle in the vehicle coordinate system of the next frame, and the corresponding rotation vector and translation matrix of the next frame, the predicted position information of the first target obstacle in the next frame is predicted.

8. A data processing apparatus, characterized in that, The device includes: The rotation vector and translation matrix determination module is used to determine the corresponding rotation vector and translation matrix for the current frame based on the vehicle's position information and heading angle information in the current frame, the vehicle's position information and heading angle information in the previous frame, and the vehicle's heading angle information in the initial frame. The motion information acquisition module is used to acquire the motion information of the vehicle in the previous frame, as well as the motion information of at least one obstacle in the point cloud data collected in the previous frame. The predicted position information acquisition module is used to acquire the predicted position information of the obstacle that appeared in the previous frame in the current frame based on the vehicle's motion information in the previous frame, the motion information of at least one obstacle in the point cloud data collected in the previous frame, and the corresponding rotation vector and translation matrix in the current frame. The current frame actual position information acquisition module is used to acquire the actual position information of at least one obstacle in the point cloud data collected in the current frame; The non-interest obstacle determination module is used to determine whether there are non-interest obstacles among the obstacles that appeared in the previous frame, based on the actual position information of the obstacles that appeared in the current frame and the predicted position information of the obstacles that appeared in the previous frame in the current frame. The point cloud data deletion module is used to delete the point cloud data corresponding to non-interested obstacles from the point cloud data collected in the previous frame, if they exist.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images