An automatic driving perception method based on three-dimensional point cloud data

Abstract

The application discloses an automatic driving perception method based on three-dimensional point cloud data, which comprises the following steps: S1, acquiring three-dimensional point cloud data; S2, processing the three-dimensional point cloud data and performing three-dimensional target detection by using an improved PV-RCNN++ network, which comprises the following steps: S21, voxelizing the point cloud data; S22, processing the voxel data by using a deformable sparse convolution; S23, stacking and aggregating multi-scale voxel features to generate a feature map, and using a region proposal network to generate a candidate region; S24, performing farthest point sampling on the candidate region; S25, encoding multi-scale voxel features by using a voxel set abstraction module and a spatial attention mechanism; S26, aggregating multi-scale voxel features, point cloud bird's eye view features and key point features by using a region of interest pooling module; S27, performing fine processing on the candidate region; S3, calculating the distance and speed of a target by using a deep learning network; S4, predicting a target motion trajectory by using Kalman filtering; and S5, outputting the target motion trajectory, so that the detection and prediction precision of various dynamic targets in automatic driving are improved.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to an autonomous driving perception method based on three-dimensional point cloud data. Background Technology

[0002] With the rapid development of autonomous driving technology, environmental perception systems have become a core component of autonomous vehicles. These systems collect information about the surrounding environment through various sensors (such as LiDAR, cameras, and millimeter-wave radar), process this data, and then identify, track, and analyze roads, obstacles, and other traffic participants, providing a basis for autonomous driving decisions.

[0003] LiDAR, as a crucial sensor, boasts extremely high accuracy and resolution, enabling the generation of high-quality 3D point cloud data. In autonomous driving, LiDAR not only provides geometric information about objects surrounding the vehicle but also offers reliable data for calculating the distance, position, and velocity of target objects. Especially in complex driving scenarios, LiDAR's strong adaptability to various environmental conditions (such as low light and inclement weather) has made it one of the primary sensors in many autonomous driving systems.

[0004] However, despite the wealth of spatial information that lidar can provide, efficiently and accurately extracting and analyzing the dynamic information (such as position and velocity) of target objects (such as vehicles or pedestrians) from the point cloud data acquired by lidar remains a technical challenge. Furthermore, traditional trajectory prediction algorithms assume that the target's motion strictly follows pre-defined physical laws (such as uniform speed or uniform acceleration). When the target suddenly changes lanes, brakes sharply, or turns, these models, lacking closed-loop error suppression, fail to capture nonlinear motion characteristics, causing the predicted trajectory to deviate from the actual path. Summary of the Invention

[0005] The purpose of this invention is to overcome the above-mentioned defects in the prior art and provide an autonomous driving perception method based on three-dimensional point cloud data, which can improve the detection and prediction accuracy of autonomous driving systems for various dynamic targets in the vehicle's surrounding environment.

[0006] To achieve the above objectives, the present invention provides an autonomous driving perception method based on three-dimensional point cloud data, which includes the following steps:

[0007] S1: Acquire 3D point cloud data of the surrounding environment using the LiDAR onboard the autonomous vehicle;

[0008] S2: Process the 3D point cloud data acquired by the LiDAR, and perform 3D target detection on the vehicle-mounted edge device using an improved PV-RCNN++ network. This improved PV-RCNN++ network adopts a point-voxel feature extraction method and includes the following sub-steps:

[0009] S21. Voxelize the collected point cloud data to transform the sparse point cloud data into a structure suitable for convolution operations;

[0010] S22. Use deformable sparse convolution to process voxel data and extract features from voxels;

[0011] S23. Stack and aggregate multi-scale voxel features to generate a bird's-eye view feature map, and use a region proposal network to generate candidate regions containing the target;

[0012] S24. Perform farthest point sampling within a certain neighborhood of the candidate region to extract the key points of the target;

[0013] S25. Encode multi-scale voxel features through voxel set abstraction modules and spatial attention mechanisms to capture location-sensitive information;

[0014] S26. Use the region of interest pooling module to aggregate multi-scale voxel features, point cloud bird's-eye view features, and key point features;

[0015] S27. Refine the candidate regions to improve the quality of the candidate boxes and generate the best 3D target prediction boxes;

[0016] S3: Calculate the distance and velocity of the target using the target key points and target prediction boxes obtained from the deep learning network;

[0017] S4: Predict the target's trajectory using Kalman filtering;

[0018] S5: Outputs the target trajectory and related information to the decision-making and control terminals of the autonomous vehicle.

[0019] Preferably, in step S1, the lidar emits a laser beam and receives the reflected signal, thereby measuring the return time of the laser beam and calculating the spatial coordinates of each laser point, ultimately forming a map of three-dimensional point cloud data of the surrounding environment.

[0020] Preferably, in sub-step S22 of step S2, the process of processing the deformable sparse convolutional voxelized data is as follows:

[0021] Assuming the voxelized tensor in step S21 is X, X(i+m,j+n) are elements in the input tensor, Y(i,j) is the output of the convolution operation, and K(m,n) are elements of the convolution kernel, where m and n are the row and column indices of the convolution kernel, respectively, the output of the convolution operation can be expressed as:

[0022]

[0023] Here, Δm(i,j) and Δn(i,j) are offsets related to the position (i,j), which are obtained through learnable parameters in the improved PV-RCNN++ network;

[0024] The voxel data is downsampled step by step through a multi-level deformable sparse convolutional network, and finally outputs a multi-scale sparse voxel feature map.

[0025] Preferably, in sub-step S25 of step S2, the spatial attention mechanism processing procedure is as follows:

[0026] Assume the feature map obtained after the deformable sparse convolutional layer in step S22 is C out It is the number of output channels, D' x ,D' y ,D' z If is the spatial dimension of the output feature map, then the output spatial attention map A can be represented as:

[0027] F avg =AvgPool(F out )

[0028] F max =MaxPool(F out )

[0029] F concat =concat(F avg ,F max )

[0030] A=σ(W*F concat )

[0031] Where AvgPool(·) represents the average pooling operation, MaxPool(·) represents the max pooling operation, concat(·) represents concatenating the outputs of each branch along the channel dimension, W is a 1×1×1 convolution kernel used to generate the spatial attention map, and σ(·) is the Sigmoid activation function to ensure that the output spatial attention map A is in the range [0,1].

[0032] Finally, the spatial attention map A is compared with the input feature map F. out Weighted summation

[0033] F' out =A⊙F out

[0034] Where ⊙ denotes element-wise multiplication, F' out It is the weighted output feature map.

[0035] Preferably, in step S3, the key points and prediction boxes obtained in step S2 are processed. First, the key points in the prediction boxes are extracted, and a target key point coordinate set (x1, y1, z1), (x2, y2, z2), ..., (x n ,y n ,z n ), where n is the number of keypoints contained in the prediction box, and the center point coordinates (x, y). center ,y center ,z center The formula for calculating ) is:

[0036]

[0037] Where, x i ,y i ,z i Let the coordinates be the coordinates of the i-th key point.

[0038] The distance to the target can be calculated by determining the Euclidean distance from the center point to a reference point. Assuming the coordinates of the reference point are (x0, y0, z0), the target distance D is:

[0039]

[0040] The velocity of a target is typically estimated by the change in the center point position between consecutive frames. Assuming we have two consecutive frames of data, we obtain the target's center point coordinates (x, y, t) at frame t and at frame t+1. center (t),y center (t),z center (t)) and (x center (t+1),y center (t+1),z center If (t+1)), then the changes in the target in the three directions are:

[0041] Δx=x center (t+1)-x center (t)

[0042] Δy=y center (t+1)-y center (t)

[0043] Δz=z center (t+1)-zcenter (t)

[0044] Therefore, the target's velocities in all directions are:

[0045]

[0046] Where Δt is the time interval between two consecutive frames.

[0047] The target's total velocity v total It is the resultant of three velocity components, which can be calculated using the following formula:

[0048]

[0049] Preferably, in step S4, the Kalman filter uses the target's historical position and velocity information to smooth and predict the target's trajectory. The Kalman filter includes a prediction step and an update step.

[0050] In the prediction step, Kalman filtering uses the target's dynamic model to predict the target state. Assuming the target's state includes position x and velocity v, the target's state vector is represented as:

[0051]

[0052] Where, x k It is the position of the target at time k, v k It is the velocity of the target at time k, a k It is the acceleration of the target at time k.

[0053] The state model of the target can be represented as:

[0054] x k =Fx k-1 +Bu k-1 +w k-1

[0055] Where F is the state transition matrix, B is the control input matrix, and u k-1 It is the control input of the system at time k-1, w k-1 It's process noise.

[0056] Predicted covariance matrix P k Updated to:

[0057] P k =FP k-1 F T +Q

[0058] Here, Q is the process noise covariance matrix, which reflects the uncertainty and noise of the model.

[0059] In the update step, the Kalman filter corrects the prediction results based on the actual observations, assuming that the target position observed at time k is z. k Then the observation model can be expressed as:

[0060] z k =Hx k +v k

[0061] Where H is the observation matrix, v k It is observation noise.

[0062] Based on the observed data and predicted state, the Kalman filter corrects the state estimate, and the formula for calculating the Kalman gain is as follows:

[0063] K k =P k H T HP k H T +R) -1

[0064] Where R is the observation noise covariance matrix,

[0065] Simultaneously update the state estimate x k The covariance matrix P k :

[0066] x k =x k +K k (z k -Hx k )

[0067] P k =(IK k H)P k

[0068] Where I is the identity matrix.

[0069] Preferably, in step S5, the on-board edge device integrates the results of the perception system and outputs the target motion trajectory and related information to the decision-making and control ends of the autonomous vehicle.

[0070] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0071] 1. This invention acquires high-precision 3D point cloud data using LiDAR, performs 3D target detection using an improved PV-RCNN++ network, and predicts the target's motion trajectory using a deep learning network combined with a Kalman filter algorithm, thereby achieving accurate estimation of the target's position, velocity, and future motion trajectory, thus improving the detection and prediction accuracy of autonomous driving.

[0072] 2. In 3D object detection using the improved PV-RCNN++ network, a deformable convolution mechanism is further employed to dynamically adjust the sampling points of the convolution kernel based on the local structure of the input data, enhancing the model's robustness to object deformation, rotation, and scale changes. The deformable mechanism enhances the modeling ability for complex geometric structures by dynamically adjusting sampling points, reducing redundant computation and adapting to high-dimensional data requirements. A spatial attention mechanism enables the model to automatically identify key regions (such as object edges and small targets) in sparse feature maps, enhancing its ability to capture effective information from sparse point clouds. Because voxel resolutions differ across levels, features extracted by sparse convolution typically exhibit multi-scale characteristics. Spatial attention, through weighted fusion of features at different scales, maintains a balance between shallow details and deep semantic information, resolving the feature conflict problem caused by fixed weights in traditional Feature Pyramid Networks (FPNs).

[0073] 3. This method combines point cloud 3D target detection with the Kalman filter algorithm. It leverages the rich spatial information of point cloud data to improve the accuracy of target position perception, while dynamically correcting the state estimate through the Kalman filter's prediction-update mechanism. Even if the target suddenly changes lanes, brakes sharply, or turns, the state estimate can be corrected in real time based on measurement errors, significantly enhancing adaptability to nonlinear target motion. This method effectively suppresses error accumulation, improves the stability and accuracy of trajectory prediction, and possesses high real-time performance, making it particularly suitable for trajectory prediction of dynamic targets in complex traffic scenarios.

[0074] 4. In summary, the autonomous driving perception method based on three-dimensional point cloud data provided by this invention can be widely applied to the environmental perception and decision-making systems of various autonomous vehicles. Through accurate target detection, distance and speed estimation, and trajectory prediction, it can provide more accurate environmental perception information for autonomous driving systems, effectively improve the detection and prediction accuracy of various dynamic targets in the vehicle's surrounding environment, and enhance the adaptability and safety of autonomous driving technology in complex and dynamic traffic environments. Attached Figure Description

[0075] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0076] Figure 1 This is a flowchart of an autonomous driving perception method based on three-dimensional point cloud data provided by the present invention;

[0077] Figure 2This is a schematic diagram of the structure of the improved PV-RCNN++ network in step S2 provided by the present invention.

[0078] The diagram includes: Detailed Implementation

[0079] The technical solution of this embodiment of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiment is one embodiment of the present invention, and not all embodiments thereof. Based on this embodiment of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0080] Please see Figures 1 to 2 The present invention provides an autonomous driving perception method based on three-dimensional point cloud data, which includes the following steps:

[0081] S1: Acquire 3D point cloud data: Acquire 3D point cloud data of the surrounding environment using the LiDAR on the autonomous vehicle;

[0082] In step S1, the lidar emits a laser beam and receives the reflected signal, which can accurately measure the return time of the laser beam and calculate the spatial coordinates of each laser point. Finally, a map of three-dimensional point cloud data of the surrounding environment is formed. This map of three-dimensional point cloud data contains detailed spatial coordinate information of the vehicle's surrounding environment, such as the position, shape and distance of obstacles, which can accurately reflect the relative position and shape of the target object.

[0083] S2: Target Detection and Localization: The 3D point cloud data collected by the LiDAR is processed, and the improved PV-RCNN++ network is used on the vehicle edge device to perform 3D target detection. The PV-RCNN++ network can automatically identify and locate target objects from the 3D point cloud data and generate the target's 3D bounding box, which includes the target's spatial location and category information.

[0084] Among them, 3D target detection involves identifying and locating surrounding targets, such as pedestrians, vehicles, and bicycles, from lidar point cloud data.

[0085] Existing 3D target detection methods can be categorized into point-based feature extraction methods, voxel-based feature extraction methods, and point-voxel-based feature extraction methods.

[0086] The improved PV-RCNN++ network employs a point-voxel feature extraction method. This approach offers several advantages: voxelization provides global spatial information about the target, enabling the improved PV-RCNN++ network to better capture the overall structure and morphology, exhibiting strong adaptability, especially when the target size and rotation change. Simultaneously, each point in the point cloud carries rich local details (such as edges and textures). By combining local features from the point cloud with global features from voxelization, the point-voxel method better preserves the target's detailed information. This method not only significantly reduces computational cost and storage usage but also effectively improves detection accuracy and robustness.

[0087] The structural principle of the improved PV-RCNN++ network is as follows: Figure 2 As shown, it includes the following sub-steps:

[0088] S21. Voxelize the collected point cloud data to transform the sparse point cloud data into a structure more suitable for convolution operations;

[0089] S22. Use deformable sparse convolution to process voxelized data and extract features from voxels;

[0090] In step S22, the deformable sparse convolution combines the flexibility of deformable convolution with the efficiency of sparse convolution to optimize the calculation process for different types of data. This not only extracts more comprehensive features but also captures better geometric structures.

[0091] The process of processing voxelized data using deformable sparse convolution is as follows:

[0092] Assuming the voxelized tensor in step S21 is X, X(i+m,j+n) are elements in the input tensor, Y(i,j) is the output of the convolution operation, and K(m,n) are elements of the convolution kernel, where m and n are the row and column indices of the convolution kernel, respectively, the output of the convolution operation can be expressed as:

[0093]

[0094] Here, Δm(i,j) and Δn(i,j) are offsets related to the position (i,j), which are obtained through learnable parameters in the improved PV-RCNN++ network;

[0095] Voxel data is downsampled step by step through a multi-level deformable sparse convolutional network (such as 4 layers of 3×3×3 convolutions) to finally output multi-scale sparse voxel feature maps. These features include the original resolution and different resolutions of 2×, 4× and 8× downsampling, which are used to preserve the multi-scale semantic information of the scene.

[0096] Step S22 introduces a deformable convolution mechanism, which enhances the ability to model complex geometric structures, reduces redundant calculations, and adapts to the needs of high-dimensional data.

[0097] By introducing learnable offsets, the sampling points of the convolution kernel can be dynamically adjusted according to the local structure of the input data, thereby enhancing the robustness of the model to object deformation, rotation, and scale changes.

[0098] The deformable mechanism reduces reliance on fixed grids by dynamically adjusting sampling points, further reducing redundant computations and improving inference speed while maintaining accuracy.

[0099] S23. Use a Region Proposal Network (RPN) to generate candidate regions that may contain the target;

[0100] S24. Perform farthest point sampling (FPS) within a certain neighborhood of the candidate region to extract the key points of the target;

[0101] S25. Encode multi-scale voxel features through voxel set abstraction modules and spatial attention mechanisms to capture position-sensitive features;

[0102] In step S25, the spatial attention mechanism is applied to the feature map after deformable sparse convolution to enhance the expressive power of deformable sparse convolution features. It focuses on important spatial locations in local regions through spatial attention, thereby strengthening key features in sparse regions.

[0103] Assume the feature map obtained after the deformable sparse convolutional layer in step S22 is C out It is the number of output channels, D' x ,D' y ,D' z If is the spatial dimension of the output feature map, then the output spatial attention map A can be represented as:

[0104] F avg =AvgPool(F out )

[0105] F max =MaxPool(F out )

[0106] F concat =concat(F avg ,F max )

[0107] A=σ(W*F concat )

[0108] Where AvgPool(·) represents the average pooling operation, MaxPool(·) represents the max pooling operation, concat(·) represents concatenating the outputs of each branch along the channel dimension, W is a 1×1×1 convolution kernel used to generate the spatial attention map, and σ(·) is the Sigmoid activation function to ensure that the output spatial attention map A is in the range [0,1].

[0109] Finally, the spatial attention map A is compared with the input feature map F. out Weighted summation

[0110] F' out =A⊙F out

[0111] Where ⊙ denotes element-wise multiplication, F' out It is the weighted output feature map;

[0112] Therefore, S25 utilizes a spatial attention mechanism to enable the model to automatically identify key regions (such as object edges and small targets) in sparse feature maps, enhancing its ability to capture effective information in sparse point clouds. For example, attention weights can suppress background noise, highlight local features of foreground targets, and thus improve detection accuracy.

[0113] Because different levels of voxel resolutions are different, the features extracted by sparse convolution usually have multi-scale characteristics. Spatial attention can solve the feature conflict problem caused by fixed weights in the traditional feature pyramid (FPN) by weighted fusion of features at different scales, preserving the balance between shallow details and deep semantic information.

[0114] S26. Use the Region of Interest (RoIPooling) module to aggregate multi-scale voxel features, point cloud bird's-eye view features, and key point features;

[0115] S27. Refine the candidate regions to improve the quality of the candidate boxes and generate the best target detection boxes;

[0116] S3: Calculate the distance and velocity of the target using the target key points and target prediction boxes obtained from the deep learning network;

[0117] In step S3, the key points and prediction boxes obtained in step S2 are processed. First, the key points in the prediction boxes are extracted, and a set of target key point coordinates (x1, y1, z1), (x2, y2, z2), ..., (x n ,y n ,z n ), where n is the number of keypoints contained in the prediction box, and the center point coordinates (x, y).center ,y center ,z center The formula for calculating ) is:

[0118]

[0119] Where, x i ,y i ,z i Let the coordinates be the coordinates of the i-th key point.

[0120] The distance to the target can be calculated by taking the Euclidean distance from the center point to a reference point (such as the camera or the origin of the world coordinate system). Assuming the coordinates of the reference point are (x0, y0, z0), the distance D to the target is:

[0121]

[0122] The velocity of a target is typically estimated by the change in the center point position between consecutive frames. Assuming we have two consecutive frames of data, we obtain the target's center point coordinates (x, y, t) at frame t and at frame t+1. center (t),y center (t),z center (t)) and (x center (t+1),y center (t+1),z center If (t+1)), then the changes in the target in the three directions are:

[0123] Δx=x center (t+1)-x center (t)

[0124] Δy=y center (t+1)-y center (t)

[0125] Δz=z center (t+1)-z center (t)

[0126] Therefore, the target's velocities in all directions are:

[0127]

[0128] Where Δt is the time interval between two consecutive frames.

[0129] The target's total velocity v total It is the resultant of three velocity components, which can be calculated using the following formula:

[0130]

[0131] S4: Predict the target's trajectory using a Kalman filter;

[0132] Kalman filtering, a recursive optimal estimation algorithm, is widely used in target tracking and motion prediction, especially in autonomous driving and robot navigation, to predict the future state of a target based on observed data (such as position and velocity). Kalman filtering effectively handles system noise and uncertainty, improving prediction accuracy by integrating historical observation data.

[0133] Specifically, Kalman filtering utilizes the target's historical position and velocity information to smooth and predict the target's trajectory. Kalman filtering includes a prediction step and an update step, wherein...

[0134] In the prediction step, Kalman filtering uses the target's dynamic model to predict the target state. Assuming the target's state includes position x and velocity v, the target's state vector can be represented as:

[0135]

[0136] Where, x k It is the position of the target at time k, v k It is the velocity of the target at time k, a k It is the acceleration of the target at time k.

[0137] The state model of the target can be represented as:

[0138] x k =Fx k-1 +Bu k-1 +w k-1

[0139] Where F is the state transition matrix, B is the control input matrix, and u k-1 It is the control input of the system at time k-1, w k-1 It's process noise.

[0140] The predicted covariance matrix P k Updated to:

[0141] P k =FP k-1 F T +Q

[0142] Where Q is the process noise covariance matrix, reflecting the uncertainty and noise of the model.

[0143] In the update step, the Kalman filter corrects the prediction results based on the actual observations, assuming that the target position observed at time k is z. k Then the observation model can be expressed as:

[0144] z k =Hx k +v k

[0145] Where H is the observation matrix, v k It is observation noise.

[0146] Based on the observed data and predicted state, the Kalman filter corrects the state estimate, and the formula for calculating the Kalman gain is as follows:

[0147] K k =P k H T HP k H T +R) -1

[0148] Where R is the observation noise covariance matrix,

[0149] Simultaneously update the state estimate x k The covariance matrix P k :

[0150] x k =x k +K k (z k -Hx k )

[0151] P k =(IK k H)P k

[0152] Where I is the identity matrix.

[0153] Based on the updated state estimate, the Kalman filter can continuously predict the target's trajectory. Through multiple predictions and updates, the Kalman filter can provide accurate trajectory predictions, thus supporting the decision-making and planning of autonomous driving systems.

[0154] S5: Output the target trajectory and related information to the decision-making and control terminals of the autonomous vehicle. Specifically, in step S5, the on-board edge device integrates the results of the perception system and outputs the target trajectory and related information to the decision-making and control terminals of the autonomous vehicle.

[0155] In summary, an autonomous driving perception method based on 3D point cloud data can be widely applied to the environmental perception and decision-making systems of various autonomous vehicles. Through accurate target detection, distance and speed estimation, and trajectory prediction, it can provide more accurate environmental perception information for autonomous driving systems, effectively improve the detection and prediction accuracy of various dynamic targets in the vehicle's surrounding environment, and enhance the adaptability and safety of autonomous driving technology in complex and dynamic traffic environments.

[0156] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An autonomous driving perception method based on 3D point cloud data, characterized in that, It includes the following steps: S1: Acquire 3D point cloud data of the surrounding environment using the LiDAR onboard the autonomous vehicle; S2: Process the 3D point cloud data acquired by the LiDAR, and perform 3D target detection on the vehicle-mounted edge device using an improved PV-RCNN++ network. This improved PV-RCNN++ network adopts a point-voxel feature extraction method and includes the following sub-steps: S21. Voxelize the collected point cloud data to transform the sparse point cloud data into a structure suitable for convolution operations; S22. Use deformable sparse convolution to process voxel data and extract features from voxels; S23. Stack and aggregate multi-scale voxel features to generate a bird's-eye view feature map, and use a region proposal network to generate candidate regions containing the target; S24. Perform farthest point sampling within a certain neighborhood of the candidate region to extract the key points of the target; S25. Encode multi-scale voxel features through voxel set abstraction modules and spatial attention mechanisms to capture location-sensitive information; S26. Use the region of interest pooling module to aggregate multi-scale voxel features, point cloud bird's-eye view features, and key point features; S27. Refine the candidate regions to improve the quality of the candidate boxes and generate the best 3D target prediction boxes; S3: Calculate the distance and velocity of the target using the target key points and target prediction boxes obtained from the deep learning network; S4: Predict the target's trajectory using Kalman filtering; S5: Outputs the target trajectory and related information to the decision-making and control terminals of the autonomous vehicle; In step S1, the lidar emits a laser beam and receives the reflected signal, which allows it to measure the return time of the laser beam and calculate the spatial coordinates of each laser point, ultimately forming a map of three-dimensional point cloud data of the surrounding environment. In sub-step S22 of step S2, the process of processing the deformable sparse convolutional voxelized data is as follows: Assume the tensor after voxelization in step S21 is , These are the elements in the input tensor. It is the output of the convolution operation. Let m and n be the elements of the convolution kernel, respectively, and m and n be the row and column indices of the kernel. Then the output of the convolution operation can be represented as: ， in, and Is with position The relevant offsets are obtained through learnable parameters in the improved PV-RCNN++ network; Voxel data is downsampled step by step through a multi-level deformable sparse convolutional network, and finally outputs a multi-scale sparse voxel feature map. In sub-step S25 of step S2, the spatial attention mechanism processes as follows: Assume the feature map obtained after the deformable sparse convolutional layer in step S22 is , It is the number of output channels. If the spatial dimension of the output feature map is [the dimension of the output feature map], then the output spatial attention map is [the spatial attention map]. It can be represented as: ， ， ， ， in This indicates the average pooling operation. This indicates a max pooling operation. This means concatenating the outputs of each branch along the channel dimension. It is The convolutional kernel is used to generate the spatial attention map. Activation function to ensure the output spatial attention map exist Within the range; Finally, the spatial attention map With input feature map Weighted summation ， in This indicates element-wise multiplication. It is the weighted output feature map; In step S3, the key points and prediction boxes obtained in step S2 are processed. First, the key points in the prediction boxes are extracted, and a set of target key point coordinates is established. ,in It is the number of keypoints contained in the prediction frame, and the coordinates of the center point. The calculation formula is: ， in, Represented as the first The coordinates of the key points The distance to the target can be calculated by taking the Euclidean distance from the center point to a reference point, assuming the coordinates of the reference point are... Then the distance to the target for: ， The velocity of a target is typically estimated by the change in the center point position between consecutive frames. Assuming we have two consecutive frames of data, in the first... Frame and the The center point coordinates of the target are obtained from each frame. and Then the changes in the target in the three directions are: ， Therefore, the target's velocities in all directions are: ， in, It is the time interval between two consecutive frames. Total speed of the target It is the resultant of three velocity components, which can be calculated using the following formula: ； In step S4, the Kalman filter uses the target's historical position and velocity information to smooth and predict the target's trajectory. The Kalman filter includes a prediction step and an update step. In the prediction step, the Kalman filter uses a dynamic model of the target to predict the target state, assuming that the target state includes position. and speed The state vector of the target is then expressed as: ， in, The goal is to Location at any given moment The goal is to The speed of time The goal is to acceleration at any moment The state model of the target can be represented as: ， in, It is the state transition matrix. It is a control input matrix. It is a system Time-based control input, It's process noise. Predicted covariance matrix Updated to: ， in, It is the process noise covariance matrix, which reflects the uncertainty and noise of the model; In the update step, the Kalman filter corrects the prediction results based on the actual observations, assuming that... The target position observed at any time is Then the observation model can be expressed as: ， in, It is the observation matrix. It is observation noise. Based on the observed data and predicted state, the Kalman filter corrects the state estimate, and the formula for calculating the Kalman gain is as follows: ， in, It is the observation noise covariance matrix. Simultaneously update the state estimate Covariance Matrix : ， ， in, It is the identity matrix; In step S5, the on-board edge device integrates the results of the perception system and outputs the target motion trajectory and related information to the decision-making and control terminals of the autonomous vehicle.

Images