Point cloud dynamic vehicle detection method in unmanned driving scene

Abstract

The present application belongs to the field of unmanned driving, and proposes a point cloud dynamic vehicle detection method in an unmanned driving scene, which comprises: step 1. Dynamic vehicle labeling and construction of a training data set; step 2. Building a point cloud dynamic vehicle detection model and running a point cloud dynamic vehicle detection algorithm. The method of the present application provides a method that can directly perceive the motion of vehicles in an unmanned driving scene without the need for a large amount of prior knowledge, so that it can accurately determine whether the relative motion of the vehicle and the laser radar causes point cloud distortion, thereby improving the accuracy of intelligent perception of unmanned vehicles, and is expected to break through the obstacle of intelligent decision-making of unmanned motion behavior.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving, specifically to a point cloud dynamic vehicle detection method in autonomous driving scenarios. Background Technology

[0002] The safe operation of self-driving cars relies on a robust perception system. Current self-driving technology depends on sensors to perceive the surrounding environment, primarily including cameras, LiDAR, and ultrasonic radar. However, different self-driving companies use different sensors in significantly different ways. Many companies prefer to perceive the environment through the fusion of multiple sensors, as this approach of using multiple sensors to compensate for each other's weaknesses to handle most scenarios seems to be a superior choice.

[0003] LiDAR, as a crucial perception sensor for autonomous vehicles, is widely used by many companies such as Baidu, Uber, Waymo, and Toyota. LiDAR scans the surrounding environment by emitting laser beams and receiving the reflected beams from objects in the environment. It then calculates the distance from the scanned object to the LiDAR, generating point cloud data containing the object's position information. Currently, LiDAR data collected by autonomous vehicles is widely used in 3D perception, 3D localization, and 3D reconstruction. However, some problems remain to be solved when applying LiDAR to autonomous driving scenarios. For example, while LiDAR can obtain accurate object position information, it cannot directly perceive the movement of objects within the scene.

[0004] Detecting dynamic targets, primarily moving vehicles, in point clouds collected in autonomous driving scenarios is a crucial part of the perception system and has extremely high research value.

[0005] Segmentation-based point cloud dynamic object detection methods are mainly divided into traditional algorithm-based and deep learning-based methods. Traditional algorithm-based methods primarily construct feature descriptors to segment the state of point cloud data points, and then use clustering algorithms to separate static and dynamic targets: They distinguish dynamic point cloud sets from static ones by using viewpoint feature histograms to describe the point cloud; and remove dynamic targets using height descriptors. This method first divides the point cloud into grids, calculates a height feature descriptor for each grid to represent the point cloud distribution, and then compares the values ​​of the feature descriptors in the same region of different point clouds. If the values ​​are less than a threshold, the corresponding grid is marked as a suspected dynamic region, and the ground is fitted into the suspected dynamic region to obtain a point cloud map containing only static objects. The system detects point pairings between point clouds, identifies erroneous pairings, and extracts dynamic targets using a clustering algorithm. It extracts three frames from the point cloud, comparing the first two and verifying the last. If a point identified as potentially moving in the first two frames is crossed in the last frame, it is considered a dynamic point, and dynamic targets are segmented using a region growing clustering algorithm. During 3D reconstruction, point-to-edge and point-to-surface feature residuals are used as objective equations for multi-objective optimization. When iteratively calculating these residuals, the first 20% of residuals are ignored to eliminate the impact of dynamic targets on point cloud registration. An improved Random Sample Consensus (RANSAC) algorithm is used to detect and track dynamic targets. Finally, a particle filter-based approach is used to filter the point cloud, segment moving objects, and measure their velocities.

[0006] Deep learning-based methods significantly improve detection performance by constructing models to detect the motion of objects in point clouds: They detect scene flow between two point cloud frames and then segment dynamic targets using object detection or clustering algorithms; they use network learning methods that fuse point cloud semantic features to predict object motion in the point cloud by incorporating other semantic features into the prediction results; they eliminate dynamic blurring caused by LiDAR during point cloud acquisition by using static map evolution methods based on multi-resolution distance images; they improve RANSAC using deep learning algorithms to detect objects and distinguish their motion states; and they construct a spherical semantic map and segment the point cloud using the RangeNet++ network, filtering out dynamic objects.

[0007] In summary, although there has been extensive research in the field of point cloud dynamic vehicle detection, the following problems still exist when applied to the field of autonomous driving:

[0008] (1) Requires a lot of prior knowledge: Existing dynamic vehicle detection algorithms rely on feature descriptors or point cloud semantic features for judgment, and most of them are for a specific scenario, requiring a lot of mathematical modeling and preprocessing work before detection.

[0009] (2) The unmanned point cloud dataset has difficulty in labeling dynamic vehicles: The current unmanned point cloud dataset has difficulty in labeling dynamic vehicles, and lacks labels or ground truth for the dynamic vehicle detection problem in point clouds.

[0010] (3) Inability to obtain high-precision output directly: Existing deep learning-based methods often require calculating a feature or state of a moving target in the point cloud before inferring the motion state of its corresponding point cloud data points, such as scene flow features. The model cannot directly obtain the motion state of each data point, resulting in continuous accumulation of errors and low accuracy of the output results. Summary of the Invention

[0011] To address the above problems, this invention discloses a point cloud dynamic vehicle detection method for autonomous driving scenarios. First, it proposes a method for generating point cloud data and dynamically labeling vehicles in autonomous driving scenarios through simulation. Based on this, a point cloud dynamic vehicle detection method based on feature embedding is presented, and ablation and comparative experiments are conducted. Experimental results show that this method effectively detects the motion state of vehicles in the point cloud of autonomous driving scenarios, and its prediction accuracy is higher than other methods. Therefore, it can directly perceive the motion of vehicles in autonomous driving scenarios without relying on a large amount of prior knowledge, enabling accurate determination of whether point cloud distortion occurs due to the relative motion between the vehicle and the lidar, thereby improving the accuracy of intelligent perception for autonomous vehicles.

[0012] The technical solution of the present invention specifically includes the following steps:

[0013] Step 1. Dynamic vehicle annotation and construction of training dataset

[0014] Step 2. Build a point cloud dynamic vehicle detection model and run the point cloud dynamic vehicle detection algorithm.

[0015] Step 2.1 Data Preprocessing Module

[0016] Step 2.2 Feature coding layer for feature extraction of point cloud

[0017] Step 2.3 Feature Embedding Layer

[0018] Step 2.4 Feature Decoding Layer

[0019] Step 2.5 Output Layer

[0020] Step 2.6 Design the loss function

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] This invention discloses a point cloud dynamic vehicle detection method in an autonomous driving scenario. It provides a method that can directly perceive the movement of vehicles in an autonomous driving scenario without relying on a large amount of prior knowledge. This enables accurate determination of whether point cloud distortion occurs due to the relative motion between the vehicle and the lidar, thereby improving the accuracy of intelligent perception of autonomous vehicles and is expected to overcome the obstacles to intelligent decision-making in autonomous driving motion behavior. Attached Figure Description

[0023] Figure 1 A schematic diagram illustrating the dynamic vehicle annotations in the simulation data.

[0024] Figure 2 This describes the process of a point cloud dynamic vehicle detection model.

[0025] Figure 3 This is a schematic diagram of the experimental data downsampling.

[0026] Figure 4 This describes the specific process for downsampling.

[0027] Figure 5 This is a flowchart of a point cloud dynamic vehicle detection algorithm.

[0028] Figure 6 The image shows the truth plot (top) and the prediction plot (bottom) when the downsampling ratio is 0%.

[0029] Figure 7 The images show the true value (top) and the prediction (bottom) when the downsampling ratio is 60%.

[0030] Figure 8 The training loss varies with the period.

[0031] Figure 9 The training accuracy varies over time.

[0032] Figure 10 For comparison of ablation experiments.

[0033] Figure 11 The accuracy performance of various dynamic vehicle detection methods and FEMD is presented.

[0034] Figure 12 The performance of various dynamic vehicle detection methods and FEMD in terms of recall rates.

[0035] Figure 13 The accuracy of various dynamic vehicle detection methods and FEMD is evaluated.

[0036] Figure 14 This is a visualization of the FEMD method for predicting stationary vehicles.

[0037] Figure 15 This is a visualization of the FEMD method for predicting moving vehicles.

[0038] Figure 16 This is a flowchart of the present invention. Detailed Implementation

[0039] The technical solutions provided in this application will be further described below with reference to specific embodiments and accompanying drawings. The advantages and features of this application will become clearer from the following description.

[0040] This invention specifically includes the following three aspects:

[0041] Step 1. Dynamic vehicle annotation and construction of training dataset

[0042] Step 2. Build a point cloud dynamic vehicle detection model and run the point cloud dynamic vehicle detection algorithm.

[0043] Step 2.1 Data Preprocessing Module

[0044] Step 2.2 Feature coding layer for feature extraction of point cloud

[0045] Step 2.3 Feature Embedding Layer

[0046] Step 2.4 Feature Decoding Layer

[0047] Step 2.5 Output Layer

[0048] Step 2.6 Design the loss function

[0049] Step 3. Simulation experiment verification

[0050] Details are as follows:

[0051] Step 1: Dynamic vehicle annotation and construction of training dataset

[0052] Existing point cloud vehicle detection datasets often only label the approximate range of moving vehicles, such as detection boxes, and cannot distinguish different vehicle instances based on their motion state. This invention labels point cloud data points in a simulated point cloud dataset with different labels based on their motion state, serving as the dataset for subsequent training and simulation experiments.

[0053] Based on the simulation parameters, we know the position and speed of the LiDAR and vehicles in the scene at the start of collecting a frame of data.

[0054] Based on the principle of lidar data acquisition, a data point in the point cloud originates from reflection from the object's surface. The time it takes for the laser beam to travel from the lidar to the object's surface and back to the lidar is negligible. Even if the lidar is in motion, without considering noise and instrument errors, the acquired original point cloud is uniformly distributed in both the vertical and horizontal directions. A specific data point in the point cloud... The size can be expressed by equation (1):

[0055] (1)

[0057] in, For data points in a point cloud, This refers to the scanning resolution of the lidar. This is the unit vector of the laser beam corresponding to the point cloud; The distance from the surface of the object being scanned to the lidar; Data points for LiDAR The number of columns already scanned; To indicate the scanning resolution of the lidar; This represents the vertical field of view angle of the laser beam corresponding to this data point.

[0058] As shown in the above formula, with the lidar position as the origin of the three-dimensional coordinate system, the vector direction from the origin to the point cloud data point is determined only by the lidar's resolution, vertical field of view, and the current number of scanning columns. Since the lidar's resolution and vertical field of view are determined solely by its physical characteristics, the direction of the laser beam is fixed relative to the lidar. To determine whether a point is formed by reflection from the surface of a specified object, it is only necessary to know the object's position relative to the lidar when the laser beam was emitted.

[0059] For each cloud data point The acquisition time is calculated using coordinates. Then, the relative positions of the lidar and the vehicle are calculated based on their initial positions and speeds. Next, it is determined whether the corresponding laser beam emitted by the lidar passes through the vehicle. Labels are then added to the corresponding point cloud data points to serve as the dataset for subsequent processing.

[0060] In autonomous driving scenarios, the dynamic target is the vehicle traveling on the road. While labeling dynamic vehicles, static vehicles are also labeled using other tags. For each data point in each frame of the point cloud, this invention uses 0, 1, and 2 to distinguish which motion state the object reflects, as shown in equation (2):

[0061] (2)

[0062] in, Point cloud frames The corresponding tag set; Point cloud frames Mid-data points The corresponding tag, when When, it represents a data point. It is reflected from stationary objects other than vehicles, when When, it represents a data point. Reflected by moving vehicles, when When, it represents a data point. It was reflected from a stationary vehicle.

[0063] Example of a point cloud after annotation of simulation data: Figure 1 As shown, green represents moving vehicles, red represents stationary vehicles, and cyan represents the stationary surrounding environment.

[0064] To enrich the data format, this invention generates differentiated data by adjusting the position and speed of vehicles and the position and speed of lidar in the scene.

[0065] When the lidar is stationary, the vehicle's motion relative to the lidar is entirely determined by its own motion. This invention assigns different speeds to different vehicle models, while also including some stationary vehicles in the scene, to train the model's ability to distinguish between stationary and moving vehicles.

[0066] When the lidar moves, the vehicle's motion relative to the lidar is determined by both its own motion and the lidar's motion. This invention also assigns different speeds to vehicle models in the scene. Furthermore, in some scenarios, this invention maintains part of the vehicle moving at the same speed as the lidar, keeping the vehicle stationary relative to the lidar, thus training the model's ability to distinguish between relative and absolute stillness.

[0067] Step 2: Build a point cloud dynamic vehicle detection model and FEMD detection algorithm

[0068] Existing deep learning-based point cloud dynamic vehicle detection methods often detect scene flow information in the point cloud and then use post-processing such as threshold segmentation to obtain the motion state of each data point in the point cloud, resulting in low detection accuracy. To address this issue, this invention proposes a feature embedding-based point cloud dynamic vehicle detection method (FEMD). It extracts two consecutive frames of point clouds as input, fuses the features of the two frames in a feature embedding layer after feature extraction, and directly outputs the motion state prediction results of all data points in the first frame of the point cloud after feature decoding and output layer processing. The network structure of the model is as follows: Figure 2 As shown.

[0069] from Figure 2As can be seen, the point cloud dynamic vehicle detection system built in this invention consists of a model input layer, a data preprocessing module, four feature encoding layers, a feature embedding layer, four feature decoding layers, and a model output layer.

[0070] The model inputs two consecutive frames of point clouds. After data preprocessing, a feature encoding layer and a feature embedding layer are used to extract depth feature information at different scales. This depth feature information is then input into the feature decoding layer to restore it to the same dimension as the first frame of point cloud. During the decoding process, the output features of the feature encoding layer are fused to help recover the original image information. Finally, the decoded features are output through the softmax function to output the state corresponding to each point cloud data point in the first frame of point cloud.

[0071] The model comprises four feature encoding layers, each consisting of sampling, region aggregation, MLP, and max pooling modules. Its main function is to extract features from the point cloud to obtain higher-dimensional features. Specifically:

[0072] The feature encoding layer 1 has 1024 sampling points and 16 selected center points. The input is two point cloud data from the model input layer, and the output is two 64-dimensional features.

[0073] The number of sampling points in feature coding layer 2 is 256, the number of selected center points is 16, the input is the two features output from feature coding layer 1, and the output is two 128-dimensional features;

[0074] The feature encoding layer 3 has 64 sampling points and 8 selected center points. The input is the fused feature output from the feature embedding layer, and the output is a 256-dimensional feature.

[0075] The number of sampling points in feature coding layer 4 is 16, the number of selected center points is 8, the input is the feature output from feature coding layer 3, and the output is a 512-dimensional feature.

[0076] The model comprises four feature decoding layers, each consisting of an upsampling module and a unit feature extraction module. Its function is to reconstruct the point cloud from the extracted high-dimensional features and determine its motion state through upsampling. During the upsampling process, the output features of each feature encoding layer are fused to help better recover the original data information. Specifically:

[0077] The input to feature decoding layer 1 is the four output features of feature encoding layer 4;

[0078] The input to feature decoding layer 2 is obtained by adding the output features of feature decoding layer 1, feature embedding layer, and feature coding layer 3;

[0079] The input to feature decoding layer 3 is obtained by adding the first output features of feature decoding layer 2 and feature coding layer 2;

[0080] The input to feature decoding layer 4 is obtained by adding the first output features of feature decoding layer 3 and feature encoding layer 1. The output of all feature decoding layers is a 256-dimensional feature.

[0081] The model includes a feature embedding layer, which embeds and fuses the features extracted from two point cloud frames by the feature encoding layer. This fused feature data from the two branches is then used by the subsequent feature encoding layer for further deep feature extraction. The input to the feature embedding layer is the two features output from feature encoding layer 2, and the output is a 128-dimensional feature.

[0082] The model output is for A dimensional vector, where The first frame contains the number of points in the point cloud. Each column vector represents the predicted value of a data point that is stationary (excluding stationary vehicles), a data point that is moving (corresponding to a vehicle), and a data point that is stationary (corresponding to a vehicle).

[0083] Details are as follows:

[0084] (1) The data preprocessing module runs the following algorithm:

[0085] Step 1.1 Point Cloud Denoising

[0086] Point cloud noise mainly consists of two parts: outliers that deviate from the object surface and offsets of point cloud data points relative to the real data. This invention eliminates point cloud noise using PointCleanNet. The method first evaluates each point in the point cloud, selects outliers that deviate from the overall point cloud data based on their relationship with surrounding points, and removes these outliers from the point cloud. Then, it estimates the offset values ​​of the remaining points and reconstructs the point cloud based on the estimated offsets to remove point cloud noise. The denoising process is shown in formula (3):

[0087] (3)

[0088] in Point clouds for noise removal; The original point cloud contains noise; The set of outliers in the point cloud; This represents the estimated point cloud offset value.

[0089] Step 1.2 Data Downsampling

[0090] In autonomous driving scenarios, the surrounding environment often contains many stationary objects such as houses, trees, and roads. These objects constitute a large proportion of a point cloud frame, while moving vehicles account for a relatively small proportion and are mainly concentrated within the road area. This leads to a significant difference in the proportion of positive samples (dynamic objects) and negative samples (static objects) in a point cloud frame, affecting the model's classification accuracy. In the original simulated point cloud dataset, stationary point cloud data points account for more than 90%. Under such a sample distribution ratio, machine learning prediction models tend to predict the majority set (stationary data points), while the minority set (moving data points) may be treated as noise or outliers, resulting in low model classification accuracy or even failure to make effective judgments.

[0091] Therefore, in the experiment of this invention, the simulated point cloud is downsampled to remove some static data point samples such as houses and road surfaces on both sides of the road, so as to increase the proportion of dynamic data points in the point cloud and achieve the purpose of balancing positive and negative samples.

[0092] To ensure the integrity of vehicles and roads in the point cloud, this invention deletes a certain number of stationary point cloud data points perpendicular to the road direction inwards. An example of downsampling is shown below. Figure 3 As shown.

[0093] Specific steps for data downsampling processing (detailed process as follows) Figure 4 (as shown)

[0094] Step 1.2.1 Input a denoised point cloud frame containing motion state labels. and downsampling ratio .

[0095] Step 1.2.2 Initialize the set of data points to be removed Each time, the proportion of the point cloud set span perpendicular to the road direction is removed. ,counter and the downsampled point cloud .

[0096] Step 1.2.3 Calculate the point cloud Number of midpoints and point clouds perpendicular to the road direction span .

[0097] Step 1.2.4 If The number of data points in the mid-point cloud is less than If yes, proceed to step 1.2.5; otherwise, proceed to step 1.2.10.

[0098] Step 1.2.5 will convert the point cloud Divide the remaining point cloud along the length perpendicular to the road direction. The span from the boundary in the perpendicular direction of the road is The point cloud is divided into sets .

[0099] Step 1.2.6 If the point cloud set If no data point is marked as a dynamic point, proceed to step 1.2.7; otherwise, proceed to step 1.2.8.

[0100] Step 1.2.7 Remove In point cloud collection The data points in the point cloud are used to create a collection of point clouds. Merging Sets Then reset the counter to 0 and proceed to step 1.2.4.

[0101] Step 1.2.8 Halve the size and set the counter Increase by 1.

[0102] Step 1.2.9 If the size of the calculator is equal to 4, exit the loop and go to step 1.2.10; otherwise, go to step 1.2.4.

[0103] Step 1.2.10 Output the downsampled point cloud .

[0104] (2) Feature coding layer

[0105] The main function of the feature encoding layer is to extract features from the point cloud to obtain higher-dimensional features. The feature encoding layer mainly consists of three modules: (2.1) sampling module; (2.2) region aggregation module; (2.3) PointNet feature extraction module.

[0106] (2.1) Sampling Module

[0107] Raw point clouds often contain tens of thousands or even hundreds of thousands of data points. Direct processing would require massive computation; therefore, sampling is necessary to reduce model complexity while preserving point cloud features. The Farthest Point Sampling (FPS) algorithm is used to sample the point cloud. This algorithm ensures that the sampled points cover as much of the point cloud space as possible, which helps preserve the distribution and features of the points. The algorithm steps are as follows:

[0108] Step 2.1.1 Randomly select a point from the point cloud and add it to the sampling set. ;

[0109] Step 2.1.2 Never Select distance from the remaining point cloud Add the point with the furthest distance from all points. Repeat this process until a sufficient number of points have been collected.

[0110] (2.2) Regional Aggregation Module

[0111] The region aggregation module uses the point selected by the sampling module as the center and employs the K-nearest neighbor (KNN) algorithm to select regions within a spherical region of radius r. If the number of points in the region is insufficient Then, randomly select points within the area to copy, so that the number of collected points is... The region aggregation module is mainly for facilitating the extraction of local features from point clouds.

[0112] (2.3) PointNet Feature Extraction Module

[0113] The PointNet feature extraction module consists of multiple MLP layers and a max pooling layer. The region aggregation module divides the point cloud into individual points. The PointNet feature extraction module extracts features from these regions, similar to image convolution operations.

[0114] (3) Feature embedding layer

[0115] The main function of the feature embedding layer is to embed and fuse the features extracted from two point cloud frames through the feature encoding layer. A single point cloud frame mainly reflects the positional information of vehicles in the scene, but lacks information about vehicle motion. Furthermore, the motion of vehicles in a scene often exhibits local variability; different vehicles in the same point cloud frame may have different motion patterns. Therefore, it is difficult to extract features that simultaneously represent multiple complex motions from a single point cloud frame. Thus, when performing dynamic vehicle detection on point clouds, it is necessary to extract the relative features of two consecutive point cloud frames to represent vehicle motion.

[0116] To determine point clouds Points in The motion state needs to be determined in the corresponding point cloud frame. Find its corresponding point in To achieve this, features from two consecutive point cloud frames are fused to obtain the associated features between the two frames. The specific steps are as follows: for each feature point in the first frame of the point cloud... The KNN algorithm is used to search for the second frame data points within a region of radius r. The system selects k points from the second frame to form a region, extracts features from this region using PointNet, and outputs the features of the fused point clouds from the two frames. It can be expressed by equation (4):

[0117] (4)

[0118] in, It is a nonlinear function; This indicates a max pooling layer operation that takes the maximum value of each feature.

[0119] By using the point cloud data points in the first frame Extracting data points from the second frame in the surrounding area and performing feature extraction allows for judgment through fitting. Whether a corresponding point exists in the second frame point cloud, and then determine. The state of motion.

[0120] (4) Feature decoding layer

[0121] The function of the feature decoding layer is to reconstruct the point cloud from the extracted high-dimensional features through upsampling and to determine its motion state. The feature decoding layer mainly consists of two modules: (4.1) upsampling module; (4.2) unit feature extraction module.

[0122] (4.1) Upsampling module

[0123] The upsampling module primarily uses interpolation to recover points lost during feature extraction from high-dimensional features. It employs a K-nearest neighbor inverse distance weighted algorithm to interpolate the point cloud for lost data points. Assume that the characteristics of its k nearest data points are ,but The feature values ​​after interpolation and upsampling As shown in equation (5):

[0124] (5)

[0126] in, This is a distance metric function.

[0127] (4.2) Unit Feature Extraction Module

[0128] The unit feature extraction module mainly consists of fully connected layers and the ReLU activation function. Each unit feature extraction layer operates on the features of each point, primarily concatenating and updating the features of each point to facilitate the determination of the point's motion state.

[0129] (5) Output layer

[0130] The model output is A dimensional vector, where Let represent the number of points in the first frame point cloud. Each column vector represents the predicted value for a data point that is stationary (excluding stationary vehicles), a data point that is moving (corresponding to a vehicle), and a data point that is stationary (corresponding to a vehicle). To ensure that the predicted values ​​correspond to the probabilities of the predicted states for each point, the output feature vector is processed through a SoftMax layer, resulting in the following output: A dimensional vector, and where The predicted state of a point corresponds to a vector. By taking the maximum value of this vector, the motion state of the point can be obtained. Thus, the predicted state of each point in the first frame of the point cloud is... for:

[0131] (6)

[0132] in, These are the model inputs for two consecutive frames of point clouds, respectively. This is the mapping function from the input to the output of the model; , and These are the points predicted by the model. Let be the probability of a point corresponding to a stationary object (excluding vehicles), a point corresponding to a moving vehicle, and a point corresponding to a stationary vehicle.

[0133] (6) Loss function

[0134] The loss function objectively determines the performance of a network model and is an important component of deep learning. This algorithm model achieves dynamic vehicle instance segmentation by classifying each data point in a frame of point cloud. It uses cross-entropy as the loss function of the model, and its expression is shown in equation (7):

[0135] (7)

[0137] in, This indicates the number of points in the first frame of the point cloud. Point The A true label of a movement status Points predicted by the model The corresponding number The probability of a motion state.

[0138] Based on the above-mentioned point cloud dynamic vehicle detection model, the point cloud dynamic vehicle detection algorithm (FEMD) is run. By inputting two consecutive frames of point cloud into the point cloud dynamic vehicle detection model, the motion state prediction result of each point cloud data point in the first frame of point cloud can be obtained.

[0139] The specific algorithm flow of FEMD ( Figure 5 ),include:

[0140] After the input layer enters the data preprocessing module, steps 2.1-2.7 are executed as follows:

[0141] Step 2.1 Initialize model weights .

[0142] Step 2.2 Process the simulation dataset The point cloud in the image is denoised.

[0143] Step 2.3 Transfer the dataset Divide into consecutive point cloud frame pairs to obtain .

[0144] Step 2.4 Process the dataset The point cloud frames used in the training set are downsampled separately to obtain the training set.

[0145] Step 2.5 The training set Divide into several equal-sized ,get .

[0146] Step 2.6 If it exists If the access is not successful, proceed to step 2.7; otherwise, proceed to step 2.9.

[0147] Step 2.7 will The sample data is input into the feature encoding layer, feature embedding layer, and feature decoding layer to predict the first frame point cloud. Motion state of data points .

[0148] Step 2.8 Calculate the estimated value based on the loss function. With real data The loss is calculated, and the model weights are updated according to the backpropagation algorithm. Proceed to step 2.6.

[0149] Step 2.9 Outputs the trained point cloud dynamic vehicle detection method model and its weight parameters from the output layer. .

[0150] Step 3: Simulation Experiment Verification

[0151] (1) Experimental environment

[0152] The relevant experimental parameters are shown in Table 1.

[0153] Table 1

[0154]

[0155] (2) Dataset

[0156] The experiments in this invention use the Blensor simulator to generate simulation experiment datasets. Blensor is an open-source point cloud simulation software that can simulate LiDAR (Velodyne 32 / 64-line, etc.), TOF depth cameras, Kinect, etc. It is developed based on the 3D animation software Blender. During simulation, the model and parameters of the LiDAR can be adjusted. Detailed information is shown in Table 2.

[0157] Table 2

[0158]

[0159] (3) Evaluation indicators

[0160] The classification metrics used in the experiment included accuracy, precision, and recall.

[0161] 1) Accuracy: For binary classification problems, it represents the percentage of correctly predicted samples out of the total number of samples, and can be calculated by equation (8):

[0162] (8)

[0164] in, TP For true class ( (i.e., data points whose instances are of the positive class and are predicted to be of the positive class); FN For pseudo-negative class ( (i.e., data points whose instances are of the positive class but are predicted as the negative class); FP For false positives ( (i.e., data points whose instances are negative but are predicted as positive); TN True negative class ( (i) refers to data points whose instances are negative and are predicted to be negative.

[0165] 2) Accuracy: This represents the proportion of data points that are actually in motion within the point cloud predicted by the model. It can be calculated using equation (9):

[0166] (9)

[0168] in, TP For true class ( ), FP For false positives ( ).

[0169] 3) Recall: This represents the proportion of data points that are actually in motion but are actually predicted to be in motion. It can be calculated using equation (10):

[0170] (10)

[0172] in, TP For true class ( ), FN For pseudo-negative class ( ).

[0173] (4) Analysis of experimental results

[0174] 1) Point cloud downsampling

[0175] Imbalanced class distribution in datasets often negatively impacts a model's classification ability. Since vehicles constitute a smaller proportion of the point cloud compared to the environment, this leads to an imbalance in the ratio of positive to negative samples. This invention processes the simulation dataset through point cloud downsampling, removing stationary point cloud data points around the road, such as those representing buildings and road surfaces, to balance the ratio of positive to negative samples. Experiments were conducted using datasets with downsampling ratios of approximately 0%, 20%, 40%, 50%, and 60% to verify the ability of the FEMD method trained on datasets with different positive-to-negative sample ratios to detect dynamic vehicles.

[0176] First, the algorithm's detection performance is compared with the downsampling ratios of 0% and 60% through visualization results, and then quantitative analysis is performed.

[0177] The ground truth and prediction visualization results of the FEMD method on a dataset with a point cloud downsampling ratio of approximately 0% are shown below. Figure 6 As shown in the figure, green data points represent dynamic point clouds, while red and cyan data points represent static point clouds. Red data points represent the point clouds corresponding to stationary vehicles, and cyan data points represent the point clouds corresponding to other stationary objects. Figure 6 (Above) represents the true value of the point cloud motion state. Figure 6 (Bottom) shows the prediction results of the FEMD method. As can be seen from the figure, although the FEMD method can segment the vehicle from its surrounding environment, it struggles to predict the vehicle's motion state. This is mainly because the proportion of point cloud data points corresponding to stationary vehicles is relatively small compared to the point cloud data points corresponding to the environment, approximately 1%-5%. This results in a low proportion of moving vehicle samples, making it difficult for the model to fit the distribution of the corresponding categories of moving vehicles, leading to underfitting. In summary, when the proportion of moving vehicles in the entire point cloud is small, the model trained using the FEMD method does not perform well in detecting dynamic vehicles.

[0178] The ground truth and prediction visualization results of the FEMD method on a dataset with a point cloud downsampling ratio of approximately 60% are as follows: Figure 7 As shown. Figure 7 (Above) represents the true value of the point cloud motion state. Figure 7(Bottom) shows the prediction results of the FEMD method. The speed of the moving vehicle at the top of the figure is 20 km / h, and the speed of the moving vehicle at the bottom right is 65 km / h. From Figure 7 As can be seen, when vehicles travel at high speeds in the environment, the FEMD method can completely segment moving vehicles. Even when vehicles travel at low speeds, the FEMD method can still accurately segment moving vehicles with only a few data points showing errors, primarily concentrated in planes and discrete point regions with limited feature information. Furthermore, the FEMD method can accurately predict vehicle motion states, indicating that when the dataset has a high downsampling ratio, the point cloud dynamic vehicle detection model trained using the FEMD method can effectively segment vehicles from the environment and predict their motion states.

[0179] Table 3 shows the downsampling diagram of the experimental data. The accuracy metric is used to measure the performance of the FEMD method at different point cloud downsampling ratios. As can be seen from the table, when the downsampling ratio is 0%, although FEMD cannot effectively distinguish between dynamic vehicles and static data points, the algorithm still has a high accuracy of 0.92. This is mainly because the static objects in the point cloud account for a large proportion, which inflates the algorithm's accuracy and does not reflect its performance in detecting dynamic vehicles. However, as the sampling ratio increases, the accuracy of the FEMD method also increases. When the downsampling ratio reaches about 40%, the model trained by the FEMD method can distinguish between moving and stationary vehicles. This is mainly because downsampling removes a large number of static data points from the point cloud, balancing the ratio of positive and negative samples in the dataset, allowing the model to effectively fit features related to vehicle motion.

[0180] Table 3

[0181]

[0182] 2) Point cloud dynamic vehicle detection

[0183] (a) Model training

[0184] In the experiments of this invention, the following methods are used: ① FEMD for Dropout removal; ② FEMD for feature embedding removal; ③ Scene Flow-PointNet (D-FlowNet3D); ④ Scene Flow-Threshold Segmentation (T-FlowNet3D); ⑤ PointNet as a comparison method with the FEMD method of this invention. ① and ② are ablation experiments of FEMD, the former removing the Dropout layer in the FEMD network, and the latter removing the feature embedding layer. ③ Predicting scene flow for two consecutive frames using FlowNet3D, and then performing dynamic vehicle detection using the PointNet network (D-FlowNet3D); ④ Predicting scene flow for two consecutive frames using FlowNet3D, and then performing dynamic vehicle detection using threshold segmentation (T-FlowNet3D); ⑤ Performing dynamic vehicle detection on a single frame point cloud using PointNet++ (PointNet).

[0185] The deep learning models mentioned above were all trained on a simulation dataset (training set 1294×2 frames, test set 162×2 frames, validation set 162×2 frames) (except for ④T-FlowNet3D, which is not based on a deep learning method and does not require training). The training epoch was 250, the batch size was 32, and the initial learning rate was 0.00001. The learning rate was adjusted according to the training epoch, multiplied by 0.1 at 50, 75, and 150 epochs respectively. The changes in loss value and accuracy of each method with the epoch during training are shown below. Figure 8 , Figure 9 As shown in the figure, the model trained using the FEMD method experiences a faster loss reduction during training and exhibits higher accuracy across all training epochs. Furthermore, removing the Dropout layer results in faster convergence for FEMD compared to retaining the Dropout layer, although the model accuracy decreases. This is primarily because the Dropout layer effectively prevents overfitting.

[0186] (b) Ablation study

[0187] In the ablation experiment, the algorithm ①FEMD used in this invention was compared with ② without a feature embedding module and ③ without a Dropout layer to examine the impact of the model feature embedding layer and the Dropout layer in the model training module on the FEMD network.

[0188] Figure 10The impact of removing the feature embedding layer and Dropout layer from the FEMD method on model performance was measured using metrics such as accuracy, recall, and precision. As shown in the figure, removing the feature embedding layer resulted in a decrease in the performance of all metrics compared to FEMD. Specifically, the accuracy was 0.9055, a decrease of nearly 8 percentage points compared to FEMD's 0.9786. This is mainly because a single frame of point cloud data often only contains vehicle location information, with less information related to vehicle motion. The role of the feature embedding layer is to provide information for determining the motion state of the point cloud by interacting with features from two consecutive frames. Removing the feature embedding layer reduces the information interaction between consecutive frames, resulting in fewer motion-related features that the network can extract. This reduces the model's ability to extract motion-related information from the point cloud, leading to a significant drop in accuracy. Simultaneously, removing the feature embedding layer also significantly reduced the model's recall and precision, indicating a significant decrease in the model's ability to predict the motion state of the vehicle's corresponding point cloud data points. This makes it difficult to effectively determine the vehicle's motion state and results in some false positives. When the Dropout layer is not used during model training, the model's accuracy and precision are 0.9691 and 0.9343, respectively, which are lower than the FEMD method's 0.9786 and 0.9405. However, the recall rates of the two remain similar, with the former having a recall rate of 0.9648 and the latter 0.9634. It can be seen that adding a Dropout layer during model training can effectively improve the model's performance. This is mainly because although the dataset has been downsampled by 60%, there is still a significant difference in the proportion of positive and negative samples. Adding a Dropout layer during training can effectively prevent the model from overfitting and improve the model's performance to a certain extent.

[0189] In summary, experiments have verified the positive impact of adding feature embedding layers and Dropout layers to the FEMD model on performance.

[0190] (c) Comparison of point cloud dynamic vehicle detection algorithms

[0191] After completing the ablation experiment, this invention compares the algorithm ①FEMD used in this invention with other point cloud dynamic vehicle detection methods (②FlowNet3D computes scene flow + PointNet, D-FlowNet3D; ③FlowNet3D computes scene flow + threshold segmentation, T-FlowNet3D; ④PointNet detects a single image, PointNet) in terms of accuracy, recall, and precision.

[0192] The accuracy performance of D-FlowNet3D, T-FlowNet3D, PointNet, and FEMD methods is as follows: Figure 11As shown in the figure, the end-to-end dynamic vehicle detection methods PointNet and FEMD outperform the methods D-FlowNet3D and T-FlowNet3D, which first calculate the scene flow and then detect dynamic vehicles. This is mainly because judging based on intermediate features leads to the accumulation of errors, causing a decrease in model accuracy. Comparing the scene flow-based methods D-FlowNet3D and T-FlowNet3D, the method of segmenting dynamic vehicles from the scene flow using a deep learning network has an accuracy of 0.7614, which is higher than the 0.7224 of the threshold-based segmentation method T-FlowNet3D. This is mainly because deep learning networks have stronger feature extraction capabilities and can better segment scene flow data to segment dynamic vehicles in the point cloud. Comparing the PointNet method, which detects dynamic vehicles from a single frame, and the FEMD method, which detects dynamic vehicles from two consecutive frames, it can be seen that the accuracy of the FEMD method (0.9786) is significantly higher than that of PointNet (0.8166). This is mainly because a single frame point cloud primarily contains vehicle position information, but less information related to vehicle motion. It is difficult to extract motion-related features from a single frame point cloud alone, resulting in insufficient vehicle motion state determination capability for the PointNet network. In contrast, the FEMD method, by combining point clouds from two consecutive frames, can obtain more motion-related features, and then analyzes these features through a feature extraction layer, thus exhibiting better vehicle motion state determination capability. In conclusion, the FEMD method can more accurately determine the motion state of vehicles in point clouds compared to other methods.

[0193] The recall performance of D-FlowNet3D, T-FlowNet3D, PointNet, and FEMD methods is as follows: Figure 12As shown in the figure, recall in this experiment represents the proportion of point cloud data points that are actually dynamic point cloud data points but are predicted to be dynamic points. Easily captured features in point clouds are often concentrated at the boundaries of the point cloud; these data points represent the shape of the point cloud. The ability to capture features corresponding to other points with lower information entropy reflects the robustness of the model. When the prediction accuracy is high, the recall reflects the model's ability to extract features with lower information entropy. As can be seen from the figure, PointNet's recall is 0.5981, which is lower than other methods, indicating that it is difficult to extract motion features from a single frame of point cloud data. PointNet is almost ineffective in determining the motion state of point cloud data points. The recall rate of dynamic point cloud instance segmentation methods based on intermediate feature scene streams is also lower than that of the FEMD method, at 0.8502 and 0.7895 respectively. Although segmenting scene stream data through deep learning can effectively improve the algorithm's ability to detect moving vehicles, the inherent defects of the scene stream calculation model can introduce errors in the calculation, interfering with the determination of the motion state of the point cloud. This can lead to some dynamic points being incorrectly identified as static points, thus reducing the recall rate of the method. In contrast, the end-to-end dynamic vehicle detection method FEMD can extract motion-related features from two frames of point cloud and directly determine the motion state of the point cloud, reducing the error in motion state determination and achieving a higher recall rate. In summary, the FEMD method has better motion feature extraction capabilities compared to other methods.

[0194] The accuracy performance of D-FlowNet3D, T-FlowNet3D, PointNet, and FEMD methods is as follows: Figure 13 As shown in the figure, accuracy in this experiment represents the percentage of point cloud data points predicted as dynamic points that are actually dynamic points. It effectively reflects the method's ability to distinguish between stationary and moving vehicle states, as well as the accuracy of its dynamic point predictions. The figure shows that T-FlowNet3D and D-FlowNet3D methods have relatively low accuracy, indicating that a large amount of static data is predicted as dynamic points. This is mainly because point cloud changes significantly at the boundaries between static objects such as the ground and moving vehicles, resulting in larger scene flow errors and causing many static points to be incorrectly identified as dynamic points. Meanwhile, the FEMD method has an accuracy of 0.9406, far higher than the other three algorithms, demonstrating that FEMD not only predicts vehicle motion states but also maintains high accuracy in motion state determination for some difficult-to-determine boundary regions.

[0195] In summary, experiments have verified that the FEMD method has superior performance compared to other methods when performing dynamic vehicle detection in point clouds.

[0196] 3) NuScenes dataset testing

[0197] To verify the effectiveness of the FEMD method on real-world scene data, experiments were conducted on the nuScenes dataset. nuScenes contains data from approximately 1000 driving scenarios collected by multiple sensors, including point cloud data collected by a LiDAR located on the vehicle roof. This invention's experiments demonstrate the effectiveness of the FEMD method by applying it to the point clouds collected by LiDAR in the nuScenes dataset for dynamic vehicle detection.

[0198] Figure 14 The visualization shows the prediction results of the FEMD method when the scene contains only stationary vehicles. In the prediction results, green represents points predicted as stationary by the FEMD method, and red represents points predicted as moving. The image shows that when there are many similar features in the point clouds corresponding to the same vehicle in two consecutive frames, the FEMD method can effectively determine the vehicle's motion state. However, real-world scene data contains noise due to hardware and environmental factors, and some data is missing, making it difficult for the FEMD method to capture effective features at these locations. It is prone to misjudgment in areas with less information, and the complex point cloud changes at the vehicle-ground interface also increase the likelihood of misjudgment. In summary, the FEMD method can effectively determine the state of stationary vehicles in real-world scenes.

[0199] Figure 15 The visualization shows the prediction results of the FEMD method when moving vehicles are present in the scene. In the prediction results, green represents points predicted as stationary by the FEMD method, and red represents points predicted as moving. The image shows that on real-world scene data, the FEMD method can accurately predict the position of moving vehicles. However, at the boundary between the vehicle and the ground, due to significant changes in the point cloud between consecutive frames, the FEMD method is prone to misclassification. Overall, the FEMD method can effectively predict the state of moving vehicles on real-world scene datasets.

[0200] In summary, the FEMD method can effectively predict the motion state of vehicles on real-world datasets, demonstrating its feasibility.

[0201] This invention addresses the challenges of point cloud dynamic vehicle detection in autonomous driving scenarios, which requires extensive prior knowledge and cannot directly achieve high-precision output, as well as the difficulty of labeling dynamic vehicles in autonomous driving point cloud datasets. It proposes a new method for point cloud dynamic vehicle detection in autonomous driving scenarios. First, point cloud data and dynamic vehicle labeling methods are generated through simulation in an autonomous driving scenario. Based on this, a point cloud dynamic vehicle detection method based on feature embedding is presented, and ablation and comparative experiments are conducted. Experimental results show that this method effectively detects the motion state of vehicles in point clouds in autonomous driving scenarios, and its detection accuracy is higher than that of other point cloud dynamic vehicle detection methods in non-autonomous driving scenarios. Therefore, it can directly perceive the motion of vehicles in autonomous driving scenarios without relying on extensive prior knowledge, enabling accurate determination of whether point cloud distortion occurs due to the relative motion between the vehicle and the lidar, thereby improving the accuracy of intelligent perception for autonomous vehicles and potentially overcoming the obstacles to intelligent decision-making in autonomous driving motion behavior.

[0202] The above description is merely a description of preferred embodiments of this application and is not intended to limit the scope of this application in any way. Any changes or modifications made by those skilled in the art based on the above-disclosed technical content should be considered as equivalent and valid embodiments and fall within the scope of protection of the technical solution of this application.

Claims

1. A method for dynamic vehicle detection in point clouds under autonomous driving scenarios, characterized in that, Includes the following steps: Step 1. Dynamic vehicle labeling and construction of training dataset; For each data point in each frame of the point cloud The values ​​0, 1, and 2 are used to distinguish the motion state of the object being reflected, as shown in equation (2): （2） in, Point cloud frames The corresponding tag set; Point cloud frames Mid-data points The corresponding tag, when When, it represents a data point. It is reflected from stationary objects other than vehicles, when When, it represents a data point. Reflected by moving vehicles, when When, it represents a data point. Reflected by stationary vehicles; Step 2. Build a point cloud dynamic vehicle detection model and run the point cloud dynamic vehicle detection algorithm; the building of the point cloud dynamic vehicle detection model in step 2 includes: The constructed point cloud dynamic vehicle detection model includes a model input layer, a data preprocessing module, four feature encoding layers, a feature embedding layer, four feature decoding layers, and a model output layer. The model takes two consecutive frames of point clouds as input. After data preprocessing, it extracts depth feature information at different scales through a feature encoding layer and a feature embedding layer. This depth feature information is then input into a feature decoding layer to restore the same dimension as the first frame of point cloud. During the decoding process, the output features of the feature encoding layer are fused to help recover the original image information. Finally, the decoded features output the state corresponding to each point cloud data point in the first frame of point cloud through the softmax function. The model contains four feature encoding layers, each consisting of sampling, region aggregation, MLP and max pooling modules, which are used to extract features from the point cloud to obtain higher-dimensional features. The model contains four feature decoding layers. Each feature decoding layer consists of an upsampling module and a unit feature extraction module. Its function is to reconstruct the point cloud from the extracted high-dimensional features and determine its motion state through upsampling. During the upsampling process, the output features of each feature encoding layer are fused to help better recover the original data information. The model includes a feature embedding layer, which is used to embed and fuse the features extracted from the feature encoding layer of two point clouds, and to fuse the point cloud data features of the two branches into the subsequent feature encoding layer for further deep feature extraction. The model output is for A dimensional vector, where The first frame contains the number of points in the point cloud. Each column vector represents the predicted value of the data point when it is a stationary point (excluding stationary vehicles), the data point when it is a moving data point corresponding to a vehicle, and the data point when it is a stationary data point corresponding to a vehicle. Design a loss function.

2. The point cloud dynamic vehicle detection method in an unmanned driving scenario as described in claim 1, characterized in that, Step 1: A data point in a point cloud The size can be expressed by equation (1): （1） in, For data points in a point cloud, This refers to the scanning resolution of the lidar. This is the unit vector of the laser beam corresponding to the point cloud; The distance from the surface of the object being scanned to the lidar; Data points for lidar The number of columns already scanned; This represents the vertical field of view angle of the laser beam corresponding to this data point.

3. The point cloud dynamic vehicle detection method in an unmanned driving scenario as described in claim 1, characterized in that, Step 2: in: The feature encoding layer 1 has 1024 sampling points and 16 selected center points. The input is two point cloud data from the model input layer, and the output is two 64-dimensional features. The number of sampling points in feature coding layer 2 is 256, the number of selected center points is 16, the input is the two features output from feature coding layer 1, and the output is two 128-dimensional features; The feature encoding layer 3 has 64 sampling points and 8 selected center points. The input is the fused feature output from the feature embedding layer, and the output is a 256-dimensional feature. The number of sampling points in feature coding layer 4 is 16, the number of selected center points is 8, the input is the feature output from feature coding layer 3, and the output is a 512-dimensional feature. in: The input to feature decoding layer 1 is the four output features of feature encoding layer 4; The input to feature decoding layer 2 is obtained by adding the output features of feature decoding layer 1, feature embedding layer, and feature coding layer 3; The input to feature decoding layer 3 is obtained by adding the first output features of feature decoding layer 2 and feature coding layer 2; The input to feature decoding layer 4 is obtained by adding the first output features of feature decoding layer 3 and feature encoding layer 1; the output of all feature decoding layers is a 256-dimensional feature. The input to the feature embedding layer is the two features output from the feature encoding layer 2, and the output is a 128-dimensional feature.

4. The point cloud dynamic vehicle detection method in an unmanned driving scenario as described in claim 1, characterized in that, Step 2: The data preprocessing module operates according to the following algorithm: Step 1.1 Point Cloud Denoising First, each point in the point cloud is evaluated. Outliers that deviate from the overall point cloud data are selected based on their relationship with surrounding points and removed from the point cloud. Then, the offset values ​​of the remaining points are estimated, and the point cloud is reconstructed based on the estimated offsets to remove point cloud noise, as shown in formula (3). （3） in Point clouds for noise removal; The original point cloud contains noise; The set of outliers in the point cloud; The estimated point cloud offset value; Step 1.2 Data Downsampling Step 1.2.1 Input a denoised point cloud containing motion state labels. and downsampling ratio ; Step 1.2.2 Initialize the set of data points to be removed Each time, the proportion of the point cloud set span perpendicular to the road direction is removed. ,counter and the downsampled point cloud ; Step 1.2.3 Calculate the point cloud Number of midpoints and point clouds perpendicular to the road direction span ; Step 1.2.4 If The number of data points in the mid-point cloud is less than Proceed to step 1.2.5; otherwise, proceed to step 1.2.

10. Step 1.2.5 will convert the point cloud Divide the remaining point cloud along the length perpendicular to the road direction. The span from the boundary in the perpendicular direction of the road is The point cloud is divided into sets ; Step 1.2.6 If the point cloud set If no data point is marked as a dynamic point, proceed to step 1.2.7; otherwise, proceed to step 1.2.

8. Step 1.2.7 Remove In point cloud collection The data points in the point cloud are used to create a point cloud collection. Merging Sets Then reset the counter to 0 and proceed to step 1.2.4; Step 1.2.8 Halve the size and set the counter Increase by 1; Step 1.2.9 If the calculator's size is equal to 4, exit the loop and go to step 1.2.10; otherwise, go to step 1.2.

4. Step 1.2.10 Output the downsampled point cloud .

5. The point cloud dynamic vehicle detection method in an unmanned driving scenario as described in claim 1, characterized in that, Step 2: Design a loss function, the form of which is shown in equation (7): （7） in, This indicates the number of points in the first frame of the point cloud. Point The A true label of a movement status This indicates the point corresponding to the prediction by the model. The probability of a motion state.

6. The point cloud dynamic vehicle detection method in an unmanned driving scenario as described in claim 1, characterized in that, Step 2: Based on the point cloud dynamic vehicle detection model, the point cloud dynamic vehicle detection algorithm FEMD is run. The specific algorithm flow of FEMD includes: After the input layer enters the data preprocessing module, steps 2.1-2.7 are executed as follows: Step 2.1 Initialize model weights ; Step 2.2 Process the simulation dataset Denoising is performed on the point cloud in the image; Step 2.3 Transfer the dataset Divide into consecutive point cloud frame pairs to obtain ; Step 2.4 Process the dataset The point cloud frames used in the training set are downsampled separately to obtain the training set. ; Step 2.5 The training set Divide into several equal-sized ,get ; Step 2.6 If it exists If the access is not successful, proceed to step 2.7; otherwise, proceed to step 2.

9. Step 2.7 will The sample data is input into the feature encoding layer, feature embedding layer, and feature decoding layer to predict the first frame point cloud. Motion state of data points ; Step 2.8 Calculate the estimated value based on the loss function. With real data The loss is calculated, and the model weights are updated according to the backpropagation algorithm. Proceed to step 2.6; Step 2.9 Outputs the trained point cloud dynamic vehicle detection method model and its weight parameters from the output layer. .

Images