A laser radar-based automatic driving moving obstacle segmentation method

Abstract

The application discloses an automatic driving moving obstacle segmentation method based on a laser radar, and comprises the following steps: acquiring time sequence point cloud data of a plurality of continuous time stamps in an automatic driving scene; wherein the time sequence point cloud data of the plurality of continuous time stamps comprises time sequence point cloud data of a current time stamp; determining interframe difference feature information of each time stamp according to the time sequence point cloud data of the plurality of continuous time stamps; inputting the interframe difference feature information of each time stamp and projection data of the time sequence point cloud data of the current time stamp into a trained segmentation network model, and obtaining state prediction masks and semantic prediction masks of obstacle segmentation by taking semantic information as guidance. The features of obstacle point cloud data are expressed based on the composition of point cloud projection data unique representation and an end-to-end deep learning network, so that more accurate identification of moving obstacles in an automatic driving scene is realized.

Description

Technical Field

[0001] This invention relates to the fields of autonomous driving and computer vision technology, and in particular to a method for segmenting moving obstacles in autonomous driving based on LiDAR. Background Technology

[0002] With the development of computer vision technology, cameras are widely used. Cameras can capture a wealth of environmental information and perform region segmentation using features such as color and texture. However, cameras are susceptible to interference from natural conditions such as lighting and weather. LiDAR, on the other hand, has advantages such as long measurement range, high accuracy, and is unaffected by weather factors like lighting and shadows. It also exhibits good stability and data robustness. Therefore, this research is based on LiDAR data.

[0003] Moving obstacles mainly include pedestrians and moving vehicles. In dynamic traffic environments, moving obstacles are ubiquitous and unavoidable. Semantic segmentation is the foundation for point cloud segmentation of moving obstacles. Semantic segmentation for autonomous driving is a dense classification task from points to surfaces, effectively distinguishing categories such as sky, trees, vehicles, and bicycles in a single scan dataset. Semantic segmentation can be seen as a preliminary step in moving object segmentation. Currently, LiDAR semantic segmentation tasks can be categorized into three main methods: point-based, projection-based, and voxel-based discriminative methods. Among these, projection-based methods, which directly utilize mature neural networks from the field of 2D image segmentation, have received particular attention. Moving object segmentation differs from semantic segmentation; it is not only a task of dense point mapping but also requires effectively extracting differences between consecutive scan frames to distinguish dynamic obstacles.

[0004] In autonomous driving environments, the dynamic and static recognition of target objects is crucial for achieving safe motion planning and navigation. The route of an autonomous vehicle must take into account the future coordinates and velocities of surrounding moving objects. Current obstacle detection typically involves two steps: first, finding obstacles based on road segmentation; and second, distinguishing whether the obstacle is moving or stationary. Since the vehicle is in motion, it's difficult to differentiate between the background and other moving objects. Therefore, it's necessary to estimate and compensate for the vehicle's own motion to extract moving objects from the scene. Traditional segmentation methods have limitations. For example, road segmentation relies on the assumption of flat roads, resulting in poor generalization to potholes and inclines / declines. Furthermore, the estimation and compensation of self-motion are affected by the strength of GPS signals. Currently, there is considerable research on semantic segmentation methods based on deep learning, which can replace traditional road segmentation methods and alleviate the problem of relying on the flat road assumption. While two-step obstacle segmentation detection methods can solve certain practical problems, the inconsistent training objectives at each stage may deviate from the overall macroscopic goal, making it difficult to achieve optimal detection accuracy.

[0005] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a method for segmenting moving obstacles in autonomous driving based on LiDAR, which addresses the above-mentioned deficiencies of the prior art. The method aims to solve the problems of the lack of a mature and complete end-to-end point cloud moving object obstacle segmentation method and the inaccurate segmentation and detection of target features in the existing technology.

[0007] The technical solution adopted by this invention to solve the technical problem is as follows:

[0008] A method for segmenting moving obstacles in autonomous driving based on LiDAR, comprising the following steps:

[0009] Acquire time-series point cloud data with several consecutive timestamps in an autonomous driving scenario; wherein, the time-series point cloud data with several consecutive timestamps includes the time-series point cloud data with the current timestamp;

[0010] Based on time-series point cloud data with several consecutive timestamps, determine the inter-frame difference feature information of each timestamp;

[0011] The inter-frame difference feature information of each timestamp and the projection data of the temporal point cloud data of the current timestamp are input into the trained segmentation network model to obtain the state prediction mask and semantic prediction mask for obstacle segmentation.

[0012] The trained segmentation network model includes: a temporal fusion module, an asymmetric encoder-decoder, and a point cloud data recovery module;

[0013] The step of inputting the inter-frame difference feature information of each timestamp and the projection data of the temporal point cloud data of the current timestamp into the trained segmentation network model to obtain the state prediction mask and semantic prediction mask for obstacle segmentation includes:

[0014] The inter-frame difference feature information of each timestamp and the projection data of the temporal point cloud data of the current timestamp are input into the temporal fusion module to obtain the fusion feature;

[0015] The fused features are input into the asymmetric encoder and decoder to obtain feature information data;

[0016] The feature information data is input into the point cloud data recovery module to obtain the state prediction mask and semantic prediction mask for obstacle segmentation.

[0017] The aforementioned LiDAR-based method for segmenting moving obstacles in autonomous driving includes an asymmetric encoder / decoder comprising an encoder, a context information module, a motion segmentation decoder, and a semantic information decoder; the feature information data includes an obstacle motion state prediction mask and a semantic prediction mask for moving obstacles.

[0018] The fused features are input into the asymmetric encoder and decoder to obtain feature information data, including:

[0019] The fused features are input into the encoder to obtain the encoded features;

[0020] The encoded features are input into the context information module to obtain a feature image;

[0021] The feature image is input into the motion segmentation decoder to obtain the obstacle motion state prediction mask;

[0022] The feature image is input into the semantic information decoder to obtain the semantic prediction mask of the moving obstacle.

[0023] The aforementioned LiDAR-based method for segmenting moving obstacles in autonomous driving includes a context information module comprising four parallel dilated convolutional layers with dilation rates of 6, 12, 18, and 24, respectively.

[0024] The encoder, the motion segmentation decoder, and the semantic information decoder each include four encoding modules, each of which includes a convolutional layer, a batch normalization layer, a residual module, and an activation function layer.

[0025] The aforementioned method for segmenting moving obstacles in autonomous driving based on LiDAR, wherein determining the inter-frame difference feature information of each time stamp based on temporal point cloud data with several consecutive time stamps includes:

[0026] All point cloud coordinate information of past timestamps are uniformly transformed to the current timestamp point cloud data coordinate system according to pose transformation, and the time-series point cloud data of continuous timestamps are projected according to affine transformation to obtain the projection data of time-series point cloud data of each timestamp.

[0027] For the projection data of the time-series point cloud data of each past time point in a series of consecutive time stamps, the inter-frame difference feature information of the past time stamp is obtained by performing a pixel-by-pixel multiplication operation and normalization based on the projection data of the past time stamp and the projection data of the current time stamp.

[0028] The aforementioned method for segmenting moving obstacles in autonomous driving based on lidar, wherein the inter-frame difference feature information is:

[0029]

[0030] Where diff represents inter-frame difference feature information, SP current SP represents the projected data of the time-series point cloud data at the current timestamp. i This represents the projection data of the time-series point cloud data at the i-th past timestamp.

[0031] The aforementioned method for segmenting moving obstacles in autonomous driving based on LiDAR, further comprising:

[0032] Based on the state prediction mask of the obstacle segmentation, the static / dynamic category of the obstacle is determined, and based on the semantic prediction mask of the obstacle segmentation, the semantic category of the obstacle is determined.

[0033] The aforementioned method for segmenting moving obstacles in autonomous driving based on LiDAR includes a segmentation network model whose parameters are updated during training based on an overall loss function; the overall loss function is:

[0034] Loss = L semantic +L mowing +L ls

[0035] Where Loss represents the overall loss function, L semantic L represents the semantic cross-entropy loss function. moving L represents the loss function for moving obstacles. ls This represents the Lovasz Softmax loss function.

[0036] The aforementioned LiDAR-based method for segmenting moving obstacles in autonomous driving involves evaluating the trained segmentation network model during training based on accuracy, cross-union ratio (CURRR), and network push time; wherein the CURRR is:

[0037]

[0038] Where IoU represents the intersection-union ratio, target∧prediction represents the number of pixels in the common region between the target mask and the prediction mask, and target∪prediction represents the total number of pixels between the target mask and the prediction mask.

[0039] A computer device includes a memory and a processor, the memory storing a computer program, wherein the processor, when executing the computer program, implements the steps of the method as described in any of the preceding claims.

[0040] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method as described in any of the preceding claims.

[0041] Beneficial effects: By using the unique representation of point cloud projection data and the composition of an end-to-end deep learning network to express the features of obstacle point cloud data, more accurate identification of moving obstacles in autonomous driving scenarios can be achieved. Attached Figure Description

[0042] Figure 1 This is a flowchart illustrating the LiDAR-based obstacle segmentation method for autonomous driving according to an embodiment of the present invention.

[0043] Figure 2 This is a schematic diagram illustrating the principle of calculating the inter-frame temporal residual of point clouds in an embodiment of the present invention.

[0044] Figure 3 This is a framework diagram of the segmentation network model according to an embodiment of the present invention.

[0045] Figure 4 This is a flowchart of an autonomous driving obstacle segmentation method based on LiDAR according to an embodiment of the present invention. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0047] Please also refer to Figures 1-4 This invention provides some embodiments of a method for segmenting moving obstacles in autonomous driving based on lidar.

[0048] like Figure 1 and Figure 4 As shown, the autonomous driving obstacle segmentation method based on LiDAR according to an embodiment of the present invention includes the following steps:

[0049] Step S100: Obtain time-series point cloud data of several consecutive timestamps in the autonomous driving scenario; wherein, the time-series point cloud data of several consecutive timestamps includes the time-series point cloud data of the current timestamp.

[0050] Specifically, in autonomous driving scenarios, a series of consecutive time-stamped point cloud data is acquired using an onboard 3D LiDAR. Since the onboard 3D LiDAR is in motion, the time-stamped point cloud data for each time stamp is not identical. The motion information of each frame's time-stamped point cloud data includes not only the motion information of moving obstacles within the coordinate system but also the vehicle's motion information. Within each time-stamped point cloud data, there are relatively stationary obstacles and relatively moving obstacles. Here, "relatively stationary" refers to obstacles that are stationary relative to the base surface, and "relatively moving" refers to obstacles that are moving relative to the base surface. This application identifies moving obstacles through segmentation. The series of consecutive time-stamped point cloud data includes the time-stamped point cloud data of the current time stamp and the time-stamped point cloud data of past time stamps preceding the current time stamp. It should be noted that the current time stamp and past time stamps are consecutive time stamps, and the specific number of time stamps can be set as needed. For example, acquiring time-stamped point cloud data for four consecutive time stamps, with each time stamp's point cloud data in the form of (num_point, 4).

[0051] Step S200: Determine the inter-frame difference feature information of each time stamp based on the time-series point cloud data of several consecutive timestamps.

[0052] After obtaining a number of timestamped time-series point cloud data, the inter-frame difference feature information of each timestamp is determined. Here, the inter-frame difference feature information refers to the difference feature information between the time-series point cloud data of past timestamps and the time-series point cloud data of the current timestamp. The number of inter-frame difference feature information corresponds to the number of past timestamps.

[0053] Step S200 specifically includes:

[0054] Step S210: Convert all point cloud coordinate information of past timestamps to the current timestamp point cloud data coordinate system according to pose transformation, and project the time-series point cloud data of continuous timestamps according to affine transformation to obtain the projection data of time-series point cloud data of each timestamp.

[0055] Specifically, such as Figure 2 As shown, the time-series point cloud data at the current timestamp and time-series point cloud data at past timestamps are subjected to self-motion estimation, ego motion compensation, and bird's-eye view (BEV) projection to obtain projected data. Self-motion estimation is calculated based on coordinate system transformation using the rotation matrix and translation of time-series point cloud data from several consecutive timestamps. The projected image size is set to (480, 360), and the projected data includes point cloud coordinates and intensity values, which can be represented as (x-coordinate, y-coordinate, z-coordinate, intensity value). The projected data of the time-series point cloud data at the current timestamp is denoted as SP. currentThe projected data of time-series point cloud data with past timestamps is denoted as SP. i That is, the projection data of the time-series point cloud data of the i-th past timestamp.

[0056] Step S220: For the projection data of the time-series point cloud data of each past time point in the time-series point cloud data of several consecutive timestamps, perform a pixel-by-pixel multiplication operation and normalization based on the projection data of the past time point and the projection data of the current time point to obtain the inter-frame difference feature information of the past time point.

[0057] For the projected data of the time-series point cloud data at each past timestamp, a pixel-by-pixel multiplication operation is performed between the projected data of the time-series point cloud data at the current timestamp, i.e., SP. current ×SP i Then, the projection data of the time-series point cloud data based on the current timestamp is normalized to obtain the inter-frame difference feature information of the past timestamp.

[0058] Specifically, all point cloud information from past timestamps is uniformly converted to the current timestamp point cloud data coordinates to reduce the impact of vehicle motion. Finally, the converted data from multiple consecutive timestamps are projected to obtain a bird's-eye view, and a temporal feature information representation is constructed. The temporal information map is obtained by multiplying the point cloud BEV projection maps at N time points. Since the positional information of static objects in point cloud frames at different times overlaps significantly, while the coordinate information of moving objects has less overlap between frames, the corresponding positions will become 0 after the multiplication operation. Finally, the multiplication result is normalized to the range [0, 1].

[0059] Specifically, the inter-frame difference feature information is as follows:

[0060]

[0061] Where diff represents inter-frame difference feature information, SP current SP represents the projected data of the time-series point cloud data at the current timestamp. i This represents the projection data of the time-series point cloud data at the i-th past timestamp.

[0062] Step S300: Input the inter-frame difference feature information of each timestamp and the projection data of the temporal point cloud data of the current timestamp into the trained segmentation network model to obtain the state prediction mask and semantic prediction mask for obstacle segmentation.

[0063] Specifically, such as Figure 3As shown, the inter-frame difference feature information of each timestamp and the projection data of the temporal point cloud data at the current timestamp are input into the trained segmentation network model to obtain the state prediction mask and semantic prediction mask for obstacle segmentation output by the segmentation network model, thus completing the segmentation prediction of moving obstacles in the autonomous driving scenario. The segmentation network model is specifically a bird's-eye view moving obstacle segmentation network model.

[0064] The trained segmentation network model includes: a temporal fusion module, an asymmetric encoder-decoder, and a point cloud data recovery module. Step S300 specifically includes:

[0065] Step S310: Input the inter-frame difference feature information of each timestamp and the projection data of the temporal point cloud data of the current timestamp into the temporal fusion module to obtain the fusion feature.

[0066] Step S320: Input the fused features into the asymmetric encoder and decoder to obtain feature information data.

[0067] Step S330: Input the feature information data into the point cloud data recovery module to obtain the state prediction mask and semantic prediction mask for obstacle segmentation.

[0068] Specifically, the temporal fusion module fuses the inter-frame difference feature information of each timestamp and the projection data of the temporal feature point cloud data of the current timestamp to obtain fused features. This fused feature is used as input to an asymmetric codec, which outputs feature information data. Then, the feature information data is input to the point cloud data recovery module to obtain the state prediction mask and semantic prediction mask for obstacle segmentation.

[0069] Step S310 specifically includes:

[0070] Step S310: Stack the inter-frame difference feature information of each timestamp and the projection data of the temporal point cloud data of the current timestamp to obtain the fused feature.

[0071] Specifically, the inter-frame difference feature information of each timestamp and the projection data of the temporal point cloud data of the current timestamp are stacked to obtain the fused feature.

[0072] The asymmetric codec includes: an encoder, a context information module, a motion segmentation decoder, and a semantic information decoder; the feature information data includes: an obstacle motion state prediction mask and a semantic prediction mask for moving obstacles. Step S320 specifically includes:

[0073] Step S321: Input the fused features into the encoder to obtain the encoded features.

[0074] Step S322: Input the encoded features into the context information module to obtain the feature image.

[0075] Step S323: Input the feature image into the motion segmentation decoder to obtain the obstacle motion state prediction mask.

[0076] Step S324: Input the feature image into the semantic information decoder to obtain the semantic prediction mask of the moving obstacle.

[0077] Specifically, the asymmetric encoding and decoding structure extracts and decodes the fused features, fully extracting the point cloud differences and spatial features between different time stamps. Specifically, the encoder first extracts features from the point cloud data with consecutive time stamps and inter-frame difference features. Then, the extracted encoded features are processed by a context information module to obtain multi-scale moving object feature information. Next, the decoder decodes the feature image. Finally, the deep and shallow feature information from the asymmetric encoder and decoder are fused to compensate for the feature information lost due to compression in the sparse point cloud data.

[0078] The encoder, the motion segmentation decoder, and the semantic information decoder each include four encoding modules, each of which includes a convolutional layer, a batch normalization layer, a residual module, and an activation function layer.

[0079] Specifically, an encoder is used to extract features from point cloud data with consecutive timestamps and inter-frame difference features. The encoder structure is designed based on the encoder in the DeepLab v3 network and consists of four encoder blocks. Each encoder block is composed of a convolutional layer (Conv), a batch normalization layer (BN), a residual module (Res), and an activation function layer (ReLU). The extracted information is 1 / 16 of the size of the fused features.

[0080] The context information module includes four parallel dilated convolutional layers with dilation rates of 6, 12, 18, and 24, respectively.

[0081] Specifically, the Atrous Spatial Pyramid Pooling (ASPP) module is used as the context information module to obtain multi-scale moving object feature information. The ASPP module can expand the receptive field of the model without losing resolution. The encoded image is passed through parallel dilated convolutional layers with different dilation rates to capture multi-scale information, and the output features are fused.

[0082] The decoder decodes the features after feature enrichment and performs feature upsampling size restoration. The decoder consists of two sub-parts: a motion segmentation decoder outputs a motion state prediction mask for obstacles, representing either a binary state of motion or stillness; and a semantic information decoder outputs a semantic prediction mask for moving obstacles, finally outputting a predicted label for the moving obstacle with its semantic category. Adding a semantic prediction mask enhances the segmentation prediction gain for accurately outputting the motion state prediction mask.

[0083] The point cloud reconstruction module combines the pixel coordinates contained in the point cloud projection image with the current timestamp and the feature information extracted by the asymmetric encoding and decoding module to reconstruct the feature information into point cloud data form (num_point, num_class), thus obtaining the state prediction mask and semantic prediction mask for obstacle segmentation.

[0084] Step S400: Determine the static / dynamic category of the obstacle based on the state prediction mask of the obstacle segmentation, and determine the semantic category of the obstacle based on the semantic prediction mask of the obstacle segmentation.

[0085] Specifically, the state prediction mask for obstacle segmentation is processed by a Softmax function, and the motion / static category probability of the corresponding moving object segmentation is calculated to output the motion / static category of the obstacle. The semantic prediction mask for obstacle segmentation is also processed by a Softmax function, and the semantic category probability of the corresponding moving object segmentation is calculated to output the semantic category of the obstacle. The semantic categories of obstacles include vehicles, buildings, drivable areas, etc., totaling 19 categories.

[0086] Compared with the prior art, the present invention has the following advantages:

[0087] The method proposed in this invention is an end-to-end detection network method. The input of our network is point cloud data with multiple consecutive timestamps from 3D LiDAR, and the output is a point-by-point moving obstacle mask with the current timestamp. It can realize the segmentation task in one stop. Non-industry users do not need to understand the internal structure design of the method of this invention and can directly apply it to practical problems.

[0088] This invention only requires the detection of key frames, thus meeting the real-time requirements of autonomous driving.

[0089] This invention effectively proposes a method for calculating inter-frame residuals in continuous timestamp point cloud data containing temporal information. It provides feature extraction capability for weak attention mechanisms without increasing model calculation parameters, and effectively characterizes the inter-frame differences and spatial information of moving obstacles under different timestamps.

[0090] The present invention proposes a method for identifying moving obstacles by segmentation and detection. It adopts a fusion approach based on the information of the current timestamp and the past timestamp, and combines an asymmetric encoder and decoder structure to fully extract the temporal and spatial information features of the point cloud. It can obtain high-level semantic information while preserving the edge information of the point cloud.

[0091] The method of this invention can be extended to downstream tasks such as synchronous positioning and navigation planning in dynamic traffic environments, because these downstream tasks require making corresponding decisions based on the position and information of moving and static objects.

[0092] The method of this invention can be extended to downstream tasks such as synchronous positioning and navigation planning in dynamic traffic environments, because these downstream tasks require making corresponding decisions based on the position and information of moving and static objects.

[0093] The segmentation network model is trained using the following steps:

[0094] S1. Organize the Semantic KITTI dataset data file and obtain several consecutive key timestamps before and after the point cloud data in the original Semantic KITTI dataset to be processed.

[0095] S2. Preprocess the point cloud data by performing self-motion estimation and BEV projection on the acquired point cloud data in sequence, and retain the pixel coordinates and corresponding feature values ​​of the projected images of all point clouds, and then calculate the inter-frame difference features.

[0096] S3. Construct a segmentation network model.

[0097] S4. Design the loss function of the segmentation model and use the backpropagation algorithm to optimize and update the model parameters of each layer in the segmentation network model in order to train the segmentation network model constructed in step S3.

[0098] S5. Apply the segmentation network model to test the model on the test data.

[0099] S6 outputs the state prediction mask and semantic prediction mask for obstacle segmentation, completing the segmentation and recognition of moving obstacles in autonomous driving scenarios.

[0100] In step S1, before experimental analysis, the Semantic KITTI data sequence files use publicly available 00-21 sequences and are divided into training, validation, and test set files according to a common allocation ratio. For example, the 00-10 sequence files contain publicly available ground truth values, with 00-07 and 09-10 used for training, sequence 08 used for validation, and sequence 11-21 used for testing. The point cloud data format is (num_point, 4).

[0101] In step S4, model parameters are updated based on the prediction masks (i.e., state prediction masks and semantic prediction masks) and the target mask to construct the optimal segmentation model. The loss function value is calculated based on the prediction masks and the target mask, and backpropagation is performed on the constructed segmentation model to update the model parameters of each layer. The loss value of the model's prediction mask is calculated, and the segmentation result is output. The cross-entropy function is used as part of the model's loss function. Since the task of this invention is a segmentation task, the Lovasz Softmax function is introduced to incorporate semantic segmentation loss while calculating the segmentation loss for moving obstacles. Therefore, the corresponding overall loss function calculation formula is as follows:

[0102] Loss = L semantic +L moving +L ls

[0103] Where Loss is the overall loss function, L semantic Let L be the semantic cross-entropy loss function. moving This is the loss function for moving obstacles. Due to the severe imbalance between the proportion of moving and static objects in point cloud data, the reciprocal of the proportion of different object categories in the environment is used as the weighting coefficient for the loss calculation of the corresponding obstacle category when calculating the loss function, thereby alleviating the class imbalance problem.

[0104] The parameters and function settings used during BEVMos network training and model parameter updates are as follows:

[0105] The network was trained using a stochastic gradient descent optimizer. To obtain a suitable learning rate, a learning rate decay strategy was applied, with an initial learning rate of 0.001. To avoid overfitting, a weight decay method was applied, with a coefficient set to 0.0001.

[0106] In this invention, accuracy (ACC), intersection-to-union ratio (IoU), and network inference time are used as metrics for selecting voxel sizes. IoU quantifies the percentage overlap between the target mask and the prediction mask, which is the ratio of the number of pixels in the common area between the target mask and the prediction mask to the total number of pixels in both. The calculation formula is as follows:

[0107]

[0108] Where IoU represents the Intersection over Union ratio, target∧prediction represents the number of pixels in the common region between the target mask and the prediction mask, and target∪prediction represents the total number of pixels between the target mask and the prediction mask. The final trained segmentation network model is obtained based on the evaluation metrics.

[0109] Based on the above-described LiDAR-based method for segmenting moving obstacles in autonomous driving, this invention also provides an embodiment of a computer device.

[0110] The computer device of the present invention includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0111] Acquire time-series point cloud data with several consecutive timestamps in an autonomous driving scenario; wherein, the time-series point cloud data with several consecutive timestamps includes the time-series point cloud data with the current timestamp;

[0112] Based on time-series point cloud data with several consecutive timestamps, determine the inter-frame difference feature information of each timestamp;

[0113] The inter-frame difference feature information of each timestamp and the projection data of the temporal point cloud data of the current timestamp are input into the trained segmentation network model to obtain the state prediction mask and semantic prediction mask for obstacle segmentation.

[0114] The trained segmentation network model includes: a temporal fusion module, an asymmetric encoder-decoder, and a point cloud data recovery module;

[0115] The step of inputting the inter-frame difference feature information of each timestamp and the projection data of the temporal point cloud data of the current timestamp into the trained segmentation network model to obtain the state prediction mask and semantic prediction mask for obstacle segmentation includes:

[0116] The inter-frame difference feature information of each timestamp and the projection data of the temporal point cloud data of the current timestamp are input into the temporal fusion module to obtain the fusion feature;

[0117] The fused features are input into the asymmetric encoder and decoder to obtain feature information data;

[0118] The feature information data is input into the point cloud data recovery module to obtain the state prediction mask and semantic prediction mask for obstacle segmentation.

[0119] Based on the above-described method for segmenting moving obstacles in autonomous driving using LiDAR, this invention also provides an embodiment of a computer-readable storage medium.

[0120] The present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, performs the following steps:

[0121] Acquire time-series point cloud data with several consecutive timestamps in an autonomous driving scenario; wherein, the time-series point cloud data with several consecutive timestamps includes the time-series point cloud data with the current timestamp;

[0122] Based on time-series point cloud data with several consecutive timestamps, determine the inter-frame difference feature information of each timestamp;

[0123] The inter-frame difference feature information of each timestamp and the projection data of the temporal point cloud data of the current timestamp are input into the trained segmentation network model to obtain the state prediction mask and semantic prediction mask for obstacle segmentation.

[0124] The trained segmentation network model includes: a temporal fusion module, an asymmetric encoder-decoder, and a point cloud data recovery module;

[0125] The step of inputting the inter-frame difference feature information of each timestamp and the projection data of the temporal point cloud data of the current timestamp into the trained segmentation network model to obtain the state prediction mask and semantic prediction mask for obstacle segmentation includes:

[0126] The inter-frame difference feature information of each timestamp and the projection data of the temporal point cloud data of the current timestamp are input into the temporal fusion module to obtain the fusion feature;

[0127] The fused features are input into the asymmetric encoder and decoder to obtain feature information data;

[0128] The feature information data is input into the point cloud data recovery module to obtain the state prediction mask and semantic prediction mask for obstacle segmentation.

[0129] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A method for segmenting moving obstacles in autonomous driving based on lidar, characterized in that, Including the following steps: Acquire time-series point cloud data with several consecutive timestamps in an autonomous driving scenario; wherein, the time-series point cloud data with several consecutive timestamps includes the time-series point cloud data with the current timestamp; Based on time-series point cloud data with several consecutive timestamps, determine the inter-frame difference feature information of each timestamp; The inter-frame difference feature information of each timestamp and the projection data of the temporal point cloud data of the current timestamp are input into the trained segmentation network model to obtain the state prediction mask and semantic prediction mask for obstacle segmentation. The trained segmentation network model includes: a temporal fusion module, an asymmetric encoder-decoder, and a point cloud data recovery module; The step of inputting the inter-frame difference feature information of each timestamp and the projection data of the temporal point cloud data of the current timestamp into the trained segmentation network model to obtain the state prediction mask and semantic prediction mask for obstacle segmentation includes: The inter-frame difference feature information of each timestamp and the projection data of the temporal point cloud data of the current timestamp are input into the temporal fusion module to obtain the fusion feature; The fused features are input into the asymmetric encoder and decoder to obtain feature information data; The feature information data is input into the point cloud data recovery module to obtain the state prediction mask and semantic prediction mask for obstacle segmentation.

2. The method for segmenting moving obstacles in autonomous driving based on lidar according to claim 1, characterized in that, The asymmetric codec includes: an encoder, a context information module, a motion segmentation decoder, and a semantic information decoder; the feature information data includes: an obstacle motion state prediction mask and a semantic prediction mask for moving obstacles; The fused features are input into the asymmetric encoder and decoder to obtain feature information data, including: The fused features are input into the encoder to obtain the encoded features; The encoded features are input into the context information module to obtain a feature image; The feature image is input into the motion segmentation decoder to obtain the obstacle motion state prediction mask; The feature image is input into the semantic information decoder to obtain the semantic prediction mask of the moving obstacle.

3. The method for segmenting moving obstacles in autonomous driving based on lidar according to claim 2, characterized in that, The context information module includes: four parallel dilated convolutional layers, with dilation rates of 6, 12, 18 and 24 for the four parallel dilated convolutional layers, respectively. The encoder, the motion segmentation decoder, and the semantic information decoder each include four encoding modules, each of which includes a convolutional layer, a batch normalization layer, a residual module, and an activation function layer.

4. The method for segmenting moving obstacles in autonomous driving based on lidar according to claim 1, characterized in that, The step of determining the inter-frame difference feature information of each time stamp based on time-series point cloud data with several consecutive timestamps includes: All point cloud coordinate information of past timestamps are uniformly transformed to the current timestamp point cloud data coordinate system according to pose transformation, and the time-series point cloud data of continuous timestamps are projected according to affine transformation to obtain the projection data of time-series point cloud data of each timestamp. For the projection data of the time-series point cloud data of each past time point in a series of consecutive time stamps, the inter-frame difference feature information of the past time stamp is obtained by performing a pixel-by-pixel multiplication operation and normalization based on the projection data of the past time stamp and the projection data of the current time stamp.

5. The method for segmenting moving obstacles in autonomous driving based on lidar according to claim 4, characterized in that, The inter-frame difference feature information is as follows: Where diff represents inter-frame difference feature information, SP current SP represents the projected data of the time-series point cloud data at the current timestamp. i This represents the projection data of the time-series point cloud data at the i-th past timestamp.

6. The method for segmenting moving obstacles in autonomous driving based on lidar according to claim 1, characterized in that, The LiDAR-based method for segmenting moving obstacles in autonomous driving also includes: Based on the state prediction mask of the obstacle segmentation, the static / dynamic category of the obstacle is determined, and based on the semantic prediction mask of the obstacle segmentation, the semantic category of the obstacle is determined.

7. The method for segmenting moving obstacles in autonomous driving based on lidar according to claim 1, characterized in that, The trained segmentation network model updates its model parameters based on an overall loss function during training; the overall loss function is: Loss=L semantic +L moving +L ls Where Loss represents the overall loss function, L semantic L represents the semantic cross-entropy loss function. moving L represents the loss function for moving obstacles. ls This represents the Lovasz Softmax loss function.

8. The method for segmenting moving obstacles in autonomous driving based on lidar according to claim 1, characterized in that, The trained segmentation network model is evaluated during training based on accuracy, intersection-over-union ratio (IoU), and network push time; where IoU is: Where IoU represents the intersection-union ratio, target∧prediction represents the number of pixels in the common region between the target mask and the prediction mask, and target∪prediction represents the total number of pixels between the target mask and the prediction mask.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images