Semantic Information-Guided Point Cloud Moving Target Segmentation Method

Abstract

This invention discloses a point cloud moving target segmentation method guided by semantic information, comprising the following steps: 1) enhancing the moving point cloud samples and setting random numbers to control whether to take samples from two consecutive frames with a one-frame interval; 2) for the two frames of point cloud data, obtaining corresponding two-dimensional images through spherical projection, and calculating the distance image and residual image; 3) inputting the two frames of two-dimensional images into a pre-trained semantic segmentation network to obtain corresponding semantic feature maps, and aligning the semantic feature map of the previous frame to the coordinate system of the current frame; 4) designing a cross-fusion module guided by semantic information to fuse the distance image, residual image, and semantic feature map, and inputting it into the moving target segmentation network to obtain the motion segmentation result of the current frame image; 5) post-processing the segmentation result to obtain the motion state of each frame of point cloud. This invention introduces semantic information into the point cloud moving target segmentation task for the first time, which can well balance segmentation effect and segmentation efficiency.

Description

Technical Field

[0001] This invention belongs to the field of 3D point cloud and pattern recognition, specifically relating to a point cloud moving target segmentation method based on semantic information guidance. Background Technology

[0002] With the continuous development of autonomous driving, environmental perception has become increasingly important. Distinguishing between moving and stationary targets in the surrounding environment is a crucial task in the field of autonomous driving. Due to the lack of ground-value labeled LiDAR motion segmentation data, previous moving target segmentation methods were mainly based on camera images. In recent years, with the release of the SemanticKITT moving target segmentation dataset, LiDAR-based moving target segmentation has gradually received more and more attention.

[0003] Current research on LiDAR point clouds primarily focuses on semantic segmentation, relying on the contours of objects in a single frame of the point cloud to learn the corresponding category features, thus determining whether an object in the point cloud is movable or immovable. However, it cannot estimate the object's current motion state. Considering the impact of moving target segmentation accuracy on tasks such as navigation safety, pose estimation, map building, and path planning, exploring a moving target segmentation method that can simultaneously guarantee reliability and real-time performance is a necessary and challenging task. Furthermore, most existing LiDAR-based moving target segmentation methods only focus on separating moving and static targets, neglecting the integrity of the entire autonomous driving system and the correlation between semantic segmentation and moving target segmentation. Performing the moving target segmentation task from scratch results in a waste of computational resources and makes it difficult to simultaneously guarantee accuracy and real-time performance.

[0004] In most scenarios, dynamic targets tend to account for a very small proportion compared to static targets. This unbalanced distribution increases the difficulty of deep learning-based moving target segmentation tasks and severely affects the accuracy of segmentation. Summary of the Invention

[0005] The purpose of this invention is to propose a point cloud moving target segmentation method based on semantic information guidance, which improves accuracy while meeting real-time requirements.

[0006] The technical solution to achieve the purpose of this invention is as follows: In a first aspect, this invention provides a point cloud moving target segmentation method based on semantic information guidance, comprising the following steps:

[0007] Step 1: Randomly perform data augmentation on the prepared LiDAR point cloud data training set; generate a random number to determine whether this set of training data should be taken at one frame interval;

[0008] Step 2: Convert the lidar point cloud data into a two-dimensional image through spherical projection to obtain the corresponding distance image and residual image;

[0009] Step 3: Using the distance image as input, obtain the semantic feature map based on the pre-trained semantic segmentation network, and align the semantic feature map of the previous frame to the coordinate system of the current frame;

[0010] Step 4: Through the cross-fusion module, the distance image, residual image and semantic feature map are fused together and input into the moving target segmentation deep learning network to obtain the motion state prediction result of the current frame. The loss is calculated with the labeled ground truth, and the training is iterated until convergence.

[0011] Step 5: Through KNN post-processing, the prediction results are projected back from the two-dimensional image to the point cloud to obtain the motion state prediction results of each point in the point cloud.

[0012] In a second aspect, the present invention provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the method described in the first aspect.

[0013] Compared with the prior art, the technical solution proposed in this invention has the following advantages: (1) For the first time, semantic segmentation information is introduced into the moving target segmentation task of laser radar point cloud, and the correlation between the moving target segmentation task and the semantic segmentation task is established; the point cloud semantic information obtained by the pre-trained semantic segmentation network model provides basic spatial geometric information for segmenting moving targets, and fewer consecutive data frames can be used than other methods. That is, this invention only needs to use two consecutive frames to improve the segmentation effect while meeting the real-time requirements; (2) In the process of image convolution, traditional methods only use the feature vectors of adjacent pixels of the image to perform corresponding convolution operations, ignoring the projection of the point cloud onto the two-dimensional image. Before the 3D image, the Euclidean distance between points in the point cloud is used. Before using the image for convolution, the present invention introduces a cross-fusion module, which takes the Euclidean distance between the points corresponding to the pixels in the surrounding 3x3 rectangle and the points corresponding to the center pixel as input. The multilayer perceptron (MLP) learns the change in the contribution weight of the surrounding points to the center point by the distance difference, thereby affecting the feature vector of the center point; (3) In the case of an imbalance between dynamic and static sample datasets, the present invention proposes to take two frames across frame intervals to introduce motion samples with different speeds, which increases the diversity of motion samples and enables the trained model to better adapt to different scenarios. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating a point cloud moving target segmentation method based on semantic information guidance according to the present invention.

[0015] Figure 2This is a schematic diagram of the cross-fusion module of a point cloud moving target segmentation method based on semantic information guidance according to the present invention. Detailed Implementation

[0016] This invention proposes a point cloud moving target segmentation method guided by semantic information, comprising the following steps:

[0017] Step 1: Randomly perform data augmentation on the prepared LiDAR point cloud data training set; generate a random number to determine whether this set of training data is taken at intervals of one frame, thereby increasing the diversity of training data and avoiding the motion data being too monotonous;

[0018] Step 2: Take a set of point cloud data PointCloud0 (current frame point cloud data) and PointCloud1 (previous frame point cloud data) and transform them into two-dimensional images through spherical projection to obtain the corresponding distance images RangeImage0, RangeImage1 and residual images ResidualImage; Take the truth label PointCloudLables0 corresponding to PointCloud0 and project it onto the two-dimensional image ImageLabelSet0.

[0019] Step 3: Using the distance images RangeImage0 and RangeImage1 as input, extract semantic features based on the pre-trained semantic segmentation network SalsaNext to obtain feature maps SemanticFeatures0 and SemanticFeatures1, with the data format being [h,w,20]. Align the semantic feature map of the previous frame to the coordinate system of the current frame to generate a new semantic feature map SemanticFeatures1′.

[0020] Step 4: Through the cross-fusion module, the distance image, residual image and semantic feature map are fused together and input into the moving target segmentation network MOSNet to obtain the current frame motion state prediction probability feature map PredictSet. The maximum number of iterations is set to 30, and iterative training is performed until convergence. The model is then saved and tested for verification.

[0021] Step 5: Set the network parameters according to the above training model, input the test data, and obtain the motion state probability feature map PredictSet when the current frame point cloud is projected onto the two-dimensional image. From this, the motion target segmentation result ImageResult is obtained. Through KNN (K-Nearest Neighbor) post-processing, the ImageResult is projected from the two-dimensional image back to the three-dimensional point cloud to obtain the motion state prediction result PointCloudResult of each point in the current frame point cloud.

[0022] Furthermore, in step 1, data augmentation is performed by taking two frames of point cloud data as a group. There are two ways to take a group of point cloud data: one way is to take two consecutive frames of point cloud data with a one-frame interval (such as taking the first and third frames of data in the same sequence as a group), with a probability of 0.5; the other way is to take two consecutive frames of point cloud data without interval (such as taking the first and second frames of data in the same sequence as a group), with a probability of 0.5 as well. The method of taking a group of point cloud data is controlled by generating random numbers, thereby increasing the diversity of motion point clouds.

[0023] Furthermore, in step 2, the point cloud PointCloud1 from the previous frame is aligned to the coordinate system of the current frame to obtain PointCloud1′, calculated using the following formula:

[0024]

[0025] Where (x1, y1, z1) represents the original coordinate values ​​of the previous frame's point cloud data PointCloud1. This represents the relative pose of the point cloud in the previous frame and the point cloud in the current frame. (x′1,y′1,z1′) represents the coordinate values ​​of the new point cloud PointCloud1′ formed by transforming the point cloud in the previous frame to the coordinate system of the current frame.

[0026] Furthermore, in step 2, the point cloud data format for each frame is [n,4] (n represents the total number of points in the frame, 4 represents the four feature channels x, y, z, and i, where x, y, and z represent the coordinates of the point along the x, y, and z coordinate systems, respectively, and i represents the reflection intensity information of the point), which is then transformed into a two-dimensional image [h,w,5] through spherical projection (h represents the height of the two-dimensional image, determined by the number of lines of the LiDAR; w represents the width of the two-dimensional image; 5 represents the five feature channels r, x, y, z, and i corresponding to each pixel, where r represents the Euclidean distance from the point to the center of the LiDAR). The formula for spherical projection is as follows:

[0027]

[0028] Where (u,v) represents the coordinates of a point projected onto the distance image from its three-dimensional spatial coordinates (x,y,z), and θ yaw and θ pitch Let H and W represent the yaw and pitch angles of the point, respectively, and let f = f_0. up -f down This indicates the vertical field of view of the lidar.

[0029] Furthermore, in step 2, two consecutive point clouds, PointCloud0 and PointCloud1′, are converted into distance images, corresponding to RangeImage0 and RangeImage1′ respectively. The residual image ResidualImage between the two images is calculated, where each pixel value is the relative difference between the depth value r0 of the current frame and the depth value r1′ of the previous frame, with the data format [h,w,1]. The calculation formula is as follows:

[0030]

[0031] Where r0 and r1′ represent the r-channel values ​​of the current frame image and the r-channel values ​​of the previous frame after alignment with the current coordinate system, respectively, and t1 and t2 are the set residual thresholds and noise filtering points.

[0032] Furthermore, in step 3, the pre-trained semantic segmentation network model uses SalsaNext. The distance images RangeImage0 and RangeImage1 are input into the semantic segmentation network to obtain the semantic segmentation results SemanticFeatures0 and SemanticFeatures1 of the current frame and the previous frame, respectively. Using the coordinate mapping information from the previous frame to the current frame, the semantic feature map of the previous frame is aligned to the coordinate system of the current frame, generating a new semantic feature map SemanticFeatures1′. The formula for aligning the semantic feature image of the previous frame to the coordinate system of the current frame is as follows:

[0033]

[0034] Among them, Tr is used to store the subscript mapping information from the previous frame image coordinate system to the current frame coordinate system, and maps the semantic feature vector of the previous frame image coordinate (u1,v1) to the corresponding position (u0,v0) in the current frame coordinate system as its semantic feature vector.

[0035] Furthermore, in step 4, a cross-fusion module is designed to fuse the distance image, residual image, and semantic feature map, resulting in a data format of [h,w,46] (distance image [h,w,5]; residual image [h,w,1]; current frame semantic feature map [h,w,20]; previous frame semantic feature map [h,w,20]). This data is used as input to the moving target segmentation network. The cross-fusion module in the moving target segmentation network first takes the distance image RangeImage0 as input and trains it using a multilayer perceptron (MLP) to obtain the contribution of each pixel in the current frame image to its neighboring pixels within a 3x3 area. This contribution is calculated based on the neighboring pixels within the 3x3 area and the contribution of the center pixel. The physical distance difference is used as the main influencing factor to obtain the corresponding contribution weight. The weights of adjacent pixels within a 3x3 range are multiplied by their corresponding feature vectors to obtain the feature vector of the center point. This yields the initial feature vector for each pixel in the current frame image, which is then input into the subsequent segmentation network module MOSNet. The segmentation head is set to 3, meaning the final number of categories is 3 (label=0 indicates unknown; label=1 indicates stationary state; label=2 indicates moving state). This produces the moving target segmentation probability feature map PredictSet for the current frame image. This map, along with the ground truth label ImageLabelSet0, is input into the loss functions Cross-Entropy Loss and Lovász-Softmax Loss to calculate the loss value. The model is iteratively trained until the loss function converges (the maximum number of training iterations is set to epoch=30, and the epoch value is as large as possible). The trained model is then saved, and the test results are verified.

[0036] Furthermore, in step 4, the moving target segmentation network MOSNet comprises four main components: context fusion, a downsampling module, an upsampling module, and a segmentation head. The context extraction module abstracts the initial feature map into a high-level feature map. The downsampling module compresses the feature map resolution, and the upsampling module expands the resolution in a symmetrical manner with the downsampling module, ultimately restoring the image to its original resolution. The input data is a feature map in the format [h,w,46], and the output is a PredictSet of different class probability images in the format [h,w,3]. Finally, the class with the highest probability value is taken as the segmentation class, resulting in a label image ImageResult in the format [h,w,1].

[0037] Furthermore, in step 5, based on the trained moving target segmentation model, a moving target segmentation method based on point cloud projection onto a two-dimensional image is implemented. Through KNN (K-Nearest Neighbor) post-processing, the two-dimensional image segmentation result ImageResult is projected back to the three-dimensional point cloud result PointCloudResult. The input image data format is [h,w,1], and the output point cloud target segmentation result format is [n,1].

[0038] The following is in conjunction with the appendix Figure 1-2 The present invention will be described in further detail below. The following embodiments or accompanying drawings are used to illustrate the present invention, but should not be used to limit the scope of the invention.

[0039] Example

[0040] like Figure 1 As shown, this embodiment provides a point cloud moving target segmentation method based on semantic information guidance, including the following steps:

[0041] Step 1: Perform random data augmentation on the prepared training set; generate a random number to determine whether the training data should be taken at intervals of one frame, thereby increasing the diversity of the training data and avoiding overly homogeneous motion data.

[0042] The 3D point cloud SemanticKITTI moving target segmentation dataset includes 22 point cloud sets from 00 to 21. Sequences 00-07 and 09-10 are used as the training set, sequence 08 as the validation set, and sequences 11-21 as the test set. During model training, point cloud data is grouped into sets of two. There are two methods for selecting a set of point cloud data: one method is to select two consecutive frames of point cloud data with a one-frame interval (e.g., selecting the first and third frames of the same sequence as a set), with a probability of 0.5; the other method is to select two consecutive frames of point cloud data without any interval (e.g., selecting the first and second frames of the same sequence as a set), also with a probability of 0.5. The method of selecting a set of point cloud data is controlled by a generated random number, Random.

[0043] Step 2: Using the training point cloud set as input, take a set of point cloud data PointCloud0 (current frame point cloud data) and PointCloud1 (previous frame point cloud data) and transform them into two-dimensional images through spherical projection to obtain the corresponding distance images RangeImage0, RangeImage1 and residual images ResidualImage; take the ground truth label PointCloudLables0 corresponding to PointCloud0 and project it onto the two-dimensional image ImageLabelSet0.

[0044] Align the previous frame's point cloud PointCloud1 to the current frame's coordinate system to obtain PointCloud1′. The calculation formula is as follows:

[0045]

[0046] Where (x1, y1, z1) represents the original coordinate values ​​of PointCloud1 from the previous frame of point cloud data. This represents the relative pose of the point cloud in the previous frame and the point cloud in the current frame. (x1′, y1′, z1′) represents the coordinate values ​​of the new point cloud PointCloud1′ formed by transforming the point cloud in the previous frame to the coordinate system of the current frame.

[0047] Spherical projection maps each point in the point cloud to its corresponding coordinates on a two-dimensional image. The feature vector of a point represents the initial information of the corresponding pixel in the two-dimensional image. When multiple points are mapped to the same pixel, only the point with the smallest r value is selected, i.e., the closest point.

[0048] Each frame of point cloud data is formatted as [n, 4] (where n represents the total number of points in the frame, 4 represents the four feature channels x, y, z, i, where x, y, z represent the coordinates of the point along the x, y, z coordinate systems, and i represents the reflection intensity information of the point), and is transformed into a two-dimensional image [64, 2048, 5] through spherical projection (64 represents the height of the two-dimensional image; 2048 represents the width of the two-dimensional image; 5 represents the five feature channels r, x, y, z, i corresponding to each pixel, where r represents the Euclidean distance from the point to the center of the lidar). The formula for spherical projection is as follows:

[0049]

[0050] Where (u,v) represents the coordinates of a point projected onto the distance image from its three-dimensional spatial coordinates (x,y,z), and θ yaw and θ pitch Let H and W represent the yaw and pitch angles of the point, respectively, and let H and W represent the height (64) and width (2048) of the 2D image, respectively. f = f up -f down The vertical field of view of the lidar is represented by the vector {r,x,y,z,i}.

[0051] Two consecutive point cloud frames, PointCloud0 and PointCloud1′, are transformed into distance images, corresponding to RangeImage0 and RangeImage1′ respectively. A residual image, ResidualImage, is calculated from these two distance images. The value of each pixel is the relative difference between the depth value r0 of the current frame and the depth value r1′ of the previous frame, with a data format of [64, 2048, 1]. The calculation formula is as follows:

[0052]

[0053] Where r0 and r1′ represent the r-channel value of the pixel in the current frame image RangeImage0 and the r-channel value of the same pixel coordinate in the previous frame image RangeView1′ after coordinate system alignment, respectively. t1 and t2 are the set residual thresholds to filter noise points, t1=0 and t2=0.8.

[0054] Step 3: Using distance images RangeImage0 and RangeImage1 as input, semantic feature maps SemanticFeatures0 and SemanticFeatures1 are obtained based on the pre-trained semantic segmentation network SalsaNext. The data format is [64, 2048, 20]. The semantic feature map SemanticFeatures1 from the previous frame is aligned to the coordinate system of the current frame to generate a new semantic feature map SemanticFeatures1′. The semantic feature map alignment mapping formula is as follows:

[0055]

[0056] Among them, Tr is used to store the subscript mapping information from the previous frame image coordinate system to the current frame coordinate system. Therefore, the semantic feature vector of the previous frame image coordinate (u1, v1) is mapped to the corresponding position (u0, v0) in the current frame coordinate system as the semantic feature vector of the pixel.

[0057] Since point cloud projection onto a 2D image results in multiple points projecting to the same pixel, but only the closest point is retained, during the projection of PointCloud1 onto RangeImage1, only a portion of the points in PointCloud1 are retained, denoted as PointCloud1″. PointCloud1″ is aligned to the current frame coordinate system and denoted as PointCloud1″′. Following the left-to-right and top-to-bottom order in RangeImage1, the coordinates (x1′, y1′, z1′) of each pixel (u1, v1) are read. Based on the spherical projection formula, the pixel point (u0, v0) corresponding to the current frame image is recalculated. The semantic feature map SemanticFeatures1′ after coordinate alignment in the previous frame is constructed, where the feature vector corresponding to the pixel point (u0, v0) in SemanticFeatures1′ is the feature vector corresponding to the pixel coordinate (u1, v1) in SemanticFeatures1.

[0058] Step 4: The distance image, residual image and semantic feature map are fused together through the cross-fusion module and input into the moving target segmentation network MOSNet to obtain the motion state prediction result PredictSet for the current frame. The maximum number of iterations is set to 30, and the model is trained iteratively until convergence. The model is then saved and tested for verification.

[0059] The prepared data includes: RangeImage0 (data format [64,2048,5]), ResidualImage (data format [64,2048,1]), SemanticFeatures0 (data format [64,2048,20]), and SemanticFeatures1′ (data format [64,2048,20]). These data are concatenated along the feature dimension to obtain the feature map F as input data, with a data format of [64,2048,46].

[0060] Cross-integration modules such as Figure 2 As shown.

[0061] Calculate the initial feature vector for each point in the current frame distance image RangeImage0.

[0062] When calculating the initial feature vector of any pixel, we first take that point as the center, denoted as P. i,j Calculate the corresponding weight vector for each pixel within a 3x3 range, i.e., {(u i+k1 ,v j+k2 There are 9 pixels: k1 = -1, 0, 1; k2 = -1, 0, 1. The vector value corresponding to adjacent pixels is (r... k ,x k ,y k ,z k i k ), k = 1, 2, 3...9, center point (u i ,v j The corresponding vector value is (r0, x0, y0, z0, i0), and the corresponding difference (r) is taken. k -r0,x k -x0,y k -y0,z k -z0,i k -i0), k=1,2,3···9 are used as inputs to the MLP part of the cross-fusion module, and the corresponding weights are trained based on the actual spatial distance between points. Concatting neighboring points, i.e., the initial feature value of each center point is composed of the product of the weights of all points within its adjacent 3x3 range, is calculated using the following formula:

[0063]

[0064] Where Fi+k1,j+k2 are the vector feature values ​​corresponding to the pixel coordinates (i+k1,j+k2) in the feature map F, and F′ is the new feature map generated after feature cross-fusion. F′ is passed through a 1x1 convolutional layer to obtain the output feature map F″.

[0065] The feature map F″ is input into the moving target segmentation deep learning network MOSNet, which is identical to the semantic segmentation network SalsaNext module, except that the number of classes set in the final network segmentation head is modified. Semantic segmentation is set to 20 classes, and moving target segmentation is set to 3 classes, resulting in the moving target segmentation class probability feature map PredictSet. The data format is [64, 2048, 3], where 64 is the height of the feature map, 2048 is the width, and 64x2048 describes the scale of the feature map. The 3 indicates that each pixel in the feature map is represented by a 3D vector. In the 64x2048 scale feature map PredictSet, each pixel value is a vector [pro]. i0 ,pro i1 ,pro i2 The probability of the pixel being 0 (undetermined), 1 (static point), or 2 (dynamic point) is respectively assigned. The category with the highest probability is taken as the segmentation result of the pixel, and the image motion segmentation result ImageResult is obtained. The data format is [64,2048,1].

[0066] Substitute PredictSet and the true value ImageLabelSet0 into the loss functions Cross-Entropy Loss and Lovász-Softmax Loss to calculate the loss value. Set the maximum number of iterations epoch = 30 and iterate the above process until the loss function converges. Save the model and perform testing and verification.

[0067] Step 5: Set network parameters based on the training data obtained from the above model, input the test data, and obtain the image-based moving target segmentation result ImageResult. After KNN (K-Nearest Neighbor) post-processing, project ImageResult back from the image to the point cloud to obtain the motion state prediction result PointCloudResult for each point in the point cloud.

[0068] Take each pixel in the distance image RangeImage0, define a window of size 5x5, use each pixel in the image as the center of the window, and take the neighboring points within the 5x5 window as a set N′, with the data format [64*2048, 5*5].

[0069] For each point in the current frame point cloud PointCloud0, multiple points may be projected onto the same two-dimensional image pixel, so the actual number of point cloud points may be greater than 64*2048. By the correspondence between the three-dimensional point cloud and the two-dimensional pixel, the neighbor points of each point in the point cloud on the two-dimensional image are obtained, denoted as set N″, with the data format [n, 5*5].

[0070] The motion segmentation category of each point in the point cloud depends on the consensus vote of the K closest points in 3D space (calculated by the absolute difference r between the points). K is set to 5. The KNN algorithm is used to search for the 5 nearest points to the target point as its neighbors, which are set as N, with the data format [n, 5].

[0071] The absolute difference of r between points should be less than the threshold threshold = 1.0. Neighbor points with a distance greater than the threshold are filtered out before consensus voting. The motion segmentation category of each neighbor point can be obtained from ImageResult. The category with the most frequent category in the neighbor point set is taken as the category of the target point, and the motion target segmentation result PointCloudResult of the entire point cloud is obtained. The data format is [n,1].

[0072] When using the point cloud motion segmentation model trained by this invention for motion state segmentation, it can effectively obtain the motion state category of each point in the point cloud. Compared with previous methods, the introduction of semantic information can achieve accurate segmentation with fewer point cloud frames. The model was trained on the SemanticKITTI motion target segmentation dataset 00-07 and 09-10 sequences and tested on the 11-21 sequence, achieving an intersection-over-union ratio of 69.8%.

[0073] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A point cloud moving target segmentation method based on semantic information guidance, characterized in that, Includes the following steps: Step 1: Randomly perform data augmentation on the prepared lidar point cloud data training set; Generate a random number to determine whether this set of training data should be taken at one-frame intervals; Step 2: Convert the lidar point cloud data into a two-dimensional image through spherical projection to obtain the corresponding distance image and residual image; Step 3: Using the distance image as input, obtain the semantic feature map based on the pre-trained semantic segmentation network, and align the semantic feature map of the previous frame to the coordinate system of the current frame; The pre-trained semantic segmentation network model uses SalsaNext to simultaneously obtain the semantic segmentation results of the current frame and the previous frame. Using the coordinate alignment information from the previous frame to the current frame, the semantic feature map of the previous frame is aligned to the coordinate system of the current frame. The formula for aligning the semantic feature map of the previous frame to the coordinate system of the current frame is as follows: Among them, Tr is used to store the subscript mapping information from the previous frame image coordinate system to the current frame coordinate system, and maps the semantic feature vector of the previous frame image coordinate (u1,v1) to the corresponding position (u0,v0) in the current frame coordinate system as its semantic feature vector. Step 4: Through the cross-fusion module, the distance image, residual image and semantic feature map are fused together and input into the moving target segmentation deep learning network to obtain the motion state prediction result of the current frame. The loss is calculated with the labeled ground truth, and the training is iterated until convergence. Design a cross-fusion module to fuse the distance image, residual image, and semantic feature map to obtain a data format [h,w,46] as input to the moving target segmentation network, where the distance image is [h,w,5], the residual image is [h,w,1], the semantic feature map of the current frame is [h,w,20], and the semantic feature map of the previous frame is [h,w,20]. The cross-fusion module in the moving target segmentation network first uses the distance image as input and trains a multilayer perceptron to obtain the contribution of each pixel in the current frame image to its neighboring pixels within a 3x3 radius. That is, it calculates the contribution based on the relationship between the neighboring pixels within the 3x3 radius and the center pixel. The physical distance difference between pixels is used as an influencing factor to obtain the corresponding contribution weight. The weights of adjacent pixels within a 3x3 radius are multiplied by their corresponding feature vectors to obtain the feature vector of the center point. This yields the initial feature vector for each pixel in the current frame image, which is then input into the subsequent point cloud moving target segmentation network MOSNet. The number of segmentation head categories is set to 3, with label=0 indicating unknown, label=1 indicating a stationary state, and label=2 indicating a moving state. This yields the moving target segmentation result of the current frame image, which is then compared with the original labeled ground values ​​and input into the loss functions Cross-Entropy Loss and Lovász-Softmax Loss to calculate the loss value. The network is iteratively trained until it converges. The moving target segmentation network includes a context extraction module, a downsampling module, an upsampling module, and a segmentation head. The context extraction module abstracts the initial feature map into a high-level feature map. The downsampling module compresses the feature map resolution, and the upsampling module expands the resolution in a symmetrical manner with the downsampling module, ultimately restoring the image to its original resolution. The input data is a feature map in the format [h,w,46], and the output is a probability image of different categories in the format [h,w,3]. Finally, the category with the highest probability value is taken as the segmentation category, resulting in a label image in the format [h,w,1]. Step 5: Through KNN post-processing, the prediction results are projected from the 2D image back to the point cloud to obtain the motion state prediction results of each point in the point cloud; based on the trained moving target segmentation model, the moving target segmentation task based on the point cloud projected onto the 2D image is realized. The moving target segmentation of the point cloud is realized through KNN post-processing; taking each point as the center point, the K points closest to the center point are found in the 3D point cloud space, and the category segmentation results of the image pixels projected by the K nearest points are counted. The category with the most categories is taken as the segmentation category of the center point. The input image data format is [h,w,1], and the output point cloud target segmentation result format is [n,1].

2. The point cloud moving target segmentation method based on semantic information guidance according to claim 1, characterized in that, In step 1, a random number Random is generated. When the random number is greater than or equal to 0.5, two adjacent frames of point cloud data are taken as a group after an interval of one frame. Otherwise, no interval is required, and two consecutive frames of point cloud data are taken as a group.

3. The point cloud moving target segmentation method based on semantic information guidance according to claim 1, characterized in that, In step 2, the point cloud format [n,4] is transformed into a two-dimensional image [h,w,5] through spherical projection. Here, n represents the number of points in the point cloud, 4 represents the four feature channels x, y, z, and i, and 5 represents the five feature channels r, x, y, z, and i corresponding to each pixel. The formula for spherical projection is as follows: Where (u,v) represents the coordinates of a point projected onto the distance image from its three-dimensional spatial coordinates (x,y,z), and θ yaw and θ pitch Let H and W represent the yaw and pitch angles of the point, respectively, and let f = f[0, 0, 0]. up -f down This indicates the vertical field of view of the lidar.

4. The point cloud moving target segmentation method based on semantic information guidance according to claim 3, characterized in that, In step 2, the input data is two frames of point cloud data in the format [n,3]. After being projected onto a 2D image, the format of both frames becomes [h,w,5]. The data from the previous frame is aligned to the current frame through relative pose. Based on the difference in the r-channel of the two adjacent 2D image data frames, the r-channel values ​​of the two frames are subtracted to obtain a residual image [h,w,1]. The coordinate alignment formula is as follows: Where (x1,y1,z1) and (x1′,y1′,z1′) represent the coordinate values ​​of the point cloud data in the previous frame and the coordinate values ​​after being projected onto the coordinate system of the current frame, respectively. This indicates the relative pose between the data of the previous frame and the current frame; The formula for calculating the residual image is as follows: Where r0 and r1′ represent the r-channel values ​​of the current frame image and the r-channel values ​​of the previous frame after alignment with the current coordinate system, respectively, and t1 and f2 are the set residual thresholds and noise filtering points.

5. The point cloud moving target segmentation method based on semantic information guidance according to claim 1, characterized in that, Set the maximum number of training iterations to epoch = 30.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method as described in any one of claims 1-5.

Images