4D point cloud moving target segmentation method and system

Abstract

This invention discloses a 4D point cloud moving target segmentation method and system. The method includes: acquiring a single-frame point cloud and voxelizing it; randomly using space-filling curves for serialization to obtain an initial sorting sequence; acquiring feature points of each voxel in multiple frames; inputting the initial sorting sequence into a U-Net network; extracting the sorting position features, intensity change features, and temporal features of each voxel in each frame; calculating perturbation scores frame by frame based on these features; performing local rearrangement of all voxels in all frames based on the perturbation scores to obtain a rearranged sequence; inputting the rearranged sequence into an encoder; processing it through hierarchical temporal downsampling and dual-channel Mamba modules; and then outputting the segmentation result after restoring the original resolution using a decoder; training the network with the sum of segmentation loss and topological continuity loss. During inference, the initial sorting sequence is used directly, skipping local rearrangement. This invention balances model accuracy and inference efficiency, effectively improving the performance of dynamic target segmentation.

Description

Technical Field

[0001] This invention relates to the fields of point cloud data processing and deep learning technology, and more specifically, to a 4D point cloud moving target segmentation method and system. Background Technology

[0002] LiDAR generates high-precision 3D point cloud data by emitting and receiving laser signals. This data can describe the detailed geometry of the surrounding environment. In scenarios such as autonomous driving or robot navigation, dynamic target segmentation is a crucial step in identifying and segmenting moving objects (such as vehicles and pedestrians) from point cloud data, which is of great significance for subsequent tasks such as obstacle avoidance decisions and path planning.

[0003] Traditional dynamic target segmentation methods typically rely on the point cloud geometry of moving targets, removing moving targets from all point cloud frames offline, or depend on pre-built complete point cloud maps. However, these methods are usually performed offline, requiring feature processing of all point cloud frame data throughout the entire autonomous driving process, making it difficult to meet real-time requirements.

[0004] In recent years, deep learning-based methods have been introduced into dynamic target segmentation tasks. Most methods directly concatenate the spatial coordinates and temporal information of each point to form a 4D point cloud containing spatiotemporal features, and then use convolutional neural networks to extract features. However, these methods generally suffer from insufficient spatiotemporal information coupling, making it difficult to fully model the joint changes of the target in the spatial and temporal dimensions, resulting in unsatisfactory dynamic target segmentation results.

[0005] The Mamba U-Net model, as a novel state-space model, exhibits advantages in global receptive field and linear complexity in sequence modeling tasks. When introducing Mamba into point cloud processing, it is necessary to arrange the disordered point cloud into an ordered one-dimensional sequence. Existing methods typically use space-filling curves (such as Hilbert curves or Z-order curves) for serialization to keep spatially adjacent points together. However, these methods have the following shortcomings: (1) During the training phase, frequent changes in serialization lead to unstable performance in the early stages of training; (2) Complex sorting schemes (such as local sorting based on KNN) significantly increase computational complexity; (3) Sorting operations are still required during the inference phase, introducing additional performance overhead.

[0006] Furthermore, in multi-frame point cloud processing, if voxel downsampling is used instead of projection, points from multiple frames will accumulate within a single voxel. Existing methods do not adequately consider inter-frame isolation during downsampling, making it difficult to effectively fuse spatiotemporal features. Summary of the Invention

[0007] This invention provides a 4D point cloud moving target segmentation method and system to solve the problems of strong serialization dependency, unstable training, large inference overhead, and insufficient spatiotemporal feature fusion in existing 4D point cloud moving target segmentation methods.

[0008] To achieve the above objectives, on the one hand, the present invention provides a 4D point cloud moving target segmentation method, which includes: S1, acquiring a single-frame point cloud, dividing the single-frame point cloud into multiple voxels according to a preset voxel size, and randomly using a space-filling curve to spatially sequence each voxel to obtain an initial sorting sequence for each voxel; S2, acquiring the point cloud information of each voxel over multiple frames, retaining the information of the point with the highest intensity value within each voxel in each frame, and obtaining the feature points of each voxel in each frame; S3, sorting the feature points of each voxel in each frame using the initial... Using the sorted sequence as a baseline, input it into the initial U-Net network. Extract the sorting position features, intensity change features, and temporal features of each voxel in each frame. Calculate the perturbation score of each voxel frame by frame based on the extracted features. Calculate the topological continuity loss based on the perturbation scores of each voxel and the initial sorted sequence. Perform local rearrangement of all voxels in all frames based on the perturbation scores to obtain the voxel rearrangement sequence for all frames. S4. Arrange the voxel feature points of each frame according to the voxel rearrangement sequence to obtain a multi-frame mixed sequence, and split it into a corresponding number of single voxels per frame. Frame sequence; after feature extraction of the multi-frame mixed sequence and each single-frame sequence by the dual-path Mamba module of the current layer of the encoder, they are fused to obtain the encoded fusion feature; the encoded fusion feature is downsampled in time and space and used as the input of the next layer of the encoder, and the above process is repeated layer by layer until the deepest layer of the encoder, and the encoded fusion feature of each layer is output; S5, the output of the deepest layer of the encoder is used as the initial input of the decoder, upsampled layer by layer and fused with the encoded fusion feature of the corresponding layer of the encoder during decoding, and restored to the original resolution layer by layer before outputting the moving target segmentation result through the classification head; the segmentation loss is calculated based on the moving target segmentation result output by the classification head and the ground truth; the sum of the segmentation loss and the topological continuity loss is used as the total loss of the current round, and the U-Net network parameters are updated based on the total loss of the current round; S6, repeat S1 to S5 until the total loss value of the current round fluctuates within a preset range, stop training, and obtain the target U-Net network; S7, during inference, the initial sorting sequence of each voxel is input into the target U-Net network frame by frame inverted order for prediction, and the moving target segmentation result is output.

[0009] Optionally, the step of inputting the feature points of each voxel in each frame into the initial U-Net network based on the initial sorting sequence, extracting the sorting position features, intensity change features, and temporal features of each voxel in each frame, and calculating the perturbation score of each voxel frame by frame based on the extracted features includes: inputting the feature points of each voxel in each frame into the initial U-Net network based on the initial sorting sequence, and obtaining the normalized sorting position φ of each voxel in each frame based on the initial sorting sequence. norm Based on the feature points of each voxel in each frame, calculate the intensity difference Δi between adjacent voxels and the normalized time Δt of each voxel in each frame; then, sort the normalized time by φ. norm After concatenating with the intensity difference Δi of adjacent voxels, the data is sequentially passed through a first one-dimensional convolutional layer, a batch normalization layer, a ReLU activation layer, a second one-dimensional convolutional layer, and a ReLU activation layer to obtain the spatial feature Fspat of each voxel in the current frame. The normalized time Δt is used as a one-dimensional input feature and sequentially passed through a first one-dimensional convolutional layer, a batch normalization layer, a ReLU activation layer, a second one-dimensional convolutional layer, and a ReLU activation layer to obtain the intermediate time feature of each voxel in the current frame. The intermediate time features of all voxels in the current frame are averaged and then passed through a multilayer perceptron and a sigmoid activation function to generate the weights of the current frame. The intermediate time features of each voxel in the current frame are multiplied by the weights of the current frame to obtain the temporal feature Ftemp of each voxel in the current frame. The spatial feature Fspat and the temporal feature Ftemp of each voxel in the current frame are concatenated and then input into a multilayer perceptron to output the perturbation score sj of each voxel in the current frame.

[0010] Optionally, the step of locally rearranging all voxels in all frames based on the perturbation score to obtain the voxel rearrangement sequence of all frames includes: using the initial sorting sequence of each voxel as a reference, inserting the feature points of each voxel in each frame into the initial sorting sequence in reverse chronological order to form a complete sequence in which voxels in all frames are continuously arranged according to their sequence positions; dividing the complete sequence into multiple windows using a sliding window strategy, with a window size of K and a step size of K / 2; calculating the true distance dj between adjacent voxels in three-dimensional space within each window; if the true distance dj exceeds a preset threshold τ, marking the current voxel position as a spatial mutation boundary, and truncating the window at the first mutation position in advance to obtain an effective window; re-sorting the voxels according to the perturbation score sj from smallest to largest within each effective window; if the perturbation score of a voxel in the current window is less than the minimum value of the perturbation scores of all voxels in the previous window, removing the voxel from the current window and inserting it into the corresponding sorting position in the previous window according to the perturbation score, and outputting the voxel rearrangement sequence of all frames.

[0011] Optionally, S4 includes: arranging the voxel feature points of each frame according to the voxel rearrangement sequence to obtain a multi-frame mixed sequence; splitting it into a corresponding number of single-frame sequences according to the initial time dimension frame number; feeding the multi-frame mixed sequence and each single-frame sequence into the temporal path Mamba module and spatial path Mamba module of the layer 0 encoder respectively for feature extraction and fusion to obtain the layer 0 coding fusion feature; using the previous layer coding fusion feature as the current layer's multi-frame mixed sequence; performing spatiotemporal downsampling on the multi-frame mixed sequence; splitting it into a corresponding number of single-frame sequences according to the updated time dimension frame number after downsampling; feeding the multi-frame mixed sequence and each single-frame sequence into the temporal path Mamba module and spatial path Mamba module of the current layer encoder respectively for feature extraction and fusion to obtain the current layer coding fusion feature; repeating the above process layer by layer until the time dimension frame number is downsampled to 1, at which point the single-frame sequence and multi-frame mixed sequence are consistent; directly feeding the multi-frame mixed sequence into the encoder to output the deepest layer coding fusion feature.

[0012] Optionally, the step of using the deepest encoder output as the initial input to the decoder, upsampling layer by layer and fusing it with the encoded fusion features of the corresponding encoder layer during decoding, and restoring the original resolution layer by layer before outputting the moving target segmentation result through the classification head includes: using the encoded fusion features of the deepest encoder output as the upsampling input to the deepest decoder, using the encoded fusion features of the second deepest encoder output as the skip connection features, and performing skip connection fusion of the upsampled features with the skip connection features to obtain the deepest decoded fusion features; using the features upsampled from the previous layer's decoded fusion features as the upsampling input to the current layer's decoder, using the encoded fusion features of the corresponding encoder layer as the skip connection features, and performing skip connection fusion of the upsampled features with the skip connection features to obtain the deepest decoded fusion features; using the features upsampled from the previous layer's decoded fusion features as the upsampling input to the current layer's decoder, and using the encoded fusion features of the corresponding encoder layer as the skip connection features, and performing skip connection fusion of the upsampled features with the skip connection features... The features are fused with the skip-connected features to obtain the intermediate decoding features of the current layer; the intermediate decoding features of the current layer are used as a multi-frame mixed sequence, and split into a corresponding number of single-frame sequences according to the current time dimension frame number. The multi-frame mixed sequence and the multiple single-frame sequences are respectively sent to the dual-channel Mamba module of the current layer decoder for feature extraction and fusion to obtain the current layer decoding fusion features; the above process is repeated layer by layer until the original resolution is restored, and the part of the shallowest layer decoding fusion features corresponding to the current frame is taken as the voxel features of each voxel; each point is mapped to its corresponding voxel, and the corresponding voxel features are used as the features of that point. The moving target segmentation result is output through the linear classification head.

[0013] On the other hand, the present invention provides a 4D point cloud moving target segmentation system, which includes: a serialization unit, used to acquire a single-frame point cloud, divide the single-frame point cloud into multiple voxels according to a preset voxel size, and randomly use a space filling curve to spatially serialize each voxel to obtain an initial sorting sequence for each voxel; an acquisition unit, used to acquire the point cloud information of each voxel over multiple frames, retain the information of the point with the highest intensity value in each voxel frame by frame, and obtain the feature points of each voxel in each frame; and a sequence rearrangement unit, used to rearrange the feature points of each voxel in each frame according to the initial sorting sequence. Based on the initial U-Net network, the ranking position features, intensity change features, and temporal features of each voxel in each frame are extracted frame by frame. The perturbation score of each voxel is calculated frame by frame based on the extracted features. The topological continuity loss is calculated based on the perturbation scores and the initial ranking sequence. Local rearrangement of all voxels in all frames is performed based on the perturbation scores to obtain the voxel rearrangement sequence for all frames. An encoding unit is used to arrange the voxel feature points of each frame according to the voxel rearrangement sequence to obtain a multi-frame mixed sequence, and then split it into a corresponding number of single-frame sequences. The multi-frame mixed sequence and... Each single-frame sequence is fused after feature extraction by the dual-path Mamba module of the current layer of the encoder to obtain coded fusion features. These coded fusion features are then spatiotemporally downsampled and used as input to the next layer of the encoder. This process is repeated layer by layer until the deepest layer of the encoder is reached, outputting the coded fusion features of each layer. The decoding unit uses the output of the deepest layer of the encoder as the initial input to the decoder, upsamples layer by layer, and fuses the coded fusion features with the corresponding layer of the encoder during decoding. After restoring the original resolution layer by layer, the moving target segmentation result is output through the classification head. The segmentation loss is calculated based on the moving target segmentation result output by the classification head and the ground truth. The sum of the segmentation loss and the topological continuity loss is used as the total loss for the current round, and the U-Net network parameters are updated based on the total loss for the current round. The repeated training unit repeats the serialization unit, acquisition unit, sequence rearrangement unit, encoding unit, and decoding unit until the total loss value for the current round fluctuates within a preset range, at which point training stops, and the target U-Net network is obtained. The prediction unit, during inference, inputs the initial sorted sequence of each voxel in reverse frame order into the target U-Net network for prediction and outputs the moving target segmentation result.

[0014] Optionally, the sequence rearrangement unit includes: an acquisition subunit, used to input the feature points of each voxel in each frame into the initial U-Net network based on the initial sorting sequence, and to acquire the normalized sort position φ of each voxel in each frame according to the initial sorting sequence. norm The unit calculates the intensity difference Δi between adjacent voxels and the normalized time Δt for each voxel in each frame based on the feature points of each voxel in each frame; the spatial feature extraction subunit is used to sort the normalized time φ. normAfter concatenating with the intensity difference Δi of adjacent voxels, the data is sequentially passed through a first one-dimensional convolutional layer, a batch normalization layer, a ReLU activation layer, a second one-dimensional convolutional layer, and a ReLU activation layer to obtain the spatial feature Fspat of each voxel in the current frame. An intermediate temporal feature extraction subunit is used to take the normalized time Δt as a one-dimensional input feature, sequentially passing it through a first one-dimensional convolutional layer, a batch normalization layer, a ReLU activation layer, a second one-dimensional convolutional layer, and a ReLU activation layer to obtain the intermediate temporal feature of each voxel in the current frame. A temporal feature calculation subunit is used to average the intermediate temporal features of all voxels in the current frame, generate the weights for the current frame through a multilayer perceptron and a sigmoid activation function, and multiply the intermediate temporal features of each voxel in the current frame by the weights of the current frame to obtain the temporal feature Ftemp of each voxel in the current frame. A score calculation subunit is used to concatenate the spatial feature Fspat and the temporal feature Ftemp of each voxel in the current frame and input them into a multilayer perceptron to output the perturbation score sj of each voxel in the current frame.

[0015] Optionally, the sequence rearrangement unit further includes: a window partitioning subunit, used to insert the feature points of each voxel in each frame into the initial sorting sequence in reverse chronological order based on the initial sorting sequence of each voxel, forming a complete sequence in which voxels in all frames are arranged continuously according to their sequence positions; the complete sequence is divided into multiple windows using a sliding window strategy, with a window size of K and a step size of K / 2; a mutation detection subunit, used to calculate the true distance dj between adjacent voxels in three-dimensional space within each window, and if the true distance dj exceeds a preset threshold τ, the current voxel position is marked as a spatial mutation boundary, and the window is truncated in advance at the first mutation position to obtain an effective window; a window rearrangement subunit, used to reorder the voxels in each effective window according to the perturbation score sj from smallest to largest; if the perturbation score of a voxel in the current window is less than the minimum value of the perturbation scores of all voxels in the previous window, the voxel is removed from the current window and inserted into the corresponding sorting position in the previous window according to the perturbation score, and the voxel rearrangement sequence of all frames is output.

[0016] Optionally, the encoding unit includes: arranging voxel feature points of each frame according to the voxel rearrangement sequence to obtain a multi-frame mixed sequence; splitting it into a corresponding number of single-frame sequences according to the initial time dimension frame number; feeding the multi-frame mixed sequence and each single-frame sequence into the temporal path Mamba module and spatial path Mamba module of the layer 0 encoder respectively for feature extraction and fusion to obtain the layer 0 encoding fusion feature; using the previous layer encoding fusion feature as the current layer multi-frame mixed sequence; performing spatiotemporal downsampling on the multi-frame mixed sequence; splitting it into a corresponding number of single-frame sequences according to the updated time dimension frame number after downsampling; feeding the multi-frame mixed sequence and each single-frame sequence into the temporal path Mamba module and spatial path Mamba module of the current layer encoder respectively for feature extraction and fusion to obtain the current layer encoding fusion feature; repeating the above process layer by layer until the time dimension frame number is downsampled to 1, at which point the single-frame sequence and multi-frame mixed sequence are consistent; directly feeding the multi-frame mixed sequence into the encoder and outputting the deepest layer encoding fusion feature.

[0017] Optionally, the decoding unit includes: using the coded fusion feature output from the deepest layer of the encoder as the upsampling input to the deepest layer of the decoder, using the coded fusion feature output from the second deepest layer of the encoder as the skip connection feature, and performing skip connection fusion of the upsampling feature and the skip connection feature to obtain the deepest layer decoding fusion feature; using the feature upsampled from the decoding fusion feature of the previous layer as the upsampling input to the current layer decoder, using the coded fusion feature output from the corresponding layer of the encoder as the skip connection feature, and performing skip connection fusion of the upsampling feature and the skip connection feature to obtain the current layer intermediate decoding feature; using the current layer... The intermediate decoding features, as a multi-frame hybrid sequence, are split into a corresponding number of single-frame sequences according to the current time dimension frame number. The multi-frame hybrid sequence and the multiple single-frame sequences are respectively fed into the dual-path Mamba module of the current layer decoder for feature extraction and fusion to obtain the current layer decoding fusion features. The above process is repeated layer by layer upsampling until the original resolution is restored. The part of the shallowest layer decoding fusion features corresponding to the current frame is taken as the voxel features of each voxel. Each point is mapped to its corresponding voxel, and the corresponding voxel features are used as the features of that point. The moving target segmentation result is output through the linear classification head.

[0018] The beneficial effects of this invention are:

[0019] This invention provides a 4D point cloud moving target segmentation method and system. The invention applies sequence rearrangement as a regularization technique during the training phase. A lightweight MLP generates perturbation scores for each voxel based on spatiotemporal features, performing local rearrangement within a sliding window. Combined with a strategy where each layer of the encoder and decoder independently and randomly selects space-filling curves during training, the model can adapt to various sequence arrangements, significantly reducing dependence on a single sequenceization scheme. Furthermore, topological continuity loss constrains the smooth change of perturbation scores of adjacent voxels, preventing sequence variation degradation. Simultaneously, through a decoupling design between training and inference, the initial sorted sequence is directly used during inference, skipping local rearrangements, thus avoiding performance overhead during inference and balancing model accuracy and inference efficiency. Furthermore, this invention employs a spatiotemporally independent downsampling hierarchical encoder-decoder architecture. In each layer, the multi-frame mixed sequence is split into single-frame sequences according to the time dimension. These sequences are then processed by the temporal and spatial Mamba modules and fused together, achieving deep coupling of spatiotemporal features and effectively improving the ability to distinguish between dynamic targets and static backgrounds. During the encoding stage, gating weights are generated using temporal features as a reference to guide the injection of spatial features, making the model more focused on maintaining the shape of moving targets. During the decoding stage, deep semantic features are used to suppress noise from shallow skip-connected features, further improving segmentation accuracy. Attached Figure Description

[0020] Figure 1 This is a flowchart of a 4D point cloud moving target segmentation method provided in an embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram of the structure of a 4D point cloud moving target segmentation system provided in an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0023] This embodiment uses dynamic target segmentation of LiDAR point clouds in an autonomous driving scenario as an example to provide a detailed description of the method of the present invention. Figure 1 This is a flowchart of a 4D point cloud moving target segmentation method provided in an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps.

[0024] S1. Obtain a single-frame point cloud, divide the single-frame point cloud into multiple voxels according to a preset voxel size, and randomly use a space filling curve to spatially serialize each voxel to obtain the initial sorting sequence of each voxel.

[0025] Point cloud data acquired by lidar is an unordered, sparse set of three-dimensional points, each containing spatial coordinates (x, y, z) and intensity information i. To process this data using the sequence-based Mamba U-Net model, the unordered point cloud needs to be transformed into an ordered sequence. Voxelization is an effective method for structuring point clouds, compressing the point cloud data into a voxel-level representation by dividing the space into a regular grid. Space-filling curves can map points in multi-dimensional space to one-dimensional space while preserving spatial locality, meaning that adjacent voxels in space are also as adjacent as possible in the sequence.

[0026] In this invention, when using space-filling curves for serialization, various curve types can be employed to ensure sequence diversity. Optional curves include Hilbert curves, Z-order curves, and Hilbert-trans or Z-order-trans curves with swapped x and y axis priorities.

[0027] (1) Hilbert curve: a standard three-dimensional Hilbert curve that treats all dimensions equally;

[0028] (2) Z-order curve: generated by interleaving the binary bits of each dimension coordinate;

[0029] (3) Hilbert-trans: The input order of the x and y coordinates is swapped before calculating the Hilbert code, so that the y axis has higher priority;

[0030] (4) Z-order transpose curve (Z-trans): The input order of x and y coordinates is swapped before calculating Z-order encoding, so that the y axis has higher priority.

[0031] During the model training phase, in order to ensure that the input sequence maintains both spatial local order and rich sequence arrangement possibilities, thereby enhancing the model's robustness to sequence order, this invention adopts the following strategy:

[0032] (1) Interlayer independent randomization: In the encoder and decoder of the U-Net network, each layer (i.e., each resolution level) independently selects one of a variety of space-filling curves to serialize each voxel when processing the input. That is, multiple space-filling curves (such as 4) are selected, and the sequence results generated by each encoder and decoder using different curves are different between layers.

[0033] (2) Slight changes between rounds: The four curves are rearranged one by one in each round (i.e., step 3 below) to obtain four new sequences. The serialization result changes slightly in each round. This avoids the model from relying on a single serialization strategy and decouples causality, while preventing the model from fluctuating greatly in the early learning results due to large changes in the serialization result.

[0034] (3) Handling extreme cases: When the number of frames in the time dimension of a certain layer of the network becomes 1 after downsampling, there is only one single frame sequence in that layer. At this time, there is no need for random selection between layers. The single frame sequence can be used directly or the serialization method of the previous layer can be used.

[0035] Taking the LiDAR on an autonomous vehicle as an example, the LiDAR collects point cloud data at a frequency of 10Hz. A single frame of the point cloud is taken at the current moment, covering a spatial area of ​​100m × 80m × 20m in front. The voxel size is set to 0.2m, and the entire point cloud space is divided into several cubic grids with sides of 0.2m.

[0036] For each voxel, calculate its center coordinates. The formula for calculating the voxel center coordinates is:

[0037] (x voxel ,y voxel ,z voxel = (i×0.2+0.1,j×0.2+0.1,k×0.2+0.1)

[0038] Where (i,j,k) are the index numbers of the voxels in the three dimensions.

[0039] This embodiment uses a small-scale scenario for illustration. Assume that after voxelization, four non-empty voxels are obtained within the region of interest, labeled A, B, C, and D. The center coordinates of each voxel are as follows:

[0040] Voxel A: (1.1m, 0.1m, 0.1m)

[0041] Voxel B: (2.1m, 0.1m, 0.1m)

[0042] Voxel C: (3.1m, 0.1m, 0.1m)

[0043] Voxel D: (4.1m, 0.1m, 0.1m)

[0044] In this training round, assume that the encoder randomly selected a Hilbert curve in layer 0 to sort the four voxels. Calculate the Hilbert code for each voxel, assuming the result is: hilbert(A) = 5.

[0045] hilbert(B) = 2,

[0046] hilbert(C) = 7,

[0047] hilbert(D) = 9. Sort by encoding from smallest to largest, the initial sorting sequence for this layer is: [B, C, A, D]; that is, in the initial sorting sequence, B is in position 0, C is in position 1, A is in position 2, and D is in position 3. The sorting positions of each voxel are: rank(B)=0, rank(C)=1, rank(A)=2, rank(D)=3, and the total number of primes N=4.

[0048] The initial sorting sequence reflects the relative spatial positions of the four voxels: B is at the front, C is to the right of B, A is behind C, and D is at the farthest point.

[0049] The encoder selects the first layer independently and randomly. Assuming that the Z-order curve is selected in this round, the Z-order encoding result is calculated as: Z(A)=3, Z(B)=1, Z(C)=4, Z(D)=2. Then the sorting sequence used in the first layer is [B, D, A, C].

[0050] In the next round of training, the sequences generated by the four curves are rearranged to obtain four rearranged sequences. The sequences used in the same layer in each round are similar to but different from those in the previous round, while different layers remain independent and diverse.

[0051] Through the aforementioned inter-layer independent randomization and inter-round sequence micro-variation, the model can adapt to various smoothly changing sequence arrangements during training, significantly reducing its dependence on a single serialization scheme. At the same time, stable segmentation performance can be obtained by using any serialization method (such as Hilbert curve sorting results) during inference.

[0052] After step S1 is completed, the initial sorting sequence of voxels corresponding to each layer is obtained. This sequence serves as the reference benchmark for the original sorting of the layer during training and is directly used as the input sequence by reversing the order of frames during the inference phase.

[0053] S2. Obtain the point cloud information of each voxel in multiple frames, and retain the information of the point with the largest intensity value in each voxel in each frame to obtain the feature points of each voxel in each frame.

[0054] The core of 4D point cloud moving target segmentation lies in using temporal information to determine the target's motion state. The same voxel may contain different point cloud data in different frames. By comparing the feature changes of the same voxel at different times, it can be determined whether a moving target exists at the corresponding spatial location. Therefore, it is necessary to extract the feature points of each voxel in the temporal dimension.

[0055] Preferably, the lidar continuously acquires point cloud data at a frequency of 10Hz. Eight frames of point cloud data are collected (t=0,1,2,3,4,5,6,7), with an inter-frame interval of 100ms, resulting in a total time span of 700ms. These eight frames constitute a time window for analyzing the target's motion state.

[0056] For each voxel, the point cloud data of each frame is processed separately. Specifically, for a voxel VOX_n, in frame t, multiple points may fall within this voxel. Within each voxel, only the information of the point with the highest intensity value is retained, represented as (x... k ,y k , z k i k , t k ), where x k , y k , z k Let i be the three-dimensional coordinates of the point. k The intensity value at that point, t k This is the frame number where the point is located. In this way, each voxel retains at most one feature point per frame, which greatly reduces the amount of data while retaining the most representative information.

[0057] After step S2 is completed, the feature points of each voxel in each frame are obtained, providing a data basis for subsequent feature extraction and perturbation score calculation.

[0058] In one optional implementation, point cloud data of the four voxels were acquired over eight frames. It is assumed that each voxel has an intensity value in each frame, as shown in Table 1.

[0059] Table 1. Intensity values ​​of each voxel in each frame.

[0060]

[0061] In Table 1, each row represents the intensity change of a voxel over 8 frames, and each column represents the intensity distribution of 4 voxels in a certain frame.

[0062] As can be seen from Table 1:

[0063] The intensity of voxel A is approximately 0.155 to 0.183 at t=0 to 2, increases sharply to approximately 0.408 to 0.437 at t=3 to 5, and decreases to approximately 0.000 to 0.014 at t=6 to 7, exhibiting a pulse-like change of first increasing and then decreasing.

[0064] The intensity of voxel B gradually increased from 0.099 to 0.155 between t=0 and 4, and then surged to about 0.662 to 0.718 between t=5 and 7, showing a late-stage surge pattern.

[0065] The intensity of voxel C remained relatively stable across the 8 frames, ranging from 0.197 to 0.296 with minimal fluctuations, indicating that the spatial position corresponding to this voxel was not significantly affected by the moving target.

[0066] The intensity of voxel D gradually increased from 0.042 to 0.113 from t=0 to 5, and then surged to about 0.986 to 1.000 from t=6 to 7, showing a significant late-stage jump.

[0067] These intensity variation patterns reflect the presence of moving targets at different voxel locations. For example, voxel B's intensity jumps sharply from 0.155 to 0.718 at t=5, indicating that a moving target entered the spatial location corresponding to voxel B at that moment; voxel D's intensity jumps to near its maximum value of 1.000 at t=6, and voxel A's normalized intensity forms a peak above 0.4 before rapidly declining at t=3~4, both potentially corresponding to the passage of moving targets. In contrast, voxel C's intensity curve remains consistently stable, suggesting that its corresponding spatial location may be a static background.

[0068] After step S2 is completed, the feature points of each voxel in each frame (i.e., the intensity value of each voxel in each frame and its corresponding spatial coordinates and frame number) are obtained, which provides a data foundation for subsequent feature extraction.

[0069] S3. Using the initial sorting sequence as a reference, input the feature points of each voxel in each frame into the initial U-Net network, and extract the sorting position features, intensity change features, and temporal features of each voxel in each frame. Calculate the perturbation score of each voxel frame by frame based on the extracted features, and calculate the topological continuity loss based on the perturbation score of each voxel and the initial sorting sequence. Perform local rearrangement of all voxels in all frames based on the perturbation score to obtain the voxel rearrangement sequence for all frames.

[0070] This step is the core process of transforming the raw point cloud data into a feature representation suitable for moving target segmentation and performing sequence perturbation. First, using the initial sorted sequence obtained in step S1 as a spatial reference, the feature points of each voxel in each frame are organized into a regular sequence structure and input into the U-Net network to extract the sorting position features, intensity change features, and temporal features of each voxel in each frame. The sorting position features reflect the position of the voxel in the initial sequence, the intensity change features reflect the intensity difference between adjacent voxels, and the temporal features reflect the relative position of the current frame within the time window. Then, perturbation scores are calculated based on the three extracted features. Diverse sequence arrangements are generated through local rearrangement, allowing the model to adapt to different sequence orders during training. Topological continuity loss is used to constrain the smoothness of the perturbation scores, preventing training instability.

[0071] In an optional implementation, the step of inputting the feature points of each voxel in each frame into the initial U-Net network based on the initial sorting sequence, extracting the sorting position features, intensity change features, and temporal features of each voxel in each frame, and calculating the perturbation score of each voxel frame by frame based on the extracted features includes:

[0072] S31. Input the feature points of each voxel in each frame into the initial U-Net network based on the initial sorting sequence, and obtain the normalized sort position φ of each voxel in each frame according to the initial sorting sequence. norm The intensity difference Δi between adjacent voxels and the normalized time Δt of each voxel in each frame are calculated based on the feature points of each voxel in each frame.

[0073] (1) Sort position feature acquisition: Obtain the normalized sort position φ of each voxel in the current frame according to the initial sort sequence. norm , φ norm (j)=rank(j) / N

[0074] Where rank(j) represents the position of the voxel in the initial sorting sequence (starting from 0), N is the total number of voxels, and the normalized sorting position is φ. norm The value range is [0,1], and the smaller the value, the earlier the voxel is in the sequence.

[0075] (2) Intensity change feature calculation: Calculate the intensity difference Δi between adjacent voxels based on the feature points of each voxel in the current frame. .

[0076] Where i j This represents the intensity value of the current voxel in the current frame. For the first voxel in the sequence, Δi0 = 0. The sign and magnitude of Δi reflect the trend and magnitude of the intensity change.

[0077] (3) Calculation of time characteristics: Calculate the normalized time Δt based on the current frame number.

[0078]

[0079] Among them, t min =0,t max =7.

[0080] S32, adjust the normalized sorting position φ norm After being concatenated with the intensity difference Δi of the adjacent voxels, the spatial features Fspat of each voxel in the current frame are obtained by sequentially passing through the first one-dimensional convolutional layer, batch normalization layer, ReLU activation layer, second one-dimensional convolutional layer and ReLU activation layer.

[0081] The specific formula is as follows:

[0082] Fspat=ReLU(Con v3 (ReLU(BN(Conv Ks ([φ norm ,Δi])))))

[0083] Where Fspat is the spatial feature, ReLU is the ReLU activation layer, and Con... v3 For the second one-dimensional convolutional layer, Conv Kt This is the first one-dimensional convolutional layer, BN is the batch normalization layer, and φ norm Δi represents the normalized sort position, and Δi represents the intensity difference between adjacent voxels.

[0084] S33. The normalized time Δt is used as a one-dimensional input feature and passed through the first one-dimensional convolutional layer, batch normalization layer, ReLU activation layer, second one-dimensional convolutional layer and ReLU activation layer in sequence to obtain the intermediate time features of each voxel in the current frame.

[0085] The specific formula is as follows:

[0086] Ftemp′=ReLU(Con v3 (ReLU(BN(Conv Kt (Δt)))))

[0087] Where Ftemp′ is the intermediate time feature, ReLU is the ReLU activation layer, and Con v3 For the second one-dimensional convolutional layer, Conv Kt is the first one-dimensional convolutional layer, BN is the batch normalization layer, and Δt is the normalization time.

[0088] S34. Take the average of the intermediate time features of all voxels in the current frame, generate the weight of the current frame through a multilayer perceptron and a sigmoid activation function, and multiply the intermediate time features of each voxel in the current frame by the weight of the current frame to obtain the time features Ftemp of each voxel in the current frame.

[0089] The specific formula is as follows:

[0090]

[0091] Ftemp=Ftemp′⊙αt

[0092] Where αt is the weight of the current frame, mean(Ftemp′) is the average of the intermediate time features of all voxels in the current frame, σ is the sigmoid activation function, W1 is the weight matrix of the first linear transformation, W2 is the weight vector of the second linear transformation, b1 is the bias of the first linear transformation, and b2 is the bias of the second linear transformation.

[0093] S35. After splicing the spatial features Fspat of each voxel in the current frame and the temporal features Ftemp of each voxel in the current frame, input the splice into a multilayer perceptron and output the perturbation score sj of each voxel in the current frame.

[0094] By concatenating the spatial features Fspat and temporal features Ftemp of each voxel in the current frame, a complete spatiotemporal feature vector x is obtained. j :

[0095] x j =[Fjspat,Fjtemp]

[0096] x j Input a multilayer perceptron and output the perturbation score sj for each voxel in the current frame:

[0097]

[0098] The perturbation score sj reflects the tendency of voxels to move during rearrangement: the lower the score, the further forward the voxel should be rearranged; the higher the score, the further backward the voxel should be rearranged.

[0099] In an optional implementation, the step of locally rearranging all voxels of all frames based on the perturbation score to obtain the voxel rearrangement sequence of all frames includes:

[0100] S36. Based on the initial sorting sequence of each voxel, insert the feature points of each voxel in each frame into the initial sorting sequence in reverse chronological order to form a complete sequence in which the voxels of all frames are arranged continuously according to their sequence positions; divide the complete sequence into multiple windows using a sliding window strategy, with a window size of K and a step size of K / 2.

[0101] Specifically, firstly, using the initial sorting sequence of each voxel as a reference, the feature points of each voxel in each frame are inserted into the initial sorting sequence in reverse temporal order, forming a complete sequence in which voxels are arranged continuously according to their sequence positions across all frames. For example, for 4 voxels and 8 frames of data, the complete sequence is:

[0102] [B7,B6,B5,B4,B3,B2,B1,B0,C7,C6,...,C0,A7,...,A0,D7,...,D0]

[0103] Then, the complete sequence is divided into multiple windows using a sliding window strategy. The window size is K, and the step size is K / 2. The window division method is as follows:

[0104]

[0105] S37. Within each window, calculate the true distance dj between adjacent voxels in three-dimensional space. If the true distance dj exceeds a preset threshold τ, mark the current voxel position as a spatial mutation boundary and truncate the window in advance at the first mutation position to obtain an effective window.

[0106] Within each window, calculate the true distance dj between adjacent voxels in 3D space:

[0107]

[0108] If dj exceeds the preset threshold τ, the current voxel position is marked as a spatial mutation boundary, and the window is truncated in advance at the first mutation position.

[0109] S38. Within each valid window, reorder the voxels according to the perturbation score sj from smallest to largest; if the perturbation score of a voxel in the current window is less than the minimum value of the perturbation scores of all voxels in the previous window, remove the voxel from the current window and insert it into the corresponding sorting position in the previous window according to the perturbation score, and output the voxel rearrangement sequence of all frames.

[0110] S3 further includes:

[0111] S39. Calculate the topological continuity loss based on the perturbation scores of each voxel and the initial sorting sequence:

[0112]

[0113] in π(i) is the weighting coefficient, and π(i) is the voxel adjacent to voxel i in the initial sorting sequence.

[0114] In a preferred embodiment, the feature points of each voxel in each frame are arranged in the order of the initial sorting sequence [B, C, A, D] based on the initial sorting sequence [B, C, A, D]. For frame 0, the voxel sequence is [B0, C0, A0, D0].

[0115] Calculate the sorting position features:

[0116] The normalized ordinal positions of each voxel are:

[0117] φ norm (B) = 0 / 4 = 0

[0118] φ norm (C) = 1 / 4 = 0.25

[0119] φ norm (A) = 2 / 4 = 0.5

[0120] φ norm (D) = 3 / 4 = 0.75

[0121] Calculate the intensity variation characteristics:

[0122] Based on the intensity values ​​of frame 0 in Table 1, the intensity values ​​of each voxel are as follows:

[0123] i(B) = 0.099, i(C) = 0.211, i(A) = 0.155, i(D) = 0.042

[0124] The intensity difference between adjacent voxels is (Δi of the first voxel B is set to 0):

[0125] Δi(B) = 0

[0126] Δi(C) = 0.211 - 0.099 = 0.112

[0127] Δi(A) = 0.155 - 0.211 = -0.056

[0128] Δi(D) = 0.042 - 0.155 = -0.113

[0129] Calculation time characteristics:

[0130] Normalized time (frame 0): Δt = 0 / 7 = 0

[0131] Calculate the ranking position-intensity fusion feature and weighted time feature:

[0132] [φ norm After concatenating the input sorting position-intensity feature branch, assuming the number of hidden channels H=4, after one-dimensional convolution and activation, the sorting position-intensity fusion feature Fspat of each voxel is obtained (for ease of explanation, the following values ​​are illustrative):

[0133] Fspat(B) = [0.3, 0.1, 0.5, 0.2]

[0134] Fspat(C) = [0.7, 0.4, 0.6, 0.3]

[0135] Fspat(A) = [0.5, 0.3, 0.4, 0.6]

[0136] Fspat(D) = [0.8, 0.6, 0.3, 0.5]

[0137] Inputting Δt into the temporal feature branch, after one-dimensional convolution and activation, yields the intermediate temporal feature Ftemp'. Since Δt is 0 for all voxels in frame 0, F_temp' is identical for all voxels.

[0138] Ftemp'(B) = Ftemp'(C) = Ftemp'(A) = Ftemp'(D) = [0.2, 0.3, 0.1, 0.4]

[0139] The average Ftemp' of all voxels in frame 0 remains [0.2, 0.3, 0.1, 0.4]. The frame weight α_0 = 0.6 is generated by MLP and sigmoid.

[0140] The weighted time characteristics of each voxel are Ftemp = Ftemp' × 0.6:

[0141] Ftemp(B) = Ftemp(C) = Ftemp(A) = Ftemp(D) = [0.12, 0.18, 0.06, 0.24]

[0142] Generate perturbation scores:

[0143] Concatenation of position-intensity fusion features and weighted temporal features, taking voxel B as an example:

[0144] x B =[0.3,0.1,0.5,0.2,0.12,0.18,0.06,0.24]

[0145] After passing through the multilayer perceptron, the perturbation fractions (indicative values) of each voxel are output:

[0146] s(B) = 0.3

[0147] s(C) = 0.7

[0148] s(A) = 0.5

[0149] s(D) = 0.9

[0150] Similarly, calculate the perturbation score for each voxel in the remaining 7 frames.

[0151] Table 2. Perturbation scores of each voxel in each frame.

[0152]

[0153] Construct the complete sequence:

[0154] Using the initial sorted sequence [B, C, A, D] as a reference, the feature points of each voxel in each frame are inserted in reverse temporal order to form a complete sequence:

[0155] [B7(s=0.5),B6(s=0.6),B5(s=0.2),B4(s=0.3),B3(s=0.5),B2(s=0.4),B1(s=0.3),B0(s=0.3)]

[0156] [C7(s=0.8),C6(s=0.9),C5(s=0.6),C4(s=0.8),C3(s=0.7),C2(s=0.5),C1(s=0.6),C0(s=0.7)]

[0157] [A7(s=0.4),A6(s=0.2),A5(s=0.3),A4(s=0.7),A3(s=0.8),A2(s=0.6),A1(s=0.4),A0(s=0.5)]

[0158] [D7(s=0.4),D6(s=0.3),D5(s=0.8),D4(s=0.6),D3(s=0.9),D2(s=0.7),D1(s=0.8),D0(s=0.9)]

[0159] Sliding window partitioning and partial rearrangement:

[0160] Let the window size K = 6, and the step size K / 2 = 3. The window is divided as follows:

[0161] Window 0: Positions 0-5, including [B7, B6, B5, B4, B3, B2]

[0162] Window 1: Positions 3-8, including [B4, B3, B2, B1, B0, C7]

[0163] Window 2: Positions 6-11, including [B1, B0, C7, C6, C5, C4]

[0164] Window 3: Positions 9-14, including [C5, C4, C3, C2, C1, C0]

[0165] Window 4: Positions 12-17, containing [C1, C0, A7, A6, A5, A4]

[0166] Window 5: Positions 15-20, containing [A5, A4, A3, A2, A1, A0]

[0167] Window 6: Positions 18-23, containing [A1, A0, D7, D6, D5, D4]

[0168] Window 7: Positions 21-26, including [D5, D4, D3, D2, D1, D0]

[0169] Taking window 0 as an example (positions 0~5, including B7, B6, B5, B4, B3, B2), we check for spatial abrupt changes: adjacent voxels are all B, have the same coordinates, a distance d=0, do not exceed the threshold τ=0.5, have no abrupt changes, and the window is intact. We sort by perturbation scores from smallest to largest: B5 (s=0.2), B1 (s=0.3), B4 (s=0.3), B2 (s=0.4), B7 (s=0.5), B3 (s=0.5). After rearrangement, window 0 becomes [B5, B1, B4, B2, B7, B3].

[0170] Taking window 1 as an example (positions 3-8, including B4, B3, B2, B1, B0, C7), we check for spatial mutations: the spatial coordinates of B and C are (2.1, 0.1, 0.1) and (3.1, 0.1, 0.1) respectively, and the distance d = 1.0 > τ = 0.5, triggering a spatial mutation between B_0 and C_7. The window is truncated at the mutation position, and the effective window only contains positions 3-7: [B_4, B_3, B_2, B_1, B_0]. After sorting by perturbation score from smallest to largest, it is [B1(s=0.3), B4(s=0.3), B2(s=0.4), B0(s=0.3), B3(s=0.5)].

[0171] The same applies to other windows. Cross-window movement: Compare the minimum score s=0.3 after rearranging window 1 with the minimum score s=0.2 of the previous window (window 0). If 0.3 > 0.2, there is no cross-window movement.

[0172] Finally, the voxel rearrangement sequence of all frames is obtained.

[0173] Calculation of topological continuity loss:

[0174] Taking voxel A as an example, the initial sorting sequence is [B, C, A, D], and the adjacent voxels of A are C and D. The perturbation score of each voxel is taken as its mean over all frames (or the value of the last frame, depending on the design), assumed to be: s(A)=0.5, s(C)=0.7, s(D)=0.8.

[0175]

[0176] Sum of all voxels and multiply by The topological continuity loss value for this round is obtained.

[0177] After step S3 is completed, the voxel rearrangement sequence and topological continuity loss value of all frames are obtained, providing ordered input data for subsequent encoder processing.

[0178] S4. Arrange the voxel feature points of each frame according to the voxel rearrangement sequence to obtain a multi-frame mixed sequence, and split it into a corresponding number of single-frame sequences according to the frame; after feature extraction by the multi-frame mixed sequence and each single-frame sequence through the dual-channel Mamba module of the current layer of the encoder, fuse them to obtain the coded fusion feature; after spatiotemporal downsampling of the coded fusion feature, use it as the input of the next layer encoder, and repeat the above process layer by layer until the deepest layer of the encoder, and output the coded fusion features of each layer;

[0179] In an optional implementation, S4 includes:

[0180] The voxel feature points of each frame are arranged according to the voxel rearrangement sequence to obtain a multi-frame mixed sequence. The sequence is then split into a corresponding number of single-frame sequences according to the initial time dimension frame number. The multi-frame mixed sequence and each single-frame sequence are respectively fed into the temporal path Mamba module and spatial path Mamba module of the layer 0 encoder for feature extraction and fusion to obtain the layer 0 encoded fusion feature.

[0181] Using the previous layer's coding fusion feature as the current layer's multi-frame hybrid sequence, the multi-frame hybrid sequence is spatiotemporally downsampled and split into a corresponding number of single-frame sequences according to the updated time dimension frame number after downsampling. The multi-frame hybrid sequence and each single-frame sequence are respectively sent to the temporal path Mamba module and spatial path Mamba module of the current layer encoder for feature extraction and fusion to obtain the current layer's coding fusion feature. The above process is repeated layer by layer until the time dimension frame number is downsampled to 1, at which point the single-frame sequence and the multi-frame hybrid sequence are consistent. The multi-frame hybrid sequence is then directly sent to the encoder to output the deepest layer's coding fusion feature.

[0182] 4D point cloud data contains information in both spatial and temporal dimensions. Spatial processing focuses on the spatial structure within each frame, while temporal processing focuses on the changes of the same voxel over time. Dual-path parallel processing can extract spatial and temporal features separately, and then achieve deep coupling of spatiotemporal information through fusion. The hierarchical encoder gradually expands the receptive field by downsampling layer by layer, extracting higher-level semantic features.

[0183] Layer 0 encoder: Arrange the voxel feature points of each frame according to the voxel rearrangement sequence obtained in step S3 to form a multi-frame mixed sequence. That is, all voxels of all frames are mixed and arranged in the rearranged order. It contains one sequence with a length of N × T (N is the number of voxels, T is the number of frames). Based on an initial time dimension of 8 frames T, it will... Divided into 8 single-frame sequences Each single-frame sequence contains feature data of N voxels in the current frame. It contains 8 sequences, each with a length of N.

[0184] Multi-frame mixed sequence The time-path Mamba module fed into the layer 0 encoder yields multi-frame fused features. The temporal path Mamba module can capture long-range dependencies of the same voxel in the time dimension and identify motion patterns.

[0185] Eight single-frame sequences are fed into the spatial path Mamba module of the layer 0 encoder, with each single-frame sequence processed independently by a Mamba module to obtain the single-frame features of each frame. The single-frame features of all frames are then fused using one-dimensional convolution to obtain the fused single-frame features. The spatial path Mamba module can capture spatial dependencies between voxels within the same frame and identify the spatial structure of objects.

[0186] Multi-frame fusion features Features fused with single frames During encoding, fusion is performed to obtain the 0th layer encoded fusion feature. .

[0187] Layer 1 encoder: Encodes and fuses features from layer 0. Spatiotemporal downsampling is performed. Spatial downsampling is achieved by right-shifting the spatial code, and temporal downsampling is achieved by right-shifting the temporal code. Specifically, each voxel is encoded using the space-filling curve, and the spatial and temporal codes are concatenated into a total code. Independent downsampling of the spatial and temporal dimensions is achieved by right-shifting the spatial and temporal code bits respectively. After downsampling, voxels with the same code are merged, and the merged feature is obtained by taking the maximum value of the voxel features and then performing a linear mapping. In this embodiment, spatial downsampling is doubled, and temporal downsampling is doubled (the number of frames is reduced from 8 to 4). The downsampled features are used as the input of the first-layer encoder. The processing flow of the Layer 1 encoder is the same as that of Layer 0. At this point, the time dimension frame number T=4. Split into 4 single-frame sequences Mixing multiple frames into a sequence The data is fed into the time-path Mamba module to obtain multi-frame fusion features. The four single-frame sequences are fed into the spatial path Mamba module, and the single-frame features of each frame are fused by one-dimensional convolution to obtain the single-frame fused features. According to the coding fusion method, multi-frame fusion features are used. Generate gated weights for time reference signals fused features from a single frame Adaptive injection yields the first-layer encoded fusion features. The specific formula is as follows:

[0188]

[0189]

[0190] σ is the sigmoid activation function. For Gaussian error linear units, This is the weight matrix of the first-level linear transformation during encoding. This is the weight vector for the second-level linear transformation during encoding. This is the bias of the first-level linear transform during encoding. This is the bias for the second-level linear transformation.

[0191] Layer 2 encoder: (To be continued) Perform spatiotemporal downsampling (spatial downsampling by 2x, temporal downsampling by 2x, frame rate reduced from 4 to 2) to obtain T=2, split into 2 single-frame sequences. Mixing multiple frames into a sequence The data is fed into the time-path Mamba module to obtain multi-frame fusion features. The two single-frame sequences are fed into the spatial path Mamba module respectively. The single-frame features of each frame are fused by one-dimensional convolution to obtain the single-frame fused features. The second-layer coded fusion feature is obtained according to the coded fusion method. The specific formula is as follows:

[0192]

[0193]

[0194] σ is the sigmoid activation function. For Gaussian error linear units, This is the weight matrix of the first-level linear transformation during encoding. This is the weight vector for the second-level linear transformation during encoding. This is the bias of the first-level linear transform during encoding. This is the bias for the second-level linear transformation.

[0195] Layer 3 encoder: (This likely refers to a specific type of encoder or encoder, but without further context, it's difficult to translate accurately Perform spatiotemporal downsampling (spatial downsampling by 2x, temporal downsampling by 2x, frame rate reduced from 2 to 1) to obtain At this point, T=1, the single-frame sequence and the multi-frame mixed sequence are consistent, so we can directly... The data is fed into the encoder and obtained. .

[0196] At this point, the encoder outputs the coded fusion features of each layer. , , , used for skip-connection fusion in the decoder.

[0197] S5. The deepest layer output of the encoder is used as the initial input of the decoder. The output is upsampled layer by layer and fused with the encoding fusion features of the corresponding layer of the encoder during decoding. After restoring the original resolution layer by layer, the moving target segmentation result is output by the classification head. The segmentation loss is calculated based on the moving target segmentation result output by the classification head and the ground truth. The sum of the segmentation loss and the topological continuity loss is used as the total loss of the current round. The U-Net network parameters are updated based on the total loss of the current round.

[0198] In an optional implementation, the step of using the deepest layer output of the encoder as the initial input of the decoder, upsampling layer by layer and fusing it with the encoded fusion features of the corresponding layer of the encoder during decoding, and restoring the original resolution layer by layer before outputting the moving target segmentation result through the classification head includes:

[0199] The coding fusion feature output from the deepest layer of the encoder is used as the upsampling input from the deepest layer of the decoder, and the coding fusion feature output from the second deepest layer of the encoder is used as the skip connection feature. The upsampling feature and the skip connection feature are fused together to obtain the deepest layer decoding fusion feature.

[0200] The features upsampled from the previous layer's decoding fusion features are used as the upsampled input to the current layer's decoder. The encoding fusion features output from the corresponding layer of the encoder are used as the skip connection features. The upsampled features and the skip connection features are fused together to obtain the current layer's intermediate decoding features. The current layer's intermediate decoding features are used as a multi-frame mixed sequence, which is split into a corresponding number of single-frame sequences according to the current time dimension frame number. The multi-frame mixed sequence and the multiple single-frame sequences are respectively fed into the dual-path Mamba module of the current layer's decoder for feature extraction and fusion to obtain the current layer's decoding fusion features.

[0201] Upsample layer by layer and repeat the above process until the original resolution is restored. Take the part of the shallowest layer decoded fusion feature corresponding to the current frame as the voxel feature of each voxel. Map each point to its voxel and use the corresponding voxel feature as the feature of that point. Output the moving target segmentation result through the linear classification head.

[0202] Preferably, the decoder recovers spatial resolution through layer-by-layer upsampling and incorporates detailed features from each layer of the encoder through skip-connection fusion, achieving a combination of semantic information and spatial details. The decoding fusion is performed in two steps: first, gated fusion is applied to shallow skip-connection features using deep-layer upsampled features to suppress noise in the skip-connection features; then, gated fusion is applied to temporal-layer features using spatial-path features to further optimize feature representation. Each layer of the decoder also needs to split the multi-frame mixed sequence frame by frame and process them separately.

[0203] Layer 3 decoder: Coordinated features fused from the deepest layer output of the encoder. As the upsampled input of the third layer of the decoder Encoded fusion features output from the second deep layer (layer 2) of the encoder As a jump combo feature First, perform skip-gated fusion. Then, the upsampled features... Gated weights are generated through a linear layer and a sigmoid activation layer. :

[0204]

[0205] in This is the linear transformation weight matrix for the third-level jump-gated system. The bias is used for the linear transformation of the third-level jump-gated system. The sigmoid activation function is used. The gating weight modulates the shallow jump connection features with reference to the deep upsampled features: for regions with significant deep semantic information (such as the location of moving targets), the gating weight approaches 1, and the spatial details in the jump connection features are fully preserved; for regions that are determined to be background in the deep layers, the gating weight approaches 0, and the noise in the jump connection features is effectively suppressed.

[0206] Gating weights Acting on jump combo features , with upsampling features After being concatenated along the channel dimension, the intermediate decoding features are obtained through linear layer fusion. :

[0207]

[0208] in, This represents a linear mapping layer.

[0209] Since the third layer has 1 frame T in the time dimension, the intermediate decoding features... Directly used as a multi-frame mixing sequence The Mamba module, fed into the Layer 3 decoder, outputs the Layer 3 decoded fusion features. .

[0210] Layer 2 decoder: Features are fused using Layer 3 decoding. The upsampled features are used as the upsampled input to the second-layer decoder. The encoder uses the first layer of encoding to fuse features. As a jump combo feature First, perform skip-gated fusion. Then, the upsampled features... Jump-gated weights are generated through a linear layer and a sigmoid activation function. :

[0211]

[0212] in This is the linear transformation weight matrix for the second-level jump-gated control. The bias of the linear transformation of the second-level jump-gated system is used to adjust the gating weights. Acting on jump combo features , with upsampling features After concatenation, the intermediate decoded features are obtained through linear layer fusion. :

[0213]

[0214] right Upsampling is performed (spatial upsampling by 2x, temporal upsampling by 2x, and the frame number is restored from 1 to 2). The upsampled features are denoted as... This serves as the input to the multi-frame mixed sequence of the second-layer decoder. At this point, T=2, and... Split into two single-frame sequences based on frame tags. .

[0215] Multi-frame mixed sequence The data is fed into the time-path Mamba module to obtain multi-frame fusion features. The two single-frame sequences are fed into the spatial path Mamba module respectively. The single-frame features of each frame are fused by one-dimensional convolution to obtain the single-frame fused features. .

[0216] Then, dual-channel gating fusion is performed. The intermediate decoded features are used as the basis for this process. As a reference signal, generate spatially guided gating weights. :

[0217]

[0218] in This is the weight matrix for the first-level linear transformation. This is the weight vector for the second-level linear transformation. , For the corresponding bias term, GELU is the Gaussian error linear unit activation function. This gate weight uses spatial structure features as a reference to control the injection intensity of temporal features: for voxels with clear spatial structure, the injection of temporal features is enhanced, and motion information is used to assist segmentation; for voxels with ambiguous structure, the injection mainly relies on the intermediate decoded features themselves.

[0219] Finally, the second layer decoding fusion feature is obtained. :

[0220]

[0221] Layer 1 decoder: Features are fused using Layer 2 decoding. The upsampled features are used as the upsampled input. The encoder uses layer 0 to encode and fuse features. As a jump combo feature First, perform skip-gated fusion. Then, the upsampled features... Jump-gated weights are generated through a linear layer and a sigmoid activation function. :

[0222]

[0223] Gating weights Acting on jump combo features , with upsampling features After concatenation, the intermediate decoded features are obtained through linear layer fusion. :

[0224]

[0225] right Upsampling is performed (spatial upsampling by 2x, temporal upsampling by 2x, and the frame count is restored from 2 to 4). The upsampled features are denoted as... This serves as the input to the multi-frame mixed sequence of the first-layer decoder. At this point, T=4, and... It is split into 4 single-frame sequences based on frame tags. .

[0226] Multi-frame mixed sequence The data is fed into the time-path Mamba module to obtain multi-frame fusion features. The four single-frame sequences are fed into the spatial path Mamba module, and the single-frame features of each frame are fused by one-dimensional convolution to obtain the single-frame fused features. .

[0227] Then, dual-channel gating fusion is performed. The intermediate decoded features are used as the basis for this process. As a reference signal, generate spatially guided gating weights. :

[0228]

[0229] Finally, the first layer decoding and fusion features are obtained. :

[0230]

[0231] Layer 0 decoder: fuses features from the Layer 1 decoder. Upsampling is performed (spatial upsampling by 2x, temporal upsampling by 2x, and the frame count is restored from 2 to 4). The upsampled features are denoted as... This serves as the input to the multi-frame mixed sequence of the first-layer decoder. At this point, T=8, and... It is split into 8 single-frame sequences based on frame tags. .

[0232] Multi-frame mixed sequence The data is fed into the time-path Mamba module to obtain multi-frame fusion features. Eight single-frame sequences are fed into the spatial path Mamba module, and the single-frame features of each frame are fused by one-dimensional convolution to obtain the single-frame fused features. .

[0233] At this point, since layer 0 is the shallowest layer, there is no need to perform skip-connection gating fusion (this layer does not involve encoder skip-connection). The multi-frame mixed sequence is directly used as the intermediate decoding feature for dual-path gating fusion. Spatially guided gating weights are generated. :

[0234]

[0235] Obtain the decoding and fusion features of layer 0 :

[0236]

[0237] Pick From the portion corresponding to the current frame (t=0), obtain the voxel features F of each voxel. vox .

[0238] Classification Header: The features of each point directly use the features of its corresponding voxel, i.e. , where π(i) represents the voxel to which point i belongs. A linear mapping layer is used as the classification head to obtain the three-class classification results (dynamic target, static background, unknown) for each point. The cross-entropy segmentation loss is calculated based on the classification head output and the ground truth. Total losses in the current round Update the U-Net network parameters based on the total loss of the current round.

[0239] S6. Repeat S1 to S5 until the total loss value of the current round fluctuates within the preset range, then stop training and obtain the target U-Net network.

[0240] S7. During inference, the initial sorting sequence of each voxel is input into the target U-Net network in reverse frame order, and the moving target segmentation result is output.

[0241] The initial sorting sequence [B, C, A, D] obtained in S1 is reversed frame by frame, i.e., [B7,B6,B5,B4,B3,B2,B1,B0,C7,C6,...,C0,A7,...,A0,D7,...,D0] is input into the target U-Net network, skipping the local rearrangement step in S3, and outputting the moving target segmentation results for each point.

[0242] Figure 2 This is a schematic diagram of the structure of a 4D point cloud moving target segmentation system provided in an embodiment of the present invention. Figure 2 As shown, the system includes:

[0243] The serialization unit 201 is used to acquire a single frame point cloud, divide the single frame point cloud into multiple voxels according to a preset voxel size, and randomly use a space filling curve to perform spatial serialization on each voxel to obtain an initial sorting sequence of each voxel.

[0244] The acquisition unit 202 is used to acquire the point cloud information of each voxel in multiple frames, retain the information of the point with the largest intensity value in each voxel in each frame, and obtain the feature points of each voxel in each frame.

[0245] The sequence rearrangement unit 203 is used to input the feature points of each voxel in each frame into the initial U-Net network based on the initial sorting sequence, extract the sorting position features, intensity change features, and temporal features of each voxel in each frame, calculate the perturbation score of each voxel frame by frame based on the extracted features, calculate the topological continuity loss based on the perturbation score of each voxel and the initial sorting sequence, and perform local rearrangement of all voxels in all frames based on the perturbation score to obtain the voxel rearrangement sequence of all frames.

[0246] The encoding unit 204 is used to arrange the voxel feature points of each frame according to the voxel rearrangement sequence to obtain a multi-frame mixed sequence, and split it into a corresponding number of single-frame sequences according to the frame; after feature extraction by the multi-frame mixed sequence and each single-frame sequence through the dual-channel Mamba module of the current layer of the encoder, the sequences are fused to obtain the encoded fusion features; after spatiotemporal downsampling of the encoded fusion features, they are used as the input of the next layer encoder, and the above process is repeated layer by layer until the deepest layer of the encoder, and the encoded fusion features of each layer are output.

[0247] The decoding unit 205 is used to take the deepest layer output of the encoder as the initial input of the decoder, upsample layer by layer and fuse it with the encoded fusion features of the corresponding layer of the encoder during decoding, restore the original resolution layer by layer, and output the moving target segmentation result through the classification head; calculate the segmentation loss based on the moving target segmentation result output by the classification head and the ground truth; use the sum of the segmentation loss and the topological continuity loss as the total loss of the current round, and update the U-Net network parameters based on the total loss of the current round;

[0248] The repeated training unit 206 is used to repeatedly execute the serialization unit, acquisition unit, sequence rearrangement unit, encoding unit and decoding unit until the total loss value of the current round fluctuates within a preset range, then training stops and the target U-Net network is obtained.

[0249] The prediction unit 207, during inference, inputs the initial sorting sequence of each voxel into the target U-Net network in reverse frame order for prediction and outputs the moving target segmentation result.

[0250] In an optional implementation, the sequence rearrangement unit 203 includes:

[0251] The acquisition sub-unit is used to input the feature points of each voxel in each frame into the initial U-Net network based on the initial sorting sequence, and obtain the normalized sort position φ of each voxel in each frame according to the initial sorting sequence. norm The intensity difference Δi between adjacent voxels and the normalized time Δt of each voxel in each frame are calculated based on the feature points of each voxel in each frame.

[0252] The spatial feature extraction subunit is used to extract the normalized sorting position φ. norm After being concatenated with the intensity difference Δi of the adjacent voxels, the spatial features Fspat of each voxel in the current frame are obtained by sequentially passing through the first one-dimensional convolutional layer, batch normalization layer, ReLU activation layer, second one-dimensional convolutional layer and ReLU activation layer.

[0253] The intermediate time feature extraction subunit is used to take the normalized time Δt as a one-dimensional input feature and pass it through the first one-dimensional convolutional layer, batch normalization layer, ReLU activation layer, second one-dimensional convolutional layer and ReLU activation layer in sequence to obtain the intermediate time features of each voxel in the current frame.

[0254] The time feature calculation subunit is used to average the intermediate time features of all voxels in the current frame, generate the weight of the current frame through a multilayer perceptron and a sigmoid activation function, and multiply the intermediate time features of each voxel in the current frame by the weight of the current frame to obtain the time feature Ftemp of each voxel in the current frame.

[0255] The fraction calculation subunit is used to concatenate the spatial features Fspat and the temporal features Ftemp of each voxel in the current frame and input them into the multilayer perceptron, and output the perturbation score sj of each voxel in the current frame.

[0256] In an optional implementation, the sequence rearrangement unit 203 further includes:

[0257] The window division subunit is used to insert the feature points of each voxel in each frame into the initial sorting sequence in reverse chronological order, based on the initial sorting sequence of each voxel, to form a complete sequence in which the voxels of all frames are arranged continuously according to their sequence positions; the complete sequence is divided into multiple windows using a sliding window strategy, with a window size of K and a step size of K / 2.

[0258] The mutation detection subunit is used to calculate the true distance dj between adjacent voxels in three-dimensional space within each window. If the true distance dj exceeds a preset threshold τ, the current voxel position is marked as a spatial mutation boundary, and the window is truncated in advance at the first mutation position to obtain an effective window.

[0259] The window rearrangement subunit is used to rearrange the voxels in ascending order according to the perturbation score sj within each valid window. If the perturbation score of a voxel in the current window is less than the minimum value of the perturbation scores of all voxels in the previous window, the voxel is removed from the current window and inserted into the corresponding sorting position in the previous window according to the perturbation score. The voxel rearrangement sequence of all frames is then output.

[0260] In an optional implementation, the encoding unit 204 includes:

[0261] The voxel feature points of each frame are arranged according to the voxel rearrangement sequence to obtain a multi-frame mixed sequence. The sequence is then split into a corresponding number of single-frame sequences according to the initial time dimension frame number. The multi-frame mixed sequence and each single-frame sequence are respectively fed into the temporal path Mamba module and spatial path Mamba module of the layer 0 encoder for feature extraction and fusion to obtain the layer 0 encoded fusion feature.

[0262] Using the previous layer's coding fusion feature as the current layer's multi-frame hybrid sequence, the multi-frame hybrid sequence is spatiotemporally downsampled and split into a corresponding number of single-frame sequences according to the updated time dimension frame number after downsampling. The multi-frame hybrid sequence and each single-frame sequence are respectively sent to the temporal path Mamba module and spatial path Mamba module of the current layer encoder for feature extraction and fusion to obtain the current layer's coding fusion feature. The above process is repeated layer by layer until the time dimension frame number is downsampled to 1, at which point the single-frame sequence and the multi-frame hybrid sequence are consistent. The multi-frame hybrid sequence is then directly sent to the encoder to output the deepest layer's coding fusion feature.

[0263] In an optional implementation, the decoding unit 205 includes:

[0264] The coding fusion feature output from the deepest layer of the encoder is used as the upsampling input from the deepest layer of the decoder, and the coding fusion feature output from the second deepest layer of the encoder is used as the skip connection feature. The upsampling feature and the skip connection feature are fused together to obtain the deepest layer decoding fusion feature.

[0265] The features upsampled from the previous layer's decoding fusion features are used as the upsampled input to the current layer's decoder. The encoding fusion features output from the corresponding layer of the encoder are used as the skip connection features. The upsampled features and the skip connection features are fused together to obtain the current layer's intermediate decoding features. The current layer's intermediate decoding features are used as a multi-frame mixed sequence, which is split into a corresponding number of single-frame sequences according to the current time dimension frame number. The multi-frame mixed sequence and the multiple single-frame sequences are respectively fed into the dual-path Mamba module of the current layer's decoder for feature extraction and fusion to obtain the current layer's decoding fusion features.

[0266] Upsample layer by layer and repeat the above process until the original resolution is restored. Take the part of the shallowest layer decoded fusion feature corresponding to the current frame as the voxel feature of each voxel. Map each point to its voxel and use the corresponding voxel feature as the feature of that point. Output the moving target segmentation result through the linear classification head.

[0267] The system of the present invention corresponds to the method described above, and the specific implementation of the system will not be repeated here.

[0268] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A 4D point cloud moving target segmentation method, characterized in that, include: S1. Obtain a single-frame point cloud, divide the single-frame point cloud into multiple voxels according to a preset voxel size, and randomly use a space filling curve to spatially serialize each voxel to obtain the initial sorting sequence of each voxel. S2. Obtain the point cloud information of each voxel in multiple frames, and retain the information of the point with the largest intensity value in each voxel in each frame to obtain the feature points of each voxel in each frame. S3. Using the initial sorting sequence as a reference, input the feature points of each voxel in each frame into the initial U-Net network. Extract the sorting position features, intensity change features, and temporal features of each voxel in each frame. Calculate the perturbation score of each voxel frame by frame based on the extracted features. Calculate the topological continuity loss based on the perturbation scores of each voxel and the initial sorting sequence. Perform local rearrangement of all voxels in all frames based on the perturbation scores to obtain the voxel rearrangement sequence for all frames. S4. Arrange the voxel feature points of each frame according to the voxel rearrangement sequence to obtain a multi-frame mixed sequence, and split it into a corresponding number of single-frame sequences according to the frame; after feature extraction of the multi-frame mixed sequence and each single-frame sequence by the dual-path Mamba module of the current layer of the encoder, fuse them to obtain the encoded fusion feature. After performing spatiotemporal downsampling on the coded fusion features, they are used as the input to the next layer of the encoder. The above process is repeated layer by layer until the deepest layer of the encoder is reached, and the coded fusion features of each layer are output. S5. The deepest layer output of the encoder is used as the initial input of the decoder. The output is upsampled layer by layer and fused with the encoding fusion features of the corresponding layer of the encoder during decoding. After restoring the original resolution layer by layer, the moving target segmentation result is output by the classification head. The segmentation loss is calculated based on the moving target segmentation result output by the classification head and the ground truth. The sum of the segmentation loss and the topological continuity loss is used as the total loss of the current round. The U-Net network parameters are updated based on the total loss of the current round. S6. Repeat S1 to S5 until the total loss value of the current round fluctuates within the preset range, then stop training and obtain the target U-Net network. S7. During inference, the initial sorting sequence of each voxel is input into the target U-Net network in reverse frame order for prediction, and the moving target segmentation result is output.

2. The method according to claim 1, characterized in that, The step of inputting the feature points of each voxel in each frame into the initial U-Net network based on the initial sorting sequence, extracting the sorting position features, intensity change features, and temporal features of each voxel in each frame, and calculating the perturbation score of each voxel frame by frame based on the extracted features includes: The feature points of each voxel in each frame are input into the initial U-Net network based on the initial sorting sequence, and the normalized sorting position φ of each voxel in each frame is obtained according to the initial sorting sequence. norm The intensity difference Δi between adjacent voxels and the normalized time Δt of each voxel in each frame are calculated based on the feature points of each voxel in each frame. The normalized sorting position φ norm After being concatenated with the intensity difference Δi of the adjacent voxels, the spatial features Fspat of each voxel in the current frame are obtained by sequentially passing through the first one-dimensional convolutional layer, batch normalization layer, ReLU activation layer, second one-dimensional convolutional layer and ReLU activation layer. The normalized time Δt is used as a one-dimensional input feature, and then passed through a first one-dimensional convolutional layer, a batch normalization layer, a ReLU activation layer, a second one-dimensional convolutional layer, and a ReLU activation layer in sequence to obtain the intermediate time features of each voxel in the current frame. The intermediate time features of all voxels in the current frame are averaged, and the weights of the current frame are generated by a multilayer perceptron and a sigmoid activation function. The intermediate time features of each voxel in the current frame are multiplied by the weights of the current frame to obtain the time features Ftemp of each voxel in the current frame. The spatial features Fspat and temporal features Ftemp of each voxel in the current frame are concatenated and then input into a multilayer perceptron to output the perturbation score sj of each voxel in the current frame.

3. The method according to claim 1, characterized in that, The step of locally rearranging all voxels in all frames based on the perturbation score to obtain the voxel rearrangement sequence for all frames includes: Based on the initial sorting sequence of each voxel, the feature points of each voxel in each frame are inserted into the initial sorting sequence in reverse chronological order to form a complete sequence in which the voxels of all frames are arranged continuously according to their sequence positions; the complete sequence is divided into multiple windows using a sliding window strategy, with a window size of K and a step size of K / 2. Within each window, the true distance dj between adjacent voxels in three-dimensional space is calculated. If the true distance dj exceeds a preset threshold τ, the current voxel position is marked as a spatial mutation boundary, and the window is truncated in advance at the first mutation position to obtain an effective window. Within each valid window, voxels are reordered in ascending order of the perturbation score sj. If the perturbation score of a voxel in the current window is less than the minimum perturbation score of all voxels in the previous window, the voxel is removed from the current window and inserted into the corresponding sorting position in the previous window according to the perturbation score. The voxel rearrangement sequence of all frames is then output.

4. The method according to claim 1, characterized in that, S4 includes: The voxel feature points of each frame are arranged according to the voxel rearrangement sequence to obtain a multi-frame mixed sequence. The sequence is then split into a corresponding number of single-frame sequences according to the initial time dimension frame number. The multi-frame mixed sequence and each single-frame sequence are respectively fed into the temporal path Mamba module and spatial path Mamba module of the layer 0 encoder for feature extraction and fusion to obtain the layer 0 encoded fusion feature. Using the previous layer's coding fusion feature as the current layer's multi-frame hybrid sequence, the multi-frame hybrid sequence is spatiotemporally downsampled and split into a corresponding number of single-frame sequences according to the updated time dimension frame number after downsampling. The multi-frame hybrid sequence and each single-frame sequence are respectively sent to the temporal path Mamba module and spatial path Mamba module of the current layer encoder for feature extraction and fusion to obtain the current layer's coding fusion feature. The above process is repeated layer by layer until the time dimension frame number is downsampled to 1, at which point the single-frame sequence and the multi-frame hybrid sequence are consistent. The multi-frame hybrid sequence is then directly sent to the encoder to output the deepest layer's coding fusion feature.

5. The method according to claim 4, characterized in that, The process of using the deepest layer output of the encoder as the initial input of the decoder, upsampling layer by layer and fusing it with the encoded fusion features of the corresponding layer of the encoder during decoding, and restoring the original resolution layer by layer before outputting the moving target segmentation result through the classification head includes: The coding fusion feature output from the deepest layer of the encoder is used as the upsampling input from the deepest layer of the decoder, and the coding fusion feature output from the second deepest layer of the encoder is used as the skip connection feature. The upsampling feature and the skip connection feature are fused together to obtain the deepest layer decoding fusion feature. The features upsampled from the previous layer's decoding fusion features are used as the upsampled input to the current layer's decoder. The encoding fusion features output from the corresponding layer of the encoder are used as the skip connection features. The upsampled features and the skip connection features are fused together to obtain the current layer's intermediate decoding features. The current layer's intermediate decoding features are used as a multi-frame mixed sequence, which is split into a corresponding number of single-frame sequences according to the current time dimension frame number. The multi-frame mixed sequence and the multiple single-frame sequences are respectively fed into the dual-path Mamba module of the current layer's decoder for feature extraction and fusion to obtain the current layer's decoding fusion features. Upsample layer by layer and repeat the above process until the original resolution is restored. Take the part of the shallowest layer decoded fusion feature corresponding to the current frame as the voxel feature of each voxel. Map each point to its voxel and use the corresponding voxel feature as the feature of that point. Output the moving target segmentation result through the linear classification head.

6. A 4D point cloud moving target segmentation system, characterized in that, include: The serialization unit is used to acquire a single frame point cloud, divide the single frame point cloud into multiple voxels according to a preset voxel size, and randomly use a space filling curve to perform spatial serialization on each voxel to obtain the initial sorting sequence of each voxel. The acquisition unit is used to acquire the point cloud information of each voxel over multiple frames, retain the information of the point with the largest intensity value in each voxel in each frame, and obtain the feature points of each voxel in each frame. The sequence rearrangement unit is used to input the feature points of each voxel in each frame into the initial U-Net network based on the initial sorting sequence, extract the sorting position features, intensity change features, and temporal features of each voxel in each frame, calculate the perturbation score of each voxel frame by frame based on the extracted features, calculate the topological continuity loss based on the perturbation score of each voxel and the initial sorting sequence, and perform local rearrangement of all voxels in all frames based on the perturbation score to obtain the voxel rearrangement sequence of all frames. The encoding unit is used to arrange the voxel feature points of each frame according to the voxel rearrangement sequence to obtain a multi-frame mixed sequence, and split it into a corresponding number of single-frame sequences according to the frame; the multi-frame mixed sequence and each single-frame sequence are respectively processed by the dual-path Mamba module of the current layer of the encoder for feature extraction and then fused to obtain the encoded fusion feature; After performing spatiotemporal downsampling on the coded fusion features, they are used as the input to the next layer of the encoder. The above process is repeated layer by layer until the deepest layer of the encoder is reached, and the coded fusion features of each layer are output. The decoding unit is used to take the deepest layer output of the encoder as the initial input of the decoder, upsample layer by layer and fuse it with the encoded fusion features of the corresponding layer of the encoder during decoding, restore the original resolution layer by layer, and output the moving target segmentation result through the classification head; calculate the segmentation loss based on the moving target segmentation result output by the classification head and the ground truth; use the sum of the segmentation loss and the topological continuity loss as the total loss of the current round, and update the U-Net network parameters based on the total loss of the current round; The repeated training unit is used to repeatedly execute the serialization unit, acquisition unit, sequence rearrangement unit, encoding unit, and decoding unit until the total loss value of the current round fluctuates within a preset range, at which point training stops and the target U-Net network is obtained. The prediction unit, during inference, inputs the initial sorting sequence of each voxel into the target U-Net network frame by frame for prediction and outputs the moving target segmentation result.

7. The system according to claim 6, characterized in that, The sequence rearrangement unit includes: The acquisition sub-unit is used to input the feature points of each voxel in each frame into the initial U-Net network based on the initial sorting sequence, and obtain the normalized sort position φ of each voxel in each frame according to the initial sorting sequence. norm The intensity difference Δi between adjacent voxels and the normalized time Δt of each voxel in each frame are calculated based on the feature points of each voxel in each frame. The spatial feature extraction subunit is used to extract the normalized sorting position φ. norm After being concatenated with the intensity difference Δi of the adjacent voxels, the spatial features Fspat of each voxel in the current frame are obtained by sequentially passing through the first one-dimensional convolutional layer, batch normalization layer, ReLU activation layer, second one-dimensional convolutional layer and ReLU activation layer. The intermediate time feature extraction subunit is used to take the normalized time Δt as a one-dimensional input feature and pass it through the first one-dimensional convolutional layer, batch normalization layer, ReLU activation layer, second one-dimensional convolutional layer and ReLU activation layer in sequence to obtain the intermediate time features of each voxel in the current frame. The time feature calculation subunit is used to average the intermediate time features of all voxels in the current frame, generate the weight of the current frame through a multilayer perceptron and a sigmoid activation function, and multiply the intermediate time features of each voxel in the current frame by the weight of the current frame to obtain the time feature Ftemp of each voxel in the current frame. The fraction calculation subunit is used to concatenate the spatial features Fspat and the temporal features Ftemp of each voxel in the current frame and input them into the multilayer perceptron, and output the perturbation score sj of each voxel in the current frame.

8. The system according to claim 6, characterized in that, The sequence rearrangement unit further includes: The window division subunit is used to insert the feature points of each voxel in each frame into the initial sorting sequence in reverse chronological order, based on the initial sorting sequence of each voxel, to form a complete sequence in which the voxels of all frames are arranged continuously according to their sequence positions; the complete sequence is divided into multiple windows using a sliding window strategy, with a window size of K and a step size of K / 2. The mutation detection subunit is used to calculate the true distance dj between adjacent voxels in three-dimensional space within each window. If the true distance dj exceeds a preset threshold τ, the current voxel position is marked as a spatial mutation boundary, and the window is truncated in advance at the first mutation position to obtain an effective window. The window rearrangement subunit is used to rearrange the voxels in ascending order according to the perturbation score sj within each valid window. If the perturbation score of a voxel in the current window is less than the minimum value of the perturbation scores of all voxels in the previous window, the voxel is removed from the current window and inserted into the corresponding sorting position in the previous window according to the perturbation score. The voxel rearrangement sequence of all frames is then output.

9. The system according to claim 6, characterized in that, The encoding unit includes: The voxel feature points of each frame are arranged according to the voxel rearrangement sequence to obtain a multi-frame mixed sequence. The sequence is then split into a corresponding number of single-frame sequences according to the initial time dimension frame number. The multi-frame mixed sequence and each single-frame sequence are respectively fed into the temporal path Mamba module and spatial path Mamba module of the layer 0 encoder for feature extraction and fusion to obtain the layer 0 encoded fusion feature. Using the previous layer's coding fusion feature as the current layer's multi-frame hybrid sequence, the multi-frame hybrid sequence is spatiotemporally downsampled and split into a corresponding number of single-frame sequences according to the updated time dimension frame number after downsampling. The multi-frame hybrid sequence and each single-frame sequence are respectively sent to the temporal path Mamba module and spatial path Mamba module of the current layer encoder for feature extraction and fusion to obtain the current layer's coding fusion feature. The above process is repeated layer by layer until the time dimension frame number is downsampled to 1, at which point the single-frame sequence and the multi-frame hybrid sequence are consistent. The multi-frame hybrid sequence is then directly sent to the encoder to output the deepest layer's coding fusion feature.

10. The system according to claim 9, characterized in that, The decoding unit includes: The coding fusion feature output from the deepest layer of the encoder is used as the upsampling input from the deepest layer of the decoder, and the coding fusion feature output from the second deepest layer of the encoder is used as the skip connection feature. The upsampling feature and the skip connection feature are fused together to obtain the deepest layer decoding fusion feature. The features upsampled from the previous layer's decoding fusion features are used as the upsampled input to the current layer's decoder. The encoding fusion features output from the corresponding layer of the encoder are used as the skip connection features. The upsampled features and the skip connection features are fused together to obtain the current layer's intermediate decoding features. The current layer's intermediate decoding features are used as a multi-frame mixed sequence, which is split into a corresponding number of single-frame sequences according to the current time dimension frame number. The multi-frame mixed sequence and the multiple single-frame sequences are respectively fed into the dual-path Mamba module of the current layer's decoder for feature extraction and fusion to obtain the current layer's decoding fusion features. Upsample layer by layer and repeat the above process until the original resolution is restored. Take the part of the shallowest layer decoded fusion feature corresponding to the current frame as the voxel feature of each voxel. Map each point to its voxel and use the corresponding voxel feature as the feature of that point. Output the moving target segmentation result through the linear classification head.

Images