Point cloud video behavior recognition method based on redundancy perception and decoupling modeling

Abstract

The invention belongs to the technical field of computer vision, and discloses a point cloud video behavior recognition method based on redundancy perception and decoupling modeling. The disordered 4D point cloud video data is mapped to an ordered Hash grid; calculating the space-time redundancy of each Hash grid, obtaining a local area of each point cloud through Hash search, and calculating the redundancy; adding geometric information into the point cloud through a geometric affine module, coding through a convolutional layer to obtain aggregated features of each local area, and distinguishing a static feature sequence and a dynamic feature sequence according to space-time redundancy; inputting the static feature sequence and the dynamic feature sequence into a bidirectional space-time Mama network; in a bidirectional space-time Mama network, firstly, a space topological relation of point cloud frames is constructed through space modeling, and repeated modeling of redundant static feature sequences is eliminated in time modeling; and finally, the obtained output features are sent to an MLP classification layer for behavior identification and classification.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a point cloud video behavior recognition method based on redundant perception and decoupling modeling. Background Technology

[0002] In recent years, point cloud data, as a representation of three-dimensional space, has been widely used in many fields, especially in technologies such as autonomous driving, robot navigation, augmented reality, and 3D modeling. Point cloud video, with its advantage of simultaneously capturing temporal and spatial information, has become a research hotspot. However, point cloud video data is characterized by its disorder, sparsity, and complex spatiotemporal structure, making efficient and accurate spatiotemporal modeling a challenging research problem. To address these issues, existing technologies have proposed various point cloud video understanding methods, including modeling schemes based on voxelization, convolutional neural networks, and Transformers. However, these methods all have limitations in terms of computational efficiency, long-sequence modeling capabilities, and geometric information preservation.

[0003] In recent years, State Space Models (SSMs) have been increasingly adopted in the computer vision field due to their linear time complexity and strong global dependency modeling capabilities for long sequence modeling, and have begun to be applied to tasks such as point cloud video action recognition. Compared with Transformer methods based on self-attention mechanisms, SSMs have lower computational complexity and memory consumption when processing long sequences, and can significantly improve the computational efficiency of the model while maintaining the global receptive field, making them more suitable for modeling long sequence point cloud videos.

[0004] Against this research backdrop, existing literature has proposed point cloud video understanding techniques based on state-space models, such as the "Mamba4D: Efficient 4D Point Cloud Video Understanding with Disentangled Spatial-Temporal State Space Models" method presented at CVPR 2025. This approach, centered on state-space models, constructs a four-dimensional feature modeling framework for point cloud video understanding. Its main function is to efficiently model the spatial geometry and temporal dynamics of point cloud videos. Specifically, this approach adopts a modeling approach that decouples spatial and temporal dimensions. First, by selecting temporal anchor frames and constructing short-term point cloud spatiotemporal segments, it encodes local spatial neighborhoods and short-term dynamic changes. Then, using a temporal modeling module based on state-space models, it aggregates point cloud features across frames to achieve modeling of global dynamic information in long-term point cloud video sequences. This scheme transforms disordered point clouds into a sequence that can be processed by a state-space model by spatial sampling and neighborhood search. It also combines spatial location information and temporal information to jointly encode point cloud features, thereby achieving comprehensive modeling of local geometric structures, short-term motion patterns, and long-term temporal relationships.

[0005] Point cloud videos differ significantly from traditional video or image data. Point cloud data itself possesses high spatial dimensionality and sparsity, while point cloud videos introduce a temporal dimension, increasing processing complexity. In practical applications, many point cloud data points in point cloud videos exhibit extremely small temporal changes, even showing almost no change between consecutive frames. Traditional image and video recognition methods often ignore the unique characteristics of 3D point cloud data, treating spatiotemporal redundancy as mere noise, leading to incomplete information extraction. Furthermore, the "Mamba4D: Efficient 4D PointCloud Video Understanding with Disentangled Spatial-Temporal State SpaceModels" method presented at CVPR 2025 also fails to address the issue of static redundancy in point cloud videos. Repeatedly modeling static redundancy not only consumes substantial storage and computational resources but can also cause overfitting during model training, hindering the model's learning and recognition of dynamic changes. Therefore, our invention aims to avoid repeatedly modeling these temporally redundant static structures, reducing computational overhead while enabling the network to better focus on spatiotemporal changes in point cloud videos, thereby improving network performance. Summary of the Invention

[0006] This invention aims to propose a point cloud video behavior recognition method based on redundancy perception and decoupled modeling. After constructing a reasonable ordered representation of point clouds, it reduces the interference of static redundant information on the model. While maintaining the integrity of the spatial topology of the point cloud during the processing of static and dynamic information, it avoids repeatedly modeling these temporally redundant static structures, enabling the network to better focus on the spatiotemporal change information in point cloud videos, thereby improving the accuracy and computational efficiency of point cloud video behavior recognition.

[0007] The technical solution of this invention is as follows: A point cloud video behavior recognition method based on redundancy perception and decoupling modeling, using the Mamba4D network as the foundation, constructs a hash grid, mapping disordered 4D point cloud video data to an ordered hash grid through a hash function; calculates the spatiotemporal redundancy of each hash grid using a statistical distribution method based on temporal density and mean square error, obtains local regions of each point cloud through hash search, and calculates the redundancy of each local region; adds geometric information to the point cloud through a geometric affine module; encodes the features obtained through the geometric affine module through a convolutional layer to obtain the aggregated features of each local region, and distinguishes between static and dynamic feature sequences based on spatiotemporal redundancy; inputs the static and dynamic feature sequences into a bidirectional spatiotemporal Mamba network; in the bidirectional spatiotemporal Mamba network, firstly, spatial topological relationships of point cloud frames are constructed through spatial modeling, and redundant static feature sequence repetitive modeling is eliminated in temporal modeling; finally, the obtained output features are sent to an MLP classification layer for behavior recognition and classification.

[0008] The process of constructing the ordered spatiotemporal hash grid is as follows: First, the spatial coordinates and temporal information in the point cloud video are discretized, and a spatiotemporal hash grid structure is constructed. Then, the spatial coordinates and temporal information of each point in the point cloud video are mapped to a unique spatiotemporal hash index.

[0009] The temporal information and spatial coordinates in the 4D point cloud video data are discretized in time and space. For the time dimension, the continuous time frames are divided into multiple fixed time intervals. For the spatial dimension, the 3D space of the point cloud is discretized into a series of spatial voxels of fixed size. Each point cloud data point is mapped to a fixed spatiotemporal region. Based on the spatiotemporal discretization results, a hash grid structure is designed, given the coordinates of a point in the point cloud. For point clouds with spatial dimensions, their spatial coordinates are divided into sizes according to a hash grid. Mapping to a 3D discrete mesh yields the spatial coordinates of a point within the discrete mesh. : The coordinates of a three-dimensional discrete grid are flattened into one-dimensional hash space index values ​​using a spatial hash function: in, ; By adding time information to the index value of the one-dimensional hash space based on the time dimension of the point cloud video, a unique spatiotemporal index is generated.

[0010] The spatiotemporal redundancy is obtained through a redundancy calculation method based on time density and mean square error: The temporal density of a hash grid reflects the invariance of points within a certain region. The temporal density is expressed as the sum of the spatial occupancy values ​​of points in the hash grid along the time dimension. The mean square error is used to describe the distribution of points in the hash grid along the time dimension. For each hash grid, the mean square error of the number of points in it along the time dimension is calculated to measure the dispersion of the time distribution; The spatiotemporal redundancy of each spatiotemporal grid is defined by calculating the time density and mean square error, which comprehensively reflects the redundancy of the grid. The formula for calculating the spatiotemporal redundancy is as follows: in, It is the first Frame number Each hash grid represents the grid occupancy value. Indicates the first Frame number The number of points in a hash grid Indicates the first The mean of each hash grid along the time dimension. To adjust the weights, .

[0011] The geometric affine module constructs point tubes and introduces geometric affines; A point-to-tube structure is used to perform local spatiotemporal partitioning of point cloud video; In the key time frames of 4D point cloud video, anchor points are selected based on the sampling FPS of the farthest point, and related point clouds are aggregated in their temporal and spatial neighborhoods. Points in eight adjacent hash grids are selected to form a point-tube structure with continuous spatiotemporal semantics. Through the geometric affine module, the pre-calculated geometric topology information, including the three-dimensional coordinates of each point, the Euclidean distance between adjacent points, and the relative position between adjacent points and anchor points, is fused with the original features of the point-tube structure.

[0012] For an anchor point Its corresponding point tube and spatiotemporal neighborhood Related; the spatiotemporal neighborhood Centered on the anchor point, it consists of 8 adjacent hash grids. This represents the spatiotemporal offset of a neighboring point relative to the anchor point; The following affine transformation is used: in and For learnable parameters, Represents element-wise multiplication; calculates the Euclidean distance of the transformed vectors. And obtain the coordinates of neighboring points. In the spatiotemporal neighborhood Internally, through a shared MLP aggregation, higher-level spatiotemporal relationship features are formed: Characteristic functions representing points within a spatiotemporal neighborhood Represents geometric affine functions; The aggregated features are obtained based on the convolutional pooling operation, where This represents vector concatenation. Indicates that it is located at The characteristics of the point tube, This indicates a pooling operation.

[0013] The local spatiotemporal features after aggregation of each point tube are processed by distinguishing the features corresponding to different point tubes based on the spatiotemporal redundancy information of each point tube. The local spatiotemporal features after aggregation of point tubes are passed through a skip gate and a threshold is set through a redundancy selection strategy. Features with a spatiotemporal redundancy higher than 0.65 are used as redundant static feature sequences, and features with a spatiotemporal redundancy lower than 0.65 are used as dynamic feature sequences. The resulting redundant static feature sequences are sampled by a sampler, and only one frame of static feature sequence is selected.

[0014] The bidirectional spatiotemporal Mamba network suppresses spatiotemporal redundancy while maintaining spatial topology. For spatial modeling, both the forward and backward scanning processes involve scanning from static feature sequences to dynamic feature sequences. The static feature sequence scanning is completed independently. The spatial scanning process of the bidirectional spatiotemporal Mamba block is equivalent to first completing the bidirectional scanning of the static feature sequence, then scanning forward and backward to the state of the last token as the initial state of the dynamic feature sequence for each frame, and finally performing a bidirectional scanning of the dynamic feature sequence for each frame. After spatial modeling of the point cloud video, temporal modeling is performed. The temporal encoding of the static feature sequence is separated. Since redundant static feature sequences do not change in the time dimension, they are merged and abstracted into a single static feature sequence. The temporal modeling of the point cloud video is divided into dynamic feature sequences and a unique static feature sequence, with the scanning direction shifting from the dynamic feature sequence to the static feature sequence. First, the dynamic feature sequence is scanned bidirectionally. The resulting total state of the dynamic feature sequence contains information about the changes in the dynamic feature sequence in the time dimension. The total state of the dynamic feature sequence is used as the initial state of the static feature sequence, and the static feature sequence is then scanned bidirectionally. Finally, the output features obtained from spatial modeling and temporal modeling are aggregated, passed through a max pooling layer, and then fed into an MLP classification layer for classification.

[0015] The beneficial effects of this invention are as follows: This invention constructs a hash grid to map spatiotemporally disordered points into a temporally ordered spatiotemporal hash grid structure. Through this hash grid, point cloud videos are aligned at the grid level in the temporal dimension, thereby "ordering" the disordered point cloud. Then, by using a designed statistical distribution-based method, the spatiotemporal redundancy of each part of the point cloud is calculated, thereby identifying which static parts are redundant in time.

[0016] In the modeling process, instead of modeling the whole, spatial modeling and temporal modeling were decoupled to avoid repeatedly modeling the temporally redundant static structure. In spatial modeling, both the forward and backward scanning processes were performed from static to dynamic, and the overall state obtained from the static feature sequence was used as the initial state of the dynamic feature sequence. This ensured that the spatial modeling of the dynamic part retained the complete spatial context relationship and avoided the repeated calculation of the static part. In temporal modeling, both the forward and backward scanning processes were performed from dynamic to static, and the static sequences were merged. This process also avoided repeatedly modeling the static redundant part, thus ensuring computational efficiency. Attached Figure Description

[0017] Figure 1 The flowchart shows a point cloud video behavior recognition method based on redundant perception and decoupling modeling. Figure 2 Detailed diagram of the geometric affine module; Figure 3 This is a detailed diagram of a bidirectional spatiotemporal Mamba network. Detailed Implementation

[0018] To address the problems of disordered point cloud data and static redundant modeling in existing point cloud video behavior recognition technologies, this invention proposes a point cloud video behavior recognition method based on redundant perception and decoupled modeling, such as... Figure 1As shown, referencing the Mamba4D network as a foundation, the disordered point cloud video is meshed and transformed into an ordered spatiotemporal hash grid. The spatiotemporal redundancy of each grid is calculated using the spatiotemporal hash grid. Local regions of each point cloud are obtained through hash search, and the spatiotemporal redundancy of each local region is calculated. Then, a geometric affine module adds geometric information to the point cloud to encourage the subsequent bidirectional spatiotemporal Mamba network to learn geometric topological relationships. After encoding by convolutional layers, regions with high redundancy are found according to a redundancy selection strategy. The encoded tokens are then fed into the bidirectional spatiotemporal Mamba network. In the bidirectional spatiotemporal Mamba network, spatial topological relationships of point cloud frames are first constructed through spatial modeling, and then redundant static feature sequences are eliminated through temporal modeling. This method, while maintaining the integrity of the point cloud spatial topology, avoids the repeated modeling of static redundant information in the point cloud video, thereby improving the accuracy and computational efficiency of point cloud video action recognition.

[0019] The specific steps are as follows: (i) Construct a spatiotemporal hash grid to achieve ordered representation of unordered point cloud videos; To address the problem that point cloud videos are disordered in both spatial and temporal dimensions, making direct temporal modeling difficult, this invention first discretizes the spatial coordinates and temporal information in the point cloud video and constructs a spatiotemporal hash grid structure. By mapping the spatial coordinates and temporal index of each point in the point cloud video to a unique spatiotemporal hash index, the originally disordered point cloud data achieves temporal alignment at the grid level, thereby transforming the disordered point cloud video into a structured and ordered spatiotemporal representation.

[0020] Specifically, the temporal and spatial coordinates in the 4D point cloud video are first discretized. For the temporal dimension, consecutive time frames are divided into multiple fixed time intervals; for the spatial dimension, the 3D space of the point cloud is discretized into a series of spatial voxels of fixed size. Through this discretization, each point cloud data point can be mapped to a fixed spatiotemporal region.

[0021] Based on the spatiotemporal discretization results, a hash grid structure is designed, given the coordinates of a point in the point cloud. For space, the spatial coordinates are first divided into dimensions according to a hash grid. Mapping to a discrete grid: Subsequently, the three-dimensional grid coordinates are flattened into one-dimensional hash space index values ​​using a spatial hash function: in, Then, based on the time dimension of the point cloud video, time information is added to the one-dimensional hash space index value to generate a unique spatiotemporal index. This allows all points to be mapped to a specific hash grid location. In this process, point cloud points within the same spatiotemporal region are mapped to the same hash grid. This mapping method transforms the disorder of point cloud data into a structured storage format, making it easy to find all points within the same spatiotemporal region for each hash grid.

[0022] (ii) Calculate the spatiotemporal redundancy of each part based on the spatiotemporal hash grid; By constructing a spatiotemporal hash grid, points from 4D point cloud videos can be mapped onto an ordered spatiotemporal grid. Based on this, a redundancy calculation method based on temporal density and mean square error is proposed to identify redundant information within the grid. The temporal density of the grid reflects the invariance of points within a certain region; here, the temporal density is expressed as the sum of the spatial occupancy values ​​of the point cloud hash grid along the time dimension. The mean square error describes the distribution of points within the grid along the time dimension. For each grid, the mean square error of the number of points within it along the time dimension is calculated to measure the dispersion of the temporal distribution: the higher the temporal density and the smaller the variance, the more concentrated the points in the grid are in temporal distribution, and the greater the possibility of redundancy.

[0023] By calculating time density and time variance, the spatiotemporal redundancy of each spatiotemporal grid can be defined to comprehensively reflect the redundancy of the grid. The formula for calculating spatiotemporal redundancy is: in, It is the first Frame number Each hash grid represents the grid occupancy value. Indicates the first Frame number The number of points in a hash grid Indicates the first The mean of each hash grid along the time dimension. To adjust the weight Regions with high spatiotemporal redundancy often exhibit high density and low variance. Therefore, the normalized difference between density and variance is used as the spatiotemporal redundancy.

[0024] (iii) Construct point tubes and introduce geometric embedding to realize local spatiotemporal feature encoding; Based on redundancy calculation, a point-tube structure is used to perform local spatiotemporal partitioning of point cloud videos. Anchor points are selected in key time frames, and related point clouds are aggregated within their temporal and spatial neighborhoods to form a point-tube structure with continuous spatiotemporal semantics, used for extracting local spatiotemporal features. Since the static and dynamic parts of the point cloud are modeled separately, and the distribution and behavioral characteristics of dynamic and static points differ significantly in space, the ability to capture the complete geometric topology of the point cloud is required. Therefore, existing embedding methods that rely solely on relative coordinate differences and positional encoding are insufficient to fully express the complex relationships between these two types of points. To address this, more geometric topological information is added to the network input layer, such as the 3D coordinates of each point, the Euclidean distance between adjacent points, and the relative positions between points, allowing the network to autonomously learn this geometric embedding relationship. This topological information not only helps the network better understand the spatial relationships of points within local neighborhoods but also enhances its global grasp of the entire point cloud structure, especially in the interactive modeling of dynamic and static points. Furthermore, the introduction of these geometric topological features enables the network to capture subtle geometric changes and motion patterns in point clouds more precisely, thereby enhancing its expressive power and generalization performance in complex scenes. In this way, the network can more efficiently identify dynamic and static patterns in point cloud videos and accurately model their geometric topological structures.

[0025] Specifically, we transform location information, coordinate differences, and related geometric quantities into low-level spatiotemporal geometric features, and then embed them directly using an MLP, such as for an anchor point. Its corresponding point tube and spatiotemporal neighborhood Related, among which This represents the spatiotemporal offset of a neighboring point relative to the anchor point, such as Figure 2 As shown.

[0026] To avoid overlooking subtle relative positional relationships, the following affine transformation is used: in and For learnable parameters, This represents the Hadamard (element-wise) product. Then, the Euclidean distance between the vectors after the affine transformation is calculated. And obtain the coordinates of neighboring points. Ultimately, in the spatiotemporal neighborhood... Within this framework, these low-level features are aggregated through a shared MLP to form higher-level spatiotemporal relationship features: Then, the aggregated point-tube features are obtained through operations such as convolution and pooling. This represents vector concatenation. Indicates that it is located at The characteristics of the point tube, This indicates a pooling operation.

[0027] (iv) Based on the modeling method of redundant perception and decoupled spatiotemporal, temporal redundancy is suppressed while maintaining spatial topological relationships; In spatiotemporal scanning, the encoding of static components is separated to avoid redundant modeling. However, directly stripping the static components from a coupled spatiotemporal model inevitably leads to the destruction of spatial relationships. Therefore, we decouple spatial and temporal modeling in the early stages of the network, using a parallel SSM structure to model the space of the point cloud video first. After completing the local spatiotemporal feature encoding, this invention distinguishes the features corresponding to different point tubes based on the redundancy information calculated above. We pass the obtained point tube feature sequences through a skip gate and, through a selection strategy, divide them into low-redundancy motion sequence parts and high-redundancy static sequence parts according to their redundancy levels. Here, a top-k strategy is used for selection. To prevent repeated modeling of the static parts, the static sequences are sampled again by a selector to obtain only one frame of static sequence. These sequences are then fed into the subsequent spatiotemporally decoupled model.

[0028] For the subsequent spatiotemporal decoupling model, a bidirectional spatiotemporal Mamba network is chosen, such as... Figure 3 As shown.

[0029] This invention proposes a bidirectional p4dmamba block. For the spatial modeling part, both the forward and backward scanning processes scan from the static part to the dynamic part. At this time, the scanning of the static part can be completed independently. The spatial scanning process of the bidirectional p4dmamba block can be equivalent to first completing the bidirectional scanning of the static part, scanning forward and backward to the state of the last token as the initial state for the dynamic part modeling, and finally performing a bidirectional scanning of the dynamic part for each frame. In this process, the spatial modeling of the dynamic part retains the complete spatial context relationship and avoids the repeated calculation of the static part.

[0030] After completing the spatial modeling of the point cloud video, temporal modeling is performed. To avoid redundant modeling in the temporal dimension, the temporal encoding of static regions is separated. The redundant static parts do not change in the temporal dimension and are still abstracted as a static feature sequence. At this point, the temporal modeling of the point cloud video is divided into two parts: a dynamic feature sequence and a unique static feature sequence. The scanning direction is from the dynamic feature sequence to the static feature sequence. The total state of the dynamic feature sequence obtained from the spatial modeling above contains the change information of the dynamic feature sequence in the temporal dimension. Therefore, the total state of the dynamic feature sequence is used as the initial state of the static feature sequence for bidirectional scanning of the static part. The merging process of the static redundant parts during the temporal modeling process avoids its repeated scanning in the temporal dimension. Finally, the obtained output features are fed into the MLP layer for behavior recognition and classification.

[0031] Compared with existing technologies, this invention conducted relevant experiments on three datasets widely used for point cloud video action recognition: NTU RGB+D 60, NTU RGB+D 120, and MSR-Action3D. The specific experimental results are shown in Table 1 and Table 2.

[0032] Table 1. 3D action recognition accuracy (%) on NTU RGB+D 60 and NTU RGB+D 120 datasets Table 2. 3D motion recognition accuracy (%) at different frame rate settings on the MSR-Action3D dataset. As shown in Table 1, on the widely used NTU RGB+D 60 and NTU RGB+D 120 datasets, our method P4DMamba achieved the highest accuracy in three of the four evaluation scenarios. Particularly noteworthy is that P4DMamba significantly outperforms state-of-the-art methods in cross-subject evaluation (CS), which typically has the lowest accuracy. This advantage is particularly evident in cross-subject evaluation on the widely adopted NTU RGB+D 60 dataset: P4DMamba's accuracy is 1.0% higher than the second-ranked Kinect. It is also worth mentioning that even when trained from scratch (without pre-training), our method P4DMamba outperforms all methods that have undergone self-supervised pre-training.

[0033] The proposed method was further evaluated on the MSR-Action3D dataset, and the comparison results are shown in Table 2. P4DMamba consistently achieved the highest accuracy across all frame count settings, especially in the most frequently compared 24-frame count setting, where P4DMamba's accuracy was 0.6% higher than the second-highest pre-trained PST-Transformer-based method. Notably, Mamba4D's accuracy was even lower than earlier Transformer-based methods. In comparison, the proposed P4DMamba is the first Mamba architecture to outperform state-of-the-art CNN and Transformer architectures in terms of accuracy.

[0034] Meanwhile, we also tested the efficiency of our method on the NTU 60 dataset, which is most commonly used for point cloud video action recognition. The results are shown in Table 3 below. It can be seen that the memory usage of the method of the present invention is significantly reduced, while the average inference speed of a single video is 6 times that of Transformer and 3 times that of the baseline model Mamba4D while improving accuracy. In addition, the speed of Tiny model is even dozens of times faster than previous work. It can be seen that the model method of the present invention has greatly improved efficiency, memory usage and performance compared with the previous best method.

[0035] Table 3 Efficiency comparison on the NTU RGB+D 60 dataset

Claims

1. A point cloud video behavior recognition method based on redundant perception and decoupling modeling, characterized in that, Based on the Mamba4D network, a hash grid is constructed to map unordered 4D point cloud video data into an ordered hash grid using a hash function. The spatiotemporal redundancy of each hash grid is calculated using a statistical distribution method based on temporal density and mean square error. Local regions of each point cloud are obtained through hash search, and the spatiotemporal redundancy of each local region is calculated. Geometric information is added to the point cloud through a geometric affine module. The features obtained from the geometric affine module are encoded through convolutional layers to obtain aggregated features for each local region. The aggregated features are distinguished into static and dynamic feature sequences based on spatiotemporal redundancy. The static and dynamic feature sequences are input into a bidirectional spatiotemporal Mamba network. In the bidirectional spatiotemporal Mamba network, spatial topology relationships of point cloud frames are first constructed through spatial modeling, and redundant static feature sequence modeling is eliminated during temporal modeling. Finally, the output features are fed into an MLP classification layer for behavior recognition and classification.

2. The point cloud video behavior recognition method based on redundancy perception and decoupling modeling according to claim 1, characterized in that, The process of constructing the ordered spatiotemporal hash grid is as follows: First, the spatial coordinates and temporal information in the point cloud video are discretized, and a spatiotemporal hash grid structure is constructed. Then, the spatial coordinates and temporal information of each point in the point cloud video are mapped to a unique spatiotemporal hash index.

3. The point cloud video behavior recognition method based on redundancy perception and decoupling modeling according to claim 2, characterized in that, The temporal information and spatial coordinates in the 4D point cloud video data are discretized in a spatiotemporal manner. In terms of time dimension, continuous time frames are divided into multiple fixed time intervals. In terms of spatial dimension, the 3D space of the point cloud is discretized into a series of spatial voxels of fixed size. Each point cloud data point is mapped to a fixed spatiotemporal region. Based on the spatiotemporal discretization results, a hash grid structure is designed, given the coordinates of a point in the point cloud. For point clouds with spatial dimensions, their spatial coordinates are divided into sizes according to a hash grid. Mapping to a 3D discrete mesh yields the spatial coordinates of a point within the discrete mesh. : ； The coordinates of a three-dimensional discrete grid are flattened into one-dimensional hash space index values ​​using a spatial hash function: ； in, ; By adding time information to the index value of the one-dimensional hash space based on the time dimension of the point cloud video, a unique spatiotemporal index is generated.

4. The point cloud video behavior recognition method based on redundancy perception and decoupling modeling according to claim 1, characterized in that, The spatiotemporal redundancy is obtained through a redundancy calculation method based on time density and mean square error: The temporal density of a hash grid reflects the invariance of points within a certain region. The temporal density is expressed as the sum of the spatial occupancy values ​​of points in the hash grid along the time dimension. The mean square error is used to describe the distribution of points in the hash grid along the time dimension. For each hash grid, the mean square error of the number of points in it along the time dimension is calculated to measure the dispersion of the time distribution; The spatiotemporal redundancy of each spatiotemporal grid is defined by calculating the time density and mean square error, which comprehensively reflects the redundancy of the grid. The formula for calculating the spatiotemporal redundancy is as follows: ； in, It is the first Frame number Each hash grid represents the grid occupancy value. Indicates the first Frame number The number of points in a hash grid Indicates the first The mean of each hash grid along the time dimension. To adjust the weights, .

5. The point cloud video behavior recognition method based on redundancy perception according to claim 1, characterized in that, The geometric affine module constructs point tubes and introduces geometric affines; A point-to-tube structure is used to perform local spatiotemporal partitioning of point cloud video; In the key time frames of 4D point cloud video, anchor points are selected based on the sampling FPS of the farthest point, and related point clouds are aggregated in their temporal and spatial neighborhoods. Points from eight adjacent hash grids are selected to form a point-tube structure with continuous spatiotemporal semantics. The pre-calculated geometric topology information, including the three-dimensional coordinates of each point, the Euclidean distance between adjacent points, and the relative position between neighboring points and anchor points, is fused with the original features of the point-tube structure through the geometric affine module.

6. The point cloud video behavior recognition method based on redundancy perception according to claim 5, characterized in that, For an anchor point Its corresponding point tube and spatiotemporal neighborhood Related; the spatiotemporal neighborhood Centered on the anchor point, it consists of 8 adjacent hash grids. This represents the spatiotemporal offset of a neighboring point relative to the anchor point; The following affine transformation is used: ； in and For learnable parameters, Represents element-wise product; Calculate the Euclidean distance of the transformed vectors. And obtain the coordinates of neighboring points. In the spatiotemporal neighborhood Internally, through a shared MLP aggregation, higher-level spatiotemporal relationship features are formed: ； ； Characteristic functions representing points within a spatiotemporal neighborhood Represents geometric affine functions; The aggregated features are obtained based on the convolutional pooling operation, where This represents vector concatenation. Indicates that it is located at The characteristics of the point tube, This indicates a pooling operation.

7. The point cloud video behavior recognition method based on redundancy perception and decoupling modeling according to claim 6, characterized in that, The local spatiotemporal features after aggregation of each point tube are processed by distinguishing the features corresponding to different point tubes based on the spatiotemporal redundancy information of each point tube. The local spatiotemporal features after aggregation of point tubes are passed through a skip gate and a threshold is set through a redundancy selection strategy. Features with a spatiotemporal redundancy higher than 0.65 are used as redundant static feature sequences, and features with a spatiotemporal redundancy lower than 0.65 are used as dynamic feature sequences. The resulting redundant static feature sequences are sampled by a sampler, and only one frame of static feature sequence is selected.

8. The point cloud video behavior recognition method based on redundancy perception and decoupling modeling according to claim 1, characterized in that, The bidirectional spatiotemporal Mamba network suppresses spatiotemporal redundancy while maintaining spatial topology. For spatial modeling, both the forward and backward scanning processes involve scanning from static feature sequences to dynamic feature sequences; The static feature sequence scanning is completed independently. The spatial scanning process of the bidirectional spatiotemporal Mamba block is equivalent to first completing the bidirectional scanning of the static feature sequence, scanning forward and backward to the state of the last token as the initial state of the dynamic feature sequence of each frame, and finally performing a bidirectional scanning of the dynamic feature sequence of each frame. After spatial modeling of the point cloud video, temporal modeling is performed. The temporal encoding of the static feature sequence is separated. Since redundant static feature sequences do not change in the time dimension, they are merged and abstracted into a single static feature sequence. The temporal modeling of the point cloud video is divided into dynamic feature sequences and a unique static feature sequence, with the scanning direction shifting from the dynamic feature sequence to the static feature sequence. First, the dynamic feature sequence is scanned bidirectionally. The resulting total state of the dynamic feature sequence contains information about the changes in the dynamic feature sequence in the time dimension. The total state of the dynamic feature sequence is used as the initial state of the static feature sequence, and the static feature sequence is then scanned bidirectionally. Finally, the output features obtained from spatial modeling and temporal modeling are aggregated, passed through a max pooling layer, and then fed into an MLP classification layer for classification.

Images