Skeleton motion recognition method and system based on kinematic gaussian splashing and probability topology

Abstract

The application provides a skeleton action recognition method based on kinematic Gaussian splatting and probability topology, and the method comprises the following steps: constructing a graph convolution network model based on kinematic Gaussian splatting and probability topology; using a Gaussian splatting strategy to calculate joint instantaneous velocity vectors and dynamically construct an anisotropic covariance matrix, rendering sparse discrete skeleton sequences into multi-view continuous heat maps; using a probability topology construction strategy to model joints as Gaussian distributions, generating an adaptive prior adjacency matrix by calculating the Bhattacharyya distance between joint distributions, and capturing potential long-range dependencies; through a visual context gating mechanism, adaptively injecting visual features generated by rendering into a GCN backbone network, modulating skeleton feature propagation, and training the model using a joint loss function to achieve robust skeleton action recognition. The application significantly enhances the modeling capability of the model for complex spatio-temporal dynamics, and solves the problems of fine-grained information loss and topology rigidity of sparse sensor data.

Description

Technical Field

[0001] This invention belongs to the fields of computer vision, pattern recognition and artificial intelligence technology, and relates to a skeleton action recognition method and system based on kinematic Gaussian splashing and probabilistic topology. Background Technology

[0002] Human motion recognition is a core task in applications such as human-computer interaction, intelligent monitoring, and immersive media. Skeleton data, due to its compact structure, robustness to changes in lighting and background, and privacy protection, has become an important modality for motion recognition. Traditional skeleton motion recognition methods (such as ST-GCN and its variants) primarily utilize graph convolutional networks (GCNs) to extract geometric features. However, existing technologies face two major bottlenecks: One issue is the loss of fine-grained motion information. Existing methods typically treat joints as discrete point coordinates, ignoring the motion ambiguity effect caused by velocity changes. For rapid or explosive movements, instantaneous velocity, direction, and momentum are key discriminative factors, and relying solely on sparse coordinates is insufficient to effectively encode this information.

[0003] On the other hand, there is the issue of rigid topological structures. While predefined physical skeleton maps can capture local structures, they limit the capture of potential long-range dependencies between non-physically connected joints. Although some methods introduce adaptive topological learning, their edge weights are usually implicitly learned, lacking explicit statistical meaning and interpretability, and are prone to forgetting or instability of topological information. Therefore, how to construct topological priors that are both statistically meaningful and interpretable while enhancing the ability to represent continuous motion is a pressing technical problem that needs to be solved in current skeleton motion recognition. Summary of the Invention

[0004] In view of this, the purpose of this invention is to provide a skeleton motion recognition method and system based on kinematic Gaussian splashing and probabilistic topology, which aims to solve the problems of sparsity and topological rigidity of sensor data.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A skeleton action recognition method based on kinematic Gaussian splashing and probabilistic topology, the method specifically includes the following steps: S1. Construct a skeleton action recognition network model based on kinematic Gaussian splash and probabilistic topology: Build a deep neural network model including a kinematic Gaussian splash module (KGSM), a probabilistic topology construction module, a visual context gating module (VCG), and a spatiotemporal graph convolutional ST-GCN backbone network; determine the preprocessing method of input data, the covariance construction rule of Gaussian splash, the statistical measurement method of probabilistic topology, and the fusion mechanism of visual features and skeleton features; S2. Perform kinematically driven Gaussian splashing: Normalize the input skeleton sequence and calculate the instantaneous velocity vector of each joint; dynamically construct an anisotropic covariance matrix based on the instantaneous velocity vector, and convert the joints from discrete point coordinates into a two-dimensional Gaussian distribution with directional and scale variations; project the three-dimensional skeleton onto multiple orthogonal planes, and use Gaussian splashing technology to render and generate a multi-view continuous heatmap containing spatiotemporal semantics. S3. Construct a probabilistic topology based on statistical distance: Treat each joint as a probability distribution, calculate the Bhattacharyya distance between any two joint distributions to measure their statistical correlation; generate an adaptive prior adjacency matrix based on the Bhattacharyya distance, and inject it as prior knowledge into the graph convolutional network to supplement the physical connection graph and capture potential long-range dependencies between joints. S4. Network training and action recognition based on visual context gating: The encoder extracts the visual semantic features of the continuous heatmap generated in step S2; through the visual context gating mechanism, the visual semantic features are used to adaptively modulate the skeleton features in the GCN backbone network layer by layer; a joint loss function containing classification cross-entropy loss and topology consistency regularization term is constructed to train the model end-to-end and output the action recognition results.

[0006] Furthermore, in step S2, the dynamic anisotropic covariance matrix The mathematical expression of the construction process is as follows: The covariance matrix is ​​composed of a rotation matrix. and scaling matrix The decision is made jointly, and the formula is as follows:

[0007] Wherein, scaling matrix Scale along the direction of motion Modulus with velocity Increase and stretch, vertical direction Maintain the baseline scale The formula is:

[0008] in, The stretching factor; rotation matrix From the normalized velocity direction Decide:

[0009] Through the above construction, the static joints exhibit an isotropic circular distribution, while the rapidly moving joints exhibit an elliptical distribution stretched along the motion trajectory.

[0010] Furthermore, in step S2, the process of generating the multi-view continuous heatmap is as follows: Project the 3D skeleton onto Three orthogonal planes; for any pixel on the plane , No. The joint in the first Response intensity of the frame Follows a multivariate Gaussian distribution:

[0011] in, The center position of the joint in the image coordinate system; final heatmap Aggregation of responses from all joints.

[0012] Furthermore, in step S3, the construction process of the probabilistic topology is as follows: Calculate joints using statistical parameters generated by KGSM. and joints The distance between the two Bachs :

[0013] The first term measures the spatial Euclidean distance between joints, while the second term measures the difference between the two distribution shapes. It can be represented as:

[0014] An adaptive prior adjacency matrix is ​​constructed based on Bach distance. Its formula is expressed as:

[0015] The The matrix is ​​used to initialize or weight the adjacency matrix in the GCN.

[0016] Furthermore, in step S4, the feature fusion method of the visual context gating mechanism (VCG) is as follows: Let the skeleton features of the GCN layer be... Visual context features are Modulation coefficients are generated through a nonlinear gated network. :

[0017] in, It is the Sigmoid activation function. For learnable projective weights; Final fusion features The calculation formula is:

[0018] in, This indicates element-wise multiplication, achieving complementarity of multimodal features through residual gating.

[0019] Furthermore, in step S4, the joint loss function Defined as:

[0020] in, For classification cross-entropy loss; To dynamically adjust the weight coefficients with each training round, a linear warm-up strategy is adopted to gradually increase the constraint strength. This is a topology consistency regularization term used to constrain the topology learned by the network. Approximation probability topological prior The formula is: .

[0021] This invention also provides a skeleton action recognition system based on kinematic Gaussian splashing and probabilistic topology to implement the method described above. The system includes: a network construction module for executing step S1, constructing a KGS-GCN network model containing KGSM, a probabilistic topology module, and a VCG mechanism; a kinematic rendering module for executing step S2, extracting skeleton velocity vectors, constructing a dynamic covariance matrix, and performing Gaussian splashing rendering to generate a multi-view heatmap; a topology construction module for executing step S3, generating a probabilistic topology prior matrix based on the statistical correlation between joint distributions using Bhattacharyya distance; and a training and recognition module for executing step S4, fusing features through visual gating, optimizing model parameters using a joint loss function, and predicting the action category of the input skeleton sequence.

[0022] The beneficial effects of this invention are as follows: (1) Enhanced motion representation: Through kinematically driven Gaussian splashing, velocity and direction information are explicitly transformed into visual texture (anisotropic heatmap), which effectively solves the problem of losing fine-grained motion cues in discrete coordinates.

[0023] (2) Interpretable topology construction: By introducing the Bach distance to measure the statistical distance between joint distributions, a probabilistic topology with clear physical and statistical significance is constructed, which effectively captures potential long-range dependencies.

[0024] (3) Multimodal collaborative modeling: Through the visual context gating mechanism, the deep fusion and adaptive modulation of low-level kinematic features (visual heatmap) and high-level geometric features (skeleton map) are realized, which significantly improves the accuracy and robustness of recognition.

[0025] (4) High efficiency: While ensuring performance that surpasses the state-of-the-art methods, it maintains a low number of parameters and computational complexity, making it suitable for practical deployment.

[0026] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0027] To make the objectives, technical solutions, and advantages of the present invention clearer, the preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings, wherein: Figure 1 This is a schematic diagram of the overall framework of the KGS-GCN invention. Detailed Implementation

[0028] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Unless otherwise specified, the following embodiments and features can be combined with each other.

[0029] The accompanying drawings are for illustrative purposes only and are schematic diagrams, not actual pictures. They should not be construed as limiting the invention. To better illustrate the embodiments of the invention, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual product dimensions. It is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0030] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components. In the description of the present invention, it should be understood that if terms such as "upper," "lower," "left," "right," "front," and "rear" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, they are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, the terms used to describe positional relationships in the drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those skilled in the art, the specific meaning of the above terms can be understood according to the specific circumstances.

[0031] This invention provides a skeleton motion recognition method and system based on kinematic Gaussian splashing and probabilistic topology. Figure 1 This is a schematic diagram of the overall framework of the KGS-GCN invention, showing the complete process from skeleton input to kinematic feature extraction, Gaussian splash rendering, probabilistic topology construction, and visual gating fusion.

[0032] Example 1 In this embodiment, a skeleton action recognition method based on kinematic Gaussian splashing and probabilistic topology is provided, and the specific process is as follows: S1: Model Construction The KGS-GCN network is constructed, with the backbone network employing multi-layered stacked spatiotemporal graph convolutional modules (ST-Block). The model input is... The skeleton sequence tensor.

[0033] S2: Kinematically Driven Gaussian Splash (KGSM) Data preprocessing: Calculating instantaneous joint velocities And normalize the skeleton coordinates.

[0034] Dynamic covariance construction: To simulate motion blur, the Gaussian kernel shape is adjusted according to velocity. A scaling matrix is ​​then constructed. The scale along the direction of motion According to the formula Stretch ( The vertical direction maintains the baseline scale. Rotation matrix. Determined by the direction of velocity. Final covariance matrix. .

[0035] Multi-view rendering: Projecting the skeleton onto A heatmap is generated using the Gaussian formula on a plane, and the responses of all joints are aggregated to obtain a continuous visual representation. .

[0036] S3: Probabilistic Topology Construction Statistical parameters (mean) generated using KGSM Covariance ), calculate the Bacon distance between joint pairs This distance comprehensively considers the spatial Euclidean distance and distribution shape differences of the joints. Based on Calculate the prior adjacency matrix It is used to capture motion correlations between non-physically connected joints.

[0037] S4: Visual Gating and Training Visual Encoding: Using Lightweight CNNs for Heatmaps Encode and extract visual features .

[0038] Visual Context Gating (VCG): In the GCN layer, utilizing... Generate gating coefficients Through formula Modulate the skeleton features.

[0039] Loss function: Joint loss is used. .in, The topological approximation probability and topological prior learned by the constrained network The training strategy employs the SGD optimizer, combined with warm-up and dynamic weight adjustment.

[0040] Experiments on benchmark datasets such as NTU RGB+D and Penn Action have verified that the method proposed in this invention achieves an advanced level of recognition accuracy (e.g., 99.5% on Penn Action) with only 1.4M parameters, demonstrating its effectiveness and robustness.

[0041] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A skeleton action recognition method based on kinematic Gaussian splashing and probabilistic topology, characterized in that, The method specifically includes the following steps: S1. Construct a skeleton action recognition network model based on kinematic Gaussian splash and probabilistic topology: Build a deep neural network model including a kinematic Gaussian splash module (KGSM), a probabilistic topology construction module, a visual context gating module (VCG), and a spatiotemporal graph convolutional ST-GCN backbone network; determine the preprocessing method of input data, the covariance construction rule of Gaussian splash, the statistical measurement method of probabilistic topology, and the fusion mechanism of visual features and skeleton features; S2. Perform kinematically driven Gaussian splashing: Normalize the input skeleton sequence and calculate the instantaneous velocity vector of each joint; dynamically construct an anisotropic covariance matrix based on the instantaneous velocity vector, and convert the joints from discrete point coordinates into a two-dimensional Gaussian distribution with directional and scale variations; project the three-dimensional skeleton onto multiple orthogonal planes, and use Gaussian splashing technology to render and generate a multi-view continuous heatmap containing spatiotemporal semantics. S3. Construct a probabilistic topology based on statistical distance: Treat each joint as a probability distribution, calculate the Bach distance between any two joint distributions to measure their statistical correlation; generate an adaptive prior adjacency matrix based on the Bach distance, and inject it as prior knowledge into the graph convolutional network to supplement the physical connection graph and capture potential long-range dependencies between joints. S4. Network training and action recognition based on visual context gating: The encoder extracts the visual semantic features of the continuous heatmap generated in step S2; through the visual context gating mechanism, the visual semantic features are used to adaptively modulate the skeleton features in the GCN backbone network layer by layer; a joint loss function containing classification cross-entropy loss and topology consistency regularization term is constructed to train the model end-to-end and output the action recognition results.

2. The skeleton action recognition method based on kinematic Gaussian splashing and probabilistic topology according to claim 1, characterized in that, In step S2, the anisotropic covariance matrix The mathematical expression of the construction process is as follows: The covariance matrix is ​​composed of a rotation matrix. and scaling matrix The decision is made jointly, and the formula is as follows: Wherein, scaling matrix Scale along the direction of motion Modulus with velocity Increase and stretch, vertical direction Maintain the baseline scale The formula is: in, The stretching factor; rotation matrix From the normalized velocity direction Decide: Through the above construction, the static joints exhibit an isotropic circular distribution, while the rapidly moving joints exhibit an elliptical distribution stretched along the motion trajectory.

3. The skeleton action recognition method based on kinematic Gaussian splashing and probabilistic topology according to claim 2, characterized in that, In step S2, the process of generating the multi-view continuous heatmap is as follows: Project the 3D skeleton onto Three orthogonal planes; for any pixel on the plane , No. The joint in the first Response intensity of the frame Follows a multivariate Gaussian distribution: in, The center position of the joint in the image coordinate system; final heatmap Aggregation of responses from all joints.

4. The skeleton action recognition method based on kinematic Gaussian splashing and probabilistic topology according to claim 3, characterized in that, In step S3, the construction process of the probabilistic topology is as follows: Calculate joints using statistical parameters generated by KGSM. and joints The distance between the two Bachs : The first term measures the spatial Euclidean distance between joints, while the second term measures the difference between the two distribution shapes. Represented as: An adaptive prior adjacency matrix is ​​constructed based on Bach distance. Its formula is expressed as: The The matrix is ​​used to initialize or weight the adjacency matrix in the GCN.

5. The skeleton action recognition method based on kinematic Gaussian splashing and probabilistic topology according to claim 4, characterized in that, In step S4, the feature fusion method of the visual context gating mechanism (VCG) is as follows: Let the skeleton features of the GCN layer be... Visual context features are Modulation coefficients are generated through a nonlinear gated network. : in, It is the Sigmoid activation function. For learnable projective weights; Final fusion features The calculation formula is: in, This indicates element-wise multiplication, achieving complementarity of multimodal features through residual gating.

6. The skeleton action recognition method based on kinematic Gaussian splashing and probabilistic topology according to claim 5, characterized in that, In step S4, the joint loss function Defined as: in, For classification cross-entropy loss; To dynamically adjust the weight coefficients with each training round, a linear warm-up strategy is adopted to gradually increase the constraint strength. This is a topology consistency regularization term used to constrain the topology learned by the network. Approximation probability topological prior The formula is: 。 7. A skeleton motion recognition system based on kinematic Gaussian splashing and probabilistic topology, characterized in that, To implement the method of any one of claims 1-6, the system comprises: a network construction module for performing step S1, constructing a KGS-GCN network model including KGSM, a probabilistic topology module, and a VCG mechanism; a kinematic rendering module for performing step S2, extracting skeleton velocity vectors, constructing a dynamic covariance matrix, and performing Gaussian splash rendering to generate a multi-view heatmap; a topology construction module for performing step S3, generating a probabilistic topology prior matrix based on the statistical correlation between joint distributions measured by Bhattacharyya distance; and a training and recognition module for performing step S4, fusing features through visual gating, optimizing model parameters using a joint loss function, and predicting the action category of the input skeleton sequence.

Images