A skeleton action recognition method based on explicit and implicit collaborative graph convolution network
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HOHAI UNIV
- Filing Date
- 2026-02-05
- Publication Date
- 2026-06-09
Smart Images
Figure CN122176789A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of human behavior recognition technology, and in particular to a skeletal action recognition method based on explicit-implicit cooperative graph convolutional networks. Background Technology
[0002] Human motion recognition has become a key research problem in the field of game intelligence, supporting the realization of various core functions. Skeleton data typically represents human actions in the form of keypoint sequences, which can be naturally modeled as a spatiotemporal graph structure that evolves over time. The core of efficient motion recognition lies in simultaneously modeling the spatial dependencies between joints and their dynamic evolution over time. Early skeleton motion recognition methods often treated joints as independent feature vectors and manually organized them into coordinate sequences or pseudo-images as input to RNNs or CNNs to complete motion classification. However, these methods failed to explicitly characterize the geometric configuration relationships between different joints.
[0003] In recent years, Graph Convolutional Networks (GCNs) have extended convolutional operations from regular grid structures to graph structures, achieving significant success in multiple fields. ST-GCN was the first to utilize graph structures to model inter-joint relationships and combined GCN with temporal convolution to extract spatiotemporal motion features from skeleton sequences. More recent models, CTR-GCN and InfoGCN, further introduced adaptive graph learning mechanisms, enabling the network to dynamically adjust topological connections during training, thereby enhancing the model's ability to perceive structural differences across different action categories.
[0004] However, existing skeletal motion recognition methods primarily rely on modeling the explicit geometry of the human body, utilizing the natural topological structure between joints to resolve static spatial relationships formed by anatomical connections. While such methods offer strong interpretability in characterizing observable structural features within the skeleton, they often overlook implicit associated movements arising from non-naturally connected joints during motion execution. These implicit movements, however, can reflect deeper dynamic coupling relationships within the motion, revealing potential cross-site dependencies between different body parts at the semantic level of motion. Summary of the Invention
[0005] To address the above technical problems, this invention provides a skeletal action recognition method based on explicit-latentative cooperative graph convolutional networks, comprising the following steps: S1. Obtain skeletal data, i.e. skeleton action sequences, from the skeletal dataset and construct an explicit-implicit co-operational graph convolutional network; S2. Construct an explicit geometric configuration module in an explicit-implicit co-electronic graph convolutional network. Input the skeletal data into the explicit geometric configuration module. Based on the inherent topology of the human skeleton and combined with a bidirectional time extension mechanism, complete the explicit geometric modeling of the action in Euclidean space. S3. Construct a latent association motion module in the explicit-latent collaborative graph convolutional network. In the latent association motion module, the input skeletal data is mapped to a hypergraph representation and projected onto the Fourier domain. The implicit association between non-adjacent joints over time is learned through Fourier spectrum convolution. S4. Construct a deep information aggregation module in the explicit-implicit co-convolutional graph convolutional network. In the deep information aggregation module, perform deep aggregation on the features extracted by the explicit geometric configuration module and the implicit associated motion module, and dynamically balance the contribution weights of different feature flows through an adaptive feature recalibration mechanism. S5. Input the spatiotemporal feature representation obtained in step S4 into the global average pooling layer and the fully connected layer to complete the human skeleton action recognition.
[0006] The technical solution further defined in this invention is: Furthermore, in step S1, the input skeleton action sequence is represented as... ,in T represents the frame number. Let N represent a real number, C represent the number of joints, and C represent the feature dimension of each joint. The explicit-implicit co-graph convolutional network consists of 10 explicit-implicit co-graph convolutional layers.
[0007] As described above, the skeletal action recognition method based on explicit-implicit cooperative graph convolutional networks includes the following sub-steps in step S2: S2.1 Spatial modeling based on hierarchical Euclidean graph constraints is used to capture geometric changes in motion; S2.2. Time modeling is performed by designing a bidirectional time extension mechanism to enhance the ability to represent the differentiated characteristics of different body parts.
[0008] As described above, a skeletal action recognition method based on explicit and implicit collaborative graph convolutional networks is used. In step S2.1, the human skeleton is first formalized as a graph G=(V,E), where the set of nodes V corresponds to the coordinates of each joint, and the set of edges E follows the skeletal connection relationship defined by human anatomy, thus forming a topological graph with prior constraints. Subsequently, the static Euclidean spatial relationships between joints are encoded using a graph convolution operator, assuming the features of the l-th layer are... ,in Let N represent the number of real numbers, d represent the number of joints, and d represent the feature dimension. Convolutional update is represented as:
[0009] in, This indicates that A is a subset of the human skeleton, defined by its inherent topological structure. Each of the different levels of adjacency matrices Corresponding to spatially dependent structures at different scales; elements of matrix A Indicates joint With joints There must be a connection between them, otherwise ; The degree matrix is defined as follows: , that is, the degree of node v, which represents the number of its connected neighbors; This represents an adaptive perturbation matrix that learns to adjust the weights of the adjacency matrix to adapt to geometric changes during the action. This represents a learnable weight matrix used to perform linear transformations on the node features of each layer. This represents a nonlinear activation function used to introduce nonlinear transformations.
[0010] As described above, in a skeletal action recognition method based on explicit-implicit cooperative graph convolutional networks, step S2.2 involves selecting a core frame for a given skeletal input sequence containing T frames. To determine a representative posture of a continuous human motion; using the selected core frame as the center, a bidirectional time-expansion mechanism is employed to expand forward and backward along the time axis, constructing a structure of length 2. The motion segments are analyzed to cover the temporal information before and after the action; then, using a farthest-point sampling strategy, N core joints are selected from each representative pose at a set sampling rate; for each selected core joint, M explicit neighboring joints are selected with a fixed number of hops h to construct the local spatial neighborhood of the core joint; logarithmic spatiotemporal point convolution operation is used to extract fine-grained spatiotemporal motion features in the neighborhood, as shown in the following formula:
[0011]
[0012] in, Let represent the local geometric features of the i-th core joint in frame t. A multilayer perceptron (S-MLP) is constructed for encoding to capture local motion patterns within the explicit neighborhood of this core joint. For each core joint... Its local spatial neighborhood is defined as The selected N core joints are used as the centers for local feature extraction. For each core joint... The selected M dominant adjacent key points form its local spatial neighborhood set. ; It represents the spatial displacement relative to the core joint points in the core frame, and is used to characterize local geometric changes; Subsequently, time series features Through a length of The aggregation is performed within the sliding time window; This represents the set of N core joints selected in frame t. For each core joint k, its corresponding local feature is... The operation applies to the global core node set within the same time window. Max pooling is performed to capture global dynamic dependencies across time; a multilayer perceptron (T-MLP) is constructed to further encode the aggregated features temporally.
[0013] As described above, a skeletal motion recognition method based on explicit-implicit cooperative graph convolutional networks includes the following steps in step S3: S3.1 Map the input skeletal sequence to a hypervariable graph structure; S3.2. Based on the constructed hypervariable graph, perform spectral decomposition to map the skeleton graph signal from Euclidean space to the Fourier spectral domain, and perform learnable spectral convolution operations in the Fourier spectral domain.
[0014] As described above, in a skeletal action recognition method based on explicit and implicit cooperative graph convolutional networks, step S3.1 involves targeting the skeleton sequence... In the hypervariable diagram In, node set This represents the joint trajectory unit spanning the entire sequence, i.e., each node. For the i-th joint in the t-th frame, the size of the node set expands to N×T; Represents edge set, superedge Simultaneously connecting multiple joints in different frames, the hypervariable graph uses its correlation matrix... express, This indicates the number of super edges; if the super node... Belongs to superedge ,but Otherwise .
[0015] As described above, in a skeletal action recognition method based on explicit-implicit collaborative graph convolutional networks, step S3.2 defines the hypergraph Laplacian operator as follows:
[0016] in, This represents the correlation matrix between nodes and hyperedges in the hypervariable graph; Let be the weight matrix of the hyperedges, representing the strength of each hyperedge; Let be the degree matrix of the nodes, representing the connectivity of the nodes; Let be the degree matrix of the hyperedge, representing the connectivity of the hyperedge.
[0017] As described above, in a skeletal motion recognition method based on explicit-latent cooperative graph convolutional networks, step S3.2 involves projecting the skeleton graph signal into the Fourier space and applying Fourier spectral domain convolution to capture high-order dynamic patterns between implicit joints, as follows:
[0018] in, This represents the skeleton node features of the l-th layer after stacking on the hypervariable graph, where N represents the number of joints and T represents the number of frames. Indicates the number of channels; Indicates the updated features; For the supergraph Laplace, Given its Fourier basis and eigenspectrum, the column vectors of U are the Fourier eigenvectors of the hypergraph. For eigenvalues, Used for spectral domain normalization; This represents the element-wise action of the Chebyshev polynomial on the diagonal spectrum. and It is a set of learnable spectral filter coefficients , This is the channel mapping matrix; the values within brackets [] are... and Together they constitute the hypergraph Fourier filter operator .
[0019] As described above, in a skeletal action recognition method based on explicit-implicit cooperative graph convolutional networks, step S4 represents the output of the explicit geometric configuration module as follows: The implicit flow aggregates at the spatiotemporal nodes and is projected back to the node level as follows: The fusion method then employs gated residual fusion:
[0020]
[0021] Where Pool(∙) represents channel pooling or global averaging, Indicates channel splicing. Denotes gated projection, where f(∙) represents a time-linear or nonlinear mapping that matches dimensions. For element-wise gating, The output features are represented by the gating coefficient G, which determines the strength of the compensation of the latent information to the explicit basis. The residual structure ensures the preservation of the explicit geometric prior, while the latent information is injected through a controlled path.
[0022] The beneficial effects of this invention are: (1) In this invention, an explicit geometric configuration module is designed. This module uses the inherent topological graph structure of the human skeleton to model the explicit geometric relationship between joints in Euclidean space. At the same time, a bidirectional time extension mechanism is introduced to enhance the learning of the dynamic features of heterogeneous body parts. This module focuses on capturing the explicit spatial relationship between joints and its temporal evolution, providing the model with an intuitive understanding of the explicit geometric structure of the human skeleton. (2) In this invention, an implicit association motion module is proposed. This module abstracts the spatiotemporal information of the skeleton into a new type of hypergraph representation and projects it into the Fourier domain. Through Fourier spectrum convolution, it directly learns the implicit association patterns that evolve over time between non-naturally connected joints. This module aims to overcome the limitations of Euclidean space modeling by directly learning the intrinsic implicit association patterns that evolve over time between non-naturally connected joints. (3) In this invention, a deep information aggregation module is designed. This module integrates the explicit geometric configuration module and the implicit associated motion module to construct an explicit-implicit co-graph convolutional network, which realizes a holistic and in-depth representation of skeleton dynamics. It achieves 93.6% accuracy across topics and 97.5% accuracy across views on the NTU-RGB+D 60 dataset, 90.2% accuracy across topics and 91.0% accuracy across views on the NTU-RGB+D 120 dataset, and 97.8% accuracy on the MSR-Action3D dataset, which is significantly better than the methods of the prior art. Attached Figure Description
[0023] Figure 1 This is a schematic diagram of the overall process of the present invention; Figure 2 Figure 1 is a schematic diagram of the structure of the explicit-implicit cooperative graph convolutional network in an embodiment of the present invention. Figure 2 is a schematic diagram of the overall architecture of the explicit-implicit cooperative graph convolutional network, Figure 3 is a schematic diagram of the structure of the explicit geometric configuration module, Figure 4 is a schematic diagram of the structure of the implicit associated motion module, and Figure 5 is a schematic diagram of the structure of the deep information aggregation module. Detailed Implementation
[0024] This embodiment provides a skeletal action recognition method based on explicit-latentative cooperative graph convolutional networks, such as... Figure 1 As shown, it includes the following steps: S1. Obtain skeletal data, i.e., skeletal motion sequences, from the skeletal dataset and construct a structure as follows: Figure 2 The explicit-implicit collaborative graph convolutional network is shown in (a) above. The input skeleton action sequence is represented as... ,in T represents the frame number. Let N represent a real number, C represent the number of joints, and C represent the feature dimension of each joint. The explicit-implicit co-graph convolutional network consists of 10 explicit-implicit co-graph convolutional layers.
[0025] S2. Constructing a network like this in an explicit-implicit collaborative graph convolutional network. Figure 2 The explicit geometry module shown in (b) inputs skeletal data into itself. Based on the inherent topological structure of the human skeleton and combined with a bidirectional time extension mechanism, it completes explicit geometric modeling of the motion in Euclidean space, as shown in (b). Figure 2 As shown.
[0026] Step S2 specifically includes the following sub-steps: S2.1 Spatial modeling based on hierarchical Euclidean graph constraints is used to capture subtle geometric changes in motion.
[0027] First, the human skeleton is formalized as a graph G=(V,E), where the set of nodes V corresponds to the coordinates of each joint, and the set of edges E strictly follows the skeletal connections defined by human anatomy, thus forming a topological graph with prior constraints. This design ensures that the adjacency relationships upon which convolution operations rely are not merely data-driven construction results, but are deeply embedded in the inherent geometric constraints of the human structure, enabling the feature extraction process to retain strong biophysical interpretability.
[0028] Subsequently, the static Euclidean spatial relationships between joints are encoded using a graph convolution operator, assuming the features of the l-th layer are... ,in Let N represent the number of real numbers, d represent the number of joints, and d represent the feature dimension. Convolutional update is represented as:
[0029] in, This indicates that A is a subset of the human skeleton, defined by its inherent topological structure. Each of the different levels of adjacency matrices Corresponding to spatially dependent structures at different scales; elements of matrix A Indicates joint With joints There must be a connection between them, otherwise .
[0030] The degree matrix is defined as follows: , that is, the degree of node v, which represents the number of its connected neighbors; This represents an adaptive perturbation matrix that learns to adjust the weights of the adjacency matrix to adapt to geometric changes during the action. It can capture dynamic changes in motion, allowing adjacency relationships to be flexibly adjusted as time and skeletal shape change; This represents a learnable weight matrix used to perform linear transformations on the node features of each layer. This represents a nonlinear activation function (such as ReLU), used to introduce nonlinear transformations and increase the expressive power of the model.
[0031] S2.2. Time modeling is performed by designing a bidirectional time extension mechanism to enhance the ability to represent the differentiated characteristics of different body parts.
[0032] For a given skeletal input sequence containing T frames, first select a core frame. To determine a representative posture of a continuous human motion; using the selected core frame as the center, a bidirectional time-expansion mechanism is employed to expand forward and backward along the time axis, constructing a structure of length 2. The motion segments are used to cover the temporal information before and after the action, ensuring that the temporal evolution characteristics of the action can be better captured during modeling.
[0033] Then, using the farthest point sampling strategy, N core joints are selected from each representative pose at a set sampling rate. For each selected core joint, M explicit neighboring joints are selected with a fixed number of hops h to construct the local spatial neighborhood of the core joint. Logarithmic spatiotemporal point convolution is used to extract fine-grained spatiotemporal motion features in the neighborhood, as shown in the following equation:
[0034]
[0035] in, Let represent the local geometric features of the i-th core joint in frame t. A multilayer perceptron (S-MLP) is constructed for encoding to capture local motion patterns within the explicit neighborhood of this core joint. For each core joint... Its local spatial neighborhood is defined as The selected N core joints are used as the centers for local feature extraction. For each core joint... The selected M dominant adjacent key points form its local spatial neighborhood set. ; It represents the spatial displacement relative to the core joints in the core frame and is used to characterize local geometric changes.
[0036] Subsequently, time series features Through a length of The aggregation is performed within the sliding time window; This represents the set of N core joints selected in frame t. For each core joint k, its corresponding local feature is... The operation applies to the global core node set within the same time window. Max pooling is performed to capture global dynamic dependencies across time; a multilayer perceptron (T-MLP) is constructed to further encode the aggregated features temporally.
[0037] S3. Construct implicit associated motion modules in explicit-implicit cooperative graph convolutional networks, such as... Figure 2 As shown in (c), in the implicit association motion module, the input skeletal data is mapped to a hypergraph representation and projected onto the Fourier domain. The implicit association between non-adjacent joints over time is learned through Fourier spectral convolution.
[0038] Step S3 specifically includes the following steps: S3.1 Map the input skeletal sequence to a hypervariable graph structure.
[0039] For skeleton sequences Traditional methods often in each A separate topological graph is constructed within the graph, with the adjacency matrix A fixed to reflect the anatomical structure. However, in the hypervariable graph... In, node set It no longer represents joints within a single frame, but rather joint trajectory units spanning the entire sequence, i.e., each node. For the i-th joint corresponding to the t-th frame, the size of the node set expands to N×T.
[0040] Within this framework Represents edge set, superedge Hyperedges can connect multiple joints across different frames simultaneously. For example, a hyperedge can connect the same joint in consecutive frames to capture its inherent consistency over time; it can also connect joints in the same frame that go beyond anatomical adjacency to model nonlocal spatial coordination. More generally, hyperedges can span temporal and spatial dimensions, thus revealing underlying rhythmic coupling patterns behind complex human movements. Mathematically, hypervariable graphs are characterized by their correlation matrix... express, This indicates the number of super edges; if the super node... Belongs to superedge ,but Otherwise .
[0041] S3.2. Based on the constructed hypervariable graph, perform spectral decomposition to map the skeleton graph signal from Euclidean space to the Fourier spectral domain, and perform learnable spectral convolution operations in the Fourier spectral domain.
[0042] First, define the hypergraph Laplacian operator as:
[0043] in, This represents the correlation matrix between nodes and hyperedges in the hypervariable graph; Let be the weight matrix of the hyperedges, representing the strength of each hyperedge; Let be the degree matrix of the nodes, representing the connectivity of the nodes; Let be the degree matrix of the hyperedge, representing the connectivity of the hyperedge. What is depicted is not just the local differences between adjacent joints, but the overall consistency of a group of joints within the same hyperedge, thus enabling the modeling of implicit spatiotemporal dependencies between joints.
[0044] Based on this, we project the skeleton map signal into the Fourier space and apply Fourier spectral domain convolution to capture the high-order dynamic patterns between hidden joints, as follows:
[0045] in, This represents the skeleton node features of the l-th layer after stacking on the hypervariable graph, where N represents the number of joints and T represents the number of frames. Indicates the number of channels; Indicates the updated features; For the supergraph Laplace, Given its Fourier basis and eigenspectrum, the column vectors of U are the Fourier eigenvectors of the hypergraph. For eigenvalues, Used for spectral domain normalization to ensure numerical stability; This represents the element-wise action of the Chebyshev polynomial on the diagonal spectrum. and It is a set of learnable spectral filter coefficients , This is the channel mapping matrix.
[0046] The two parts in parentheses together constitute the hypergraph Fourier filter operator. Firstly, Orthogonal polynomials are used to generate smooth and controllable transfer responses in the low-to-mid frequency band, which can be used to enhance the gradual consistency and rhythmic synchronization across frames and joints; secondly, The sensitivity to high-frequency and local mutation modes is enhanced by spectral exponentiation, thereby explicitly characterizing transient coupling and rapid coordination between unnaturally connected joints.
[0047] S4. Construct a deep information aggregation module in the explicit-implicit collaborative graph convolutional network, such as... Figure 2 As shown in (d), in the deep information aggregation module, the features extracted by the explicit geometric configuration module and the implicit correlation motion module are deeply aggregated, and the contribution weights of different feature flows are dynamically balanced through an adaptive feature recalibration mechanism.
[0048] The output of the explicit geometry module is represented as follows: The implicit flow aggregates at the spatiotemporal nodes and is projected back to the node level as follows: The fusion method then employs gated residual fusion:
[0049]
[0050] Where Pool(∙) represents channel pooling or global averaging, Indicates channel splicing. Denotes gated projection, where f(∙) represents a time-linear or nonlinear mapping that matches dimensions. For element-wise gating, The output features are represented by the gating coefficient G, which determines the compensation strength of the implicit information on the explicit basis based on the global semantics of the two paths. The residual structure ensures the stable preservation of the explicit geometric prior, while the implicit information is injected through a controlled path to enhance the discriminative ability while avoiding the destruction of geometric consistency. The static geometric basis provided by the explicit geometric configuration module and the implicit associated motion module complement each other, revealing the potential dynamic reorganization and higher-order collaboration in the action. This allows the entire bi-branch framework to take into account both explicit geometry and implicit motion, thereby more comprehensively and profoundly characterizing the spatial structure and dynamic evolution of skeletal behavior.
[0051] S5. Input the final spatiotemporal feature representation obtained in step S4 into the global average pooling layer and the fully connected layer to complete the human skeleton action recognition.
[0052] This embodiment designs an explicit-implicit co-graph convolutional network, which models skeletal information through explicit geometric imagination and implicit associative motion, achieving a comprehensive and deep understanding of human movements. It achieves 93.6% cross-topic accuracy and 97.5% cross-view accuracy on the NTU-RGB+D 60 dataset, 90.2% cross-topic accuracy and 91.0% cross-view accuracy on the NTU-RGB+D 120 dataset, and 97.8% accuracy on the MSR-Action3D dataset, significantly outperforming existing methods.
[0053] In addition to the embodiments described above, the present invention may have other implementations. All technical solutions formed by equivalent substitution or equivalent transformation fall within the protection scope claimed by the present invention.
Claims
1. A skeletal action recognition method based on explicit-implicit cooperative graph convolutional networks, characterized in that: Includes the following steps: S1. Obtain skeletal data, i.e. skeleton action sequences, from the skeletal dataset and construct an explicit-implicit co-operational graph convolutional network; S2. Construct an explicit geometric configuration module in an explicit-implicit co-electronic graph convolutional network. Input the skeletal data into the explicit geometric configuration module. Based on the inherent topology of the human skeleton and combined with a bidirectional time extension mechanism, complete the explicit geometric modeling of the action in Euclidean space. S3. Construct a latent association motion module in the explicit-latent collaborative graph convolutional network. In the latent association motion module, the input skeletal data is mapped to a hypergraph representation and projected onto the Fourier domain. The implicit association between non-adjacent joints over time is learned through Fourier spectrum convolution. S4. Construct a deep information aggregation module in the explicit-implicit co-convolutional graph convolutional network. In the deep information aggregation module, perform deep aggregation on the features extracted by the explicit geometric configuration module and the implicit associated motion module, and dynamically balance the contribution weights of different feature flows through an adaptive feature recalibration mechanism. S5. Input the spatiotemporal feature representation obtained in step S4 into the global average pooling layer and the fully connected layer to complete the human skeleton action recognition.
2. The skeletal action recognition method based on explicit-latentative cooperative graph convolutional networks according to claim 1, characterized in that: In step S1, the input skeleton action sequence is represented as follows: ,in T represents the frame number. Let N represent a real number, C represent the number of joints, and C represent the feature dimension of each joint. The explicit-implicit co-graph convolutional network consists of 10 explicit-implicit co-graph convolutional layers.
3. The skeletal action recognition method based on explicit-latentative cooperative graph convolutional networks according to claim 1, characterized in that: Step S2 specifically includes the following sub-steps: S2.1 Spatial modeling based on hierarchical Euclidean graph constraints is used to capture geometric changes in motion; S2.
2. Time modeling is performed by designing a bidirectional time extension mechanism to enhance the ability to represent the differentiated characteristics of different body parts.
4. The skeletal action recognition method based on explicit-implicit cooperative graph convolutional networks according to claim 3, characterized in that: In step S2.1, the human skeleton is first formalized as a graph G=(V,E), where the set of nodes V corresponds to the coordinates of each joint, and the set of edges E follows the skeletal connection relationship defined by human anatomy, thus forming a topological graph with prior constraints. Subsequently, the static Euclidean spatial relationships between joints are encoded using a graph convolution operator, assuming the features of the l-th layer are... ,in Let N represent the number of real numbers, d represent the number of joints, and d represent the feature dimension. Convolutional update is represented as: in, This indicates that A is a subset of the human skeleton, defined by its inherent topological structure. Each of the different levels of adjacency matrices Corresponding to spatially dependent structures at different scales; elements of matrix A Indicates joint With joints There must be a connection between them, otherwise ; The degree matrix is defined as follows: , that is, the degree of node v, which represents the number of its connected neighbors; This represents an adaptive perturbation matrix that learns to adjust the weights of the adjacency matrix to adapt to geometric changes during the action. This represents a learnable weight matrix used to perform linear transformations on the node features of each layer. This represents a nonlinear activation function used to introduce nonlinear transformations.
5. The skeletal action recognition method based on explicit-latentative cooperative graph convolutional networks according to claim 3, characterized in that: In step S2.2, for a given skeletal input sequence containing T frames, a core frame is first selected. To determine the representative posture of a continuous human movement; Using the selected core frame as the center, a bidirectional time-expanding mechanism is employed to expand forward and backward along the time axis, constructing a structure of length 2. The motion segments are analyzed to cover the temporal information before and after the action; then, using a farthest-point sampling strategy, N core joints are selected from each representative pose at a set sampling rate; for each selected core joint, M explicit neighboring joints are selected with a fixed number of hops h to construct the local spatial neighborhood of the core joint; logarithmic spatiotemporal point convolution operation is used to extract fine-grained spatiotemporal motion features in the neighborhood, as shown in the following formula: in, Let represent the local geometric features of the i-th core joint in frame t. A multilayer perceptron (S-MLP) is constructed for encoding to capture local motion patterns within the explicit neighborhood of this core joint. For each core joint... Its local spatial neighborhood is defined as The selected N core joints are used as the centers for local feature extraction. For each core joint... The selected M dominant adjacent key points form its local spatial neighborhood set. ; It represents the spatial displacement relative to the core joint points in the core frame, and is used to characterize local geometric changes; Subsequently, time series features Through a length of The aggregation is performed within the sliding time window; This represents the set of N core joints selected in frame t. For each core joint k, its corresponding local feature is... The operation applies to the global core node set within the same time window. Max pooling is performed to capture global dynamic dependencies across time; a multilayer perceptron (T-MLP) is constructed to further encode the aggregated features temporally.
6. The skeletal action recognition method based on explicit-latentative cooperative graph convolutional networks according to claim 1, characterized in that: Step S3 specifically includes the following steps: S3.1 Map the input skeletal sequence to a hypervariable graph structure; S3.
2. Based on the constructed hypervariable graph, perform spectral decomposition to map the skeleton graph signal from Euclidean space to the Fourier spectral domain, and perform learnable spectral convolution operations in the Fourier spectral domain.
7. The skeletal action recognition method based on explicit-latentative cooperative graph convolutional networks according to claim 6, characterized in that: In step S3.1, for the skeleton sequence In the hypervariable diagram In, node set This represents the joint trajectory unit spanning the entire sequence, i.e., each node. For the i-th joint corresponding to the t-th frame, the size of the node set is expanded to N×T; Represents edge set, superedge Simultaneously connecting multiple joints in different frames, the hypervariable graph uses its correlation matrix... express, This indicates the number of super edges; if the super node... Belongs to superedge ,but Otherwise .
8. The skeletal action recognition method based on explicit-latentative collaborative graph convolutional networks according to claim 6, characterized in that: In step S3.2, the hypergraph Laplacian operator is defined as: in, This represents the correlation matrix between nodes and hyperedges in the hypervariable graph; Let be the weight matrix of the hyperedges, representing the strength of each hyperedge; Let be the degree matrix of the nodes, representing the connectivity of the nodes; Let be the degree matrix of the hyperedge, representing the connectivity of the hyperedge.
9. A skeletal action recognition method based on an explicit-latentative collaborative graph convolutional network according to claim 8, characterized in that: In step S3.2, the skeleton map signal is projected into the Fourier space and Fourier spectral domain convolution is applied to capture the high-order dynamic patterns between hidden joints, as follows: in, This represents the skeleton node features of the l-th layer after stacking on the hypervariable graph, where N represents the number of joints and T represents the number of frames. Indicates the number of channels; Indicates the updated features; For the supergraph Laplace, Given its Fourier basis and eigenspectrum, the column vectors of U are the Fourier eigenvectors of the hypergraph. For eigenvalues, Used for spectral domain normalization; This represents the element-wise action of the Chebyshev polynomial on the diagonal spectrum. and It is a set of learnable spectral filter coefficients , This is the channel mapping matrix; the values within brackets [] and Together they constitute the hypergraph Fourier filter operator .
10. The skeletal action recognition method based on explicit-latentative cooperative graph convolutional networks according to claim 1, characterized in that: In step S4, the output of the explicit geometry module is represented as The implicit flow aggregates at the spatiotemporal nodes and is projected back to the node level as follows: The fusion method then employs gated residual fusion: Where Pool(∙) represents channel pooling or global averaging, Indicates channel splicing. Denotes gated projection, where f(∙) represents a time-linear or nonlinear mapping that matches dimensions. For element-wise gating, The output features are represented by the gating coefficient G, which determines the strength of the compensation of the latent information to the explicit basis. The residual structure ensures the preservation of the explicit geometric prior, while the latent information is injected through a controlled path.