Unsupervised 3D action recognition method based on context-aware topology attention enhancement

Abstract

The application relates to an unsupervised 3D motion recognition method based on context-aware topology attention enhancement, which comprises the following steps: step S1, obtaining a skeleton motion sequence set from a skeleton graph group, and dividing the skeleton motion sequence set into T clip clips after pretreatment clip ; step S2, extracting an action unit set epsilon with spatiotemporal locality from the T clip clips obtained after pretreatment by using an encoder; step S3, constructing a self-supervised recognition model, performing data enhancement on the action unit set based on a context-aware topology attention mechanism, and aggregating to obtain a context set wherein the self-supervised recognition model adopts a contrast loss L contrast that maximizes mutual information of the context set and the action unit set epsilon; and step S4, performing motion recognition by using the trained self-supervised recognition model. Compared with the prior art, the application has the advantage of high recognition accuracy.

Description

Technical Field

[0001] This invention relates to the field of action recognition technology, and in particular to an unsupervised 3D action recognition method based on context-aware topological attention enhancement. Background Technology

[0002] In recent years, skeleton-based action recognition has been an important emerging topic. However, most existing methods either model detailed but redundant information by reconstructing the coordinates of each body joint, or treat the action as a whole and ignore the semantic locality of action units that vary in space and time.

[0003] Furthermore, because individual humans possess unique gait patterns, and the intraclass variation in movement can be significant enough to identify individuals, it significantly impacts movement recognition performance. Specifically, this manifests as a performance degradation when recognizing movements performed by individuals not present in the training set. Individual adaptive behaviors lead to new patterns and regularities at higher levels, increasing the complexity of the entire system. Complex systems typically exhibit distinct hierarchies, and patterns from different levels cannot be easily transferred to other levels. Additionally, these unique patterns may be interconnected, but these connections are difficult to discern—the so-called implicit patterns.

[0004] Therefore, inspired by the fact that actions are localized in both space and time, and observing that a complete action can be divided into different clips in time and into movements of different body parts in space (topologically within the skeletal data), these unit dynamics exhibit less intra-class similarity and less intra-class variation compared to the entire action. In this context, joint-level movements constitute higher-level body-level movements, which in turn constitute higher-level human behaviors. For example, as... Figure 1 As shown, saluting can be divided into multiple action units (body-level actions), which are composed of joint-level actions. Humans can naturally infer the basic principles constituting behavior from the positional changes of different body parts and joints that are mainly involved in the action.

[0005] However, without a specially designed framework, self-supervised learning models struggle to learn these hidden patterns from unlabeled data. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the existing technology and provide an unsupervised 3D action recognition method based on context-aware topological attention enhancement. This method is based on the skeleton sequence extracted after the human body in the video is identified, and the target's behavior is judged.

[0007] The objective of this invention can be achieved through the following technical solutions:

[0008] This invention presents an unsupervised 3D action recognition method based on context-aware topological attention enhancement, which includes the following steps:

[0009] Step S1: Obtain the skeleton motion sequence set from the skeleton diagram set. After preprocessing, it is divided into T clip Editing;

[0010] Step S2: Use the encoder to obtain T after preprocessing cliip The set of action units ε with spatiotemporal locality is extracted from the editing;

[0011] Step S3: Construct a self-supervised recognition model, perform data augmentation on the action unit set based on the context-aware topological attention mechanism, and aggregate to obtain the context set. Among them, the self-supervised recognition model uses the maximization of the context set. Comparative loss L with mutual information of action unit set ε contrast Conduct training;

[0012] Step S4: Use the trained self-supervised recognition model to perform action recognition.

[0013] Preferably, step S1 includes the following sub-steps:

[0014] Step S11: Treat each skeleton graph as an undirected graph. Extracting skeleton motion sequences from skeleton diagrams The expression is:

[0015]

[0016] In the formula, For the sample set; X i Let x be the i-th skeleton sequence, representing a single sample, and N be the number of skeleton sequences; t Let be the joint skeleton diagram in frame t, where T is the frame number of the skeleton sequence; C is the joint position dimension; and V is the number of joints.

[0017] Step S12: Perform skeletal motion sequence Perform data augmentation preprocessing, dividing it into T clip Edited, preprocessed skeleton sequence set The expression is:

[0018]

[0019] U i =Preprocess(X i )

[0020] In the formula, U iFor preprocessed skeleton sequence samples, Preprocess(X) i ) represents the skeleton sequence sample X i The preprocessing process, where N is the number of skeleton sequences; each clip u t Contains a skeleton frame with a window size of K and an index of T. clip This represents the number of clips.

[0021] Preferably, the data augmentation preprocessing in step S2 includes random augmentation processing of the original skeleton motion sequence by displacement, rotation, scaling or tilting, as well as preprocessing using bilinear interpolation.

[0022] Preferably, the encoder in step S2 is a graph convolutional RNN network, and the expression for the set of action units ε with spatiotemporal locality is:

[0023]

[0024] In the formula, E i To extract from preprocessed skeleton sequence samples U i The extracted action units, ε is the action unit, e t To edit u t Extracted action units, C emb These are the parameters of a graph convolutional RNN network.

[0025] Preferably, the graph convolutional RNN includes a graph convolutional extended GRU unit, wherein the graph convolutional extended GRU unit uses a graph convolution operator. Replace the fully connected operator ω·z t The specific expression is:

[0026]

[0027]

[0028]

[0029]

[0030] In the formula, z t Indicates the update gate, r t This indicates that the door is being reset. These are candidate activation vectors; It is a graph convolution operator; the operator ⊙ represents the Hadamard product; σ is the sigmoid activation function, and ψ is the tanh activation function.

[0031] Preferably, step S3 specifically involves: embedding the action unit enhanced by the context-aware topological attention module into the aggregator in a loop to obtain the aggregated spatiotemporal context, expressed as:

[0032]

[0033]

[0034]

[0035] In the formula, C represents the context set obtained by aggregating the skeleton sequence sample set. cell This represents the number of units in each layer of the aggregator GraphGRU; Att(e,h) represents the computation process of the context-aware topological attention module, e t This represents the embedding features of action units. This represents the enhanced action unit embedding features. C represents the first hidden layer features of the GraphGRU at the previous time step. i Let c represent the context set of the i-th sample out of N samples. t This represents the context information obtained by aggregating the first t clips, and V represents the number of graph nodes in the skeletal joint graph, i.e., the number of joints.

[0036] Preferably, the context-aware topology attention module specifically comprises:

[0037] The input feature map and hidden state of the aggregator are fed into the context-aware topological attention module to generate an attention map. Then, a vector filled with 1s is added to the attention map, and then multiplied with the original feature map. The specific expression is as follows:

[0038] α i,t =softmax(σ)W ca ·concat(h i,t-1 ,e i,t ))+b ca )

[0039]

[0040] In the formula, concat represents a join operation. It is the code of the t-th action word in the i-th sample; It is the current state from GraphGRU, i.e., the encoded context information; α i,t This is the corresponding attention map, where σ is the Tanh activation function. b represents the linear layer weights. ca It is the deviation of the linear layer.

[0041] Preferably, the contrast loss in step S3 The expression is:

[0042]

[0043]

[0044] In the formula, Let each represent the predicted embedding from the i-th sample in the training batch. and actual encoding embedding e k ; Compute embedding pairs Dot product similarity.

[0045] Preferably, in step S3, the self-supervised recognition model employs maximizing the context set. The contrastive loss L is the mutual information between the action unit set ε and the action unit set ε. contrast Training is performed specifically by: based on a bidirectional prediction learning mechanism, aggregating context and predicting forward and backward embeddings to maximize the context set. The contrastive loss L is the mutual information between the action unit set ε and the action unit set ε. contrast The self-supervised recognition model is trained.

[0046] Preferably, the step of employing a bidirectional prediction learning mechanism to aggregate context and predict forward and backward embeddings to train the self-supervised recognition model includes:

[0047] Forward prediction: Using a prediction network to process contextual information c i,t Make predictions to obtain embedded prediction information. Then, the aggregator g(·) is used for aggregation to obtain the next prediction. And so on, the expression is:

[0048]

[0049]

[0050] In the formula, r predict It is the set prediction ratio, c i,0 This is the initial state of the aggregator;

[0051] Backward prediction:

[0052]

[0053]

[0054] In the formula, This is the initial state of the reverse aggregator; T clip·r predict For predictions;

[0055] Overall loss:

[0056]

[0057] In the formula, and Contrastive loss is applied to the contextual stream data based on different aggregation directions. The calculations are performed, and the expressions are as follows:

[0058]

[0059]

[0060] In the formula, the prediction number F = T clip ·r predict B represents the number of samples in a single training session. and These represent the action unit embeddings corresponding to forward prediction and backward prediction, respectively.

[0061] Compared with the prior art, the present invention has the following advantages:

[0062] 1) A self-supervised action recognition model based on topological attention enhancement of unit action words is adopted to improve the existing unsupervised 3D action recognition. The context-aware topological attention mechanism is used to enhance the more important and context-related node features in the action words, thereby obtaining better action representation. In addition, self-supervised training is carried out by comparing the predicted units with the actual encoding units, which improves the training accuracy of the model.

[0063] 2) Based on the spatiotemporal locality of actions, this invention uses the context of action units composed of position-aware maps to represent actions, which facilitates the extraction of local dynamic information;

[0064] 3) Based on the higher correlation between spatiotemporally adjacent joints caused by motion continuity and posture changes, spatiotemporal graph convolution is used to encode action units in a graph-based manner to obtain rich high-level semantics while preserving topology awareness.

[0065] 4) A graph convolutional GRU encoder is used to utilize long-term motion dynamics by fusing action units. Attached Figure Description

[0066] Figure 1 A schematic diagram of the spatiotemporal locality of action.

[0067] Figure 2 This is a flowchart of the method of the present invention;

[0068] Figure 3 Define a schematic diagram for the skeleton nodes;

[0069] Figure 4 This is a schematic diagram of random enhancement.

[0070] Figure 5 This is a schematic diagram of action unit encoding based on spatiotemporal graph convolution;

[0071] Figure 6 For graph convolutional model architecture;

[0072] Figure 7 A schematic diagram of GRU convolution for context aggregation graph;

[0073] Figure 8 This is a schematic diagram of a context-aware attention module. Detailed Implementation

[0074] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0075] Example

[0076] This embodiment presents an unsupervised 3D action recognition method based on context-aware topological attention enhancement, which includes the following steps:

[0077] 1. First, the skeletal motion sequence is viewed as multiple skeletal diagrams;

[0078] Each skeleton diagram is an undirected graph. Where V is the set of V = |V| skeletal joints, and E is the set edge (bones) of B = |E|;

[0079] The topological relationships of a skeleton graph are determined by the adjacency matrix A∈{0,1} V×V Constraints, where A ij =1 indicates the existence of a connection v i and v j If the edge is zero, then the edge is zero; otherwise, the edge is zero. Figure 3 The skeleton node defined for NTU.

[0080] 1) Skeleton sequence representation: For a given undirected graph Let X represent the N sequences with C-dimensional joint positions for V joints in frame T:

[0081]

[0082] In the formula, For the sample set; X i Let x be the i-th skeleton sequence, representing a single sample, and N be the number of skeleton sequences; t Here is the joint skeleton diagram in frame t, where T is the frame number of the skeleton sequence; C is the joint position dimension, which is set to 3 in this embodiment to represent three-dimensional coordinates; and V is the number of joints, which is set to 25 in this embodiment to represent the skeleton node defined by NTU.

[0083] 2) Preprocessing: Each skeleton sequence sample X i After preprocessing, it was divided into T clip Editing, then the preprocessed skeleton sequence set Represented as:

[0084] U i =Preprocess(X i )

[0085]

[0086] In the formula, U i For preprocessed skeleton sequence samples, Preprocess(X) i ) represents the skeleton sequence sample X i The preprocessing process, where N is the number of skeleton sequences; each clip u t Contains a skeleton frame with a window size of K and an index of T. clip This represents the number of clips.

[0087] 3) Spatiotemporal limitation feature extraction of action units: The spatiotemporal locality of action units is extracted using graph convolution through encoder f.

[0088] E i =f(U i )

[0089]

[0090] In the formula, E i To extract from preprocessed skeleton sequence samples U i The extracted action units, ε is the set of action units, e t To edit u t Extracted action units, C emb The parameters of the graph convolutional RNN network;

[0091] Topology awareness maintained by graph-based local awareness action units and context-aware topology attention mechanisms can enhance the features of more important nodes.

[0092] By considering the hidden layer information h from g(·), the attention-enhanced embedding Att(e,h) is cyclically fed into GraphGRUg(·) to aggregate the spatiotemporal context.

[0093]

[0094]

[0095]

[0096] Among them, C cell This represents the number of cells in each layer of GraphGRU. This represents the enhanced embedding features. This represents the first hidden layer feature of the GraphGRU at the previous time step. C represents the context set obtained from all samples. i Let c represent the context set of the i-th sample out of N samples. t This represents the context information obtained by aggregating the first t clips, and V represents the number of graph nodes in the skeletal joint graph, i.e., the number of joints.

[0097] The entire architecture compares the loss L contrast Maximize context The training is performed using a self-supervised mode that incorporates mutual information between the action units and ε.

[0098] Detailed process

[0099] 1. Pretreatment:

[0100] Data augmentation plays a crucial role in robust contrastive learning. To obtain effective action unit representations, clips input to the encoding network need to be appropriately transformed. These transformations should be pattern-invariant, enabling the encoder to capture the semantically invariant features of the clips.

[0101] Meanwhile, unlike the Euclidean geometry and color channels of RGB video, 3D skeletal data, especially joint connectivity graphs composed of three-dimensional coordinates, cannot be directly applied to 3D motion recognition using conventional data augmentation methods such as random rotation and random cropping. Based on observations of temporal continuity and coordinate characteristics, we employ spatiotemporal data augmentation on the skeleton sequence. The transformation can be expressed as:

[0102] The clips input to the encoding network are transformed, and the semantically invariant features of the clips are captured by the encoder to obtain effective action unit representations.

[0103] Based on temporal continuity and coordinate characteristics, spatiotemporal data augmentation is performed on the skeleton sequence. The transformation expression is as follows:

[0104]

[0105]

[0106] in, It is by Figure 3 The enhanced random composition in To generalize over time to normal random image resizing and cropping:

[0107]

[0108]

[0109] I i ∈[0,T target -K·T clip ]

[0110] R scale_ratio ∈[-0.2,0.2]

[0111] In the formula, Interpolate is a bilinear interpolation method, the first input is the input data, and the second input is the length of the output sequence; T target R represents the target length for interpolation. scale_ratio I represents the scaling factor of the randomly selected sequence length. i This indicates the starting sequence number of the slice of the randomly selected interpolated sequence.

[0112] 2. Local sensory action unit encoding:

[0113] Constructing a spacetime graph:

[0114] Inspired by the spatiotemporal locality of motion, a complete motion can be divided into multiple segments in time and into the movement of body parts in space (topologically within skeletal data).

[0115] These locations are encoded using graph convolutional networks, which provide clues to obtain distinctive local features of the skeleton, helping to overcome the challenges of inter-class and intra-class similarity. Furthermore, instead of simply flattening the embedded features into vectors, a graph structure is maintained to enhance their expressive power. In this case, nodes in the embedded feature map reflect the locality of action units. Embedding can be understood as feature encoding.

[0116] Generally, convolution aggregates information from each pixel within a local window. The information of each pixel has different weights depending on its position within the window. Essentially, these positions can be defined as different subsets with different weights. The definition of graph convolution derives from this, aggregating information from the current node and its surrounding nodes using the weights of different subsets. With a graph convolution kernel size of 3, the nodes in the window are divided into three subsets based on their distance from the skeleton: the current node, nodes closer to the centroid, and nodes farther from the centroid.

[0117] Consider the input graph Given the adjacency matrix A of the skeleton, a simplified graph convolution propagation rule is used:

[0118]

[0119] in, and Let represent the symmetric adjacency matrix with self-loops and its diagonal matrix, respectively. τ is the activation function, such as ReLU(·) = max(0,·). Therefore This is the output of the graph convolutional layer, taken as input. Θ represents the learnable weight matrix of the graph convolutional layer, M represents the attention graph representing the edge weights, and ⊙ represents element-wise multiplication. For example... Figure 3 As shown, the encoder will cut the skeleton U∈T clip ×K×V×C is used as input and output action unit code E∈T clip ×V×C emb .

[0120] In this embodiment, C emb =256, architecture as follows Figure 4 As shown, BN is a batch normalization layer that normalizes each input skeleton graph. TAP is a temporal average pooling layer that averages the temporal dimension of the feature dimension of each node in the skeleton graph. B1-B11 are spatiotemporal graph convolutional blocks. Each spatiotemporal graph convolutional block consists of a series of layers: spatial convolution, batch normalization, ReLU, temporal convolution, batch normalization, and ReLU.

[0121] At the beginning of the encoder, a batch normalization layer is set up to calculate the statistical values ​​of the coordinates to stabilize the training; at the end of the encoder, the features of each node are pooled across time only to preserve the topological awareness of the features.

[0122] 3. Action Unit Context Aggregation:

[0123] To simulate the long-term dynamics of action units, RNNs are used to aggregate the embedding context.

[0124] To best utilize the structural action unit embedding, a graph convolutional RNN is used to simultaneously extract long-term and topological dependencies. A graph convolutional extension of the GRU unit is employed, utilizing graph convolution operators. Replace the fully connected operator ω·z t :

[0125]

[0126]

[0127]

[0128]

[0129] Among them, z t Indicates the update gate, r t This indicates that the door is being reset. These are candidate activation vectors; is the graph convolution operator; the operator ⊙ represents the Hadamard product; σ is the sigmoid activation function, and ψ is the tanh activation function.

[0130] In this embodiment, g(·) is a 3-layer GraphGRU, with C in each layer. cell unit.

[0131] 4. Context-aware attention-enhanced action word features:

[0132] Fuse contextual information from the aggregator to better measure attention weights.

[0133] The aggregator's input feature map and hidden state are fed into a context-aware topological attention module to generate an attention map. A vector padded with 1s is added to the attention map before being multiplied with the original feature map to preserve the original features and enhance features that require more attention. Figure 6 As shown, the calculation is as follows:

[0134] α i,t =softmax(σ(W) ca ·concat(h i,t-1 ,e i,t ))+b ca )

[0135]

[0136] Here, 'concat' represents a connection operation. It is the encoding of the t-th action word in the i-th sample. It is the current state from GraphGRU (i.e., the encoded context information), α i,tThis is the corresponding attention map, where σ is the Tanh activation function. b represents the linear layer weights. ca It is the deviation of the linear layer.

[0137] 5. Comparative learning:

[0138] The entire model is trained using contrastive loss, as shown in the following formula:

[0139]

[0140]

[0141] in, This represents the predicted embedding from the i-th sample in the training batch. and actual encoding embedding e k 。; Compute embedding pairs Dot product similarity.

[0142] Essentially, It's a cross-entropy that forces the model to learn a way to distinguish between opposites. and negative pair The classifier is defined by (j,u) ≠ (i,k). This means that the only positive pair is the prediction at the current time point and its corresponding actual action unit encoding, and all other pairs are negative pairs.

[0143] 6. Action Unit Embedding Prediction:

[0144] After the aggregation unit is embedded, the context c i,t Used for prediction using the prediction network φ.

[0145] A hierarchical prediction technique is used to cumulatively predict and aggregate graph embeddings. Given context c i,t Using a prediction network φ to obtain embedded predictions Then, the predicted embeddings are aggregated by g(·) to obtain the next prediction. And so on:

[0146]

[0147]

[0148] Where, r predict It is set to a prediction ratio of 3 / 8, c i,0 This is the initial state of the aggregator.

[0149] To maximize the model's usable capacity (the prediction network was removed in evaluation and real-world use), a linear prediction function was used as φ, projecting the context representation onto...

[0150] 7. Two-way learning:

[0151] A bidirectional prediction learning mechanism is employed to aggregate context and predict embeddings both forward and backward, fully utilizing the temporal information of the skeleton sequence. The backward prediction can be expressed as:

[0152]

[0153]

[0154] in This is the initial state of the reverse aggregator.

[0155] Therefore, the overall loss can be expressed as follows:

[0156]

[0157] in, and Is using Calculated based on the context stream data from different aggregation directions:

[0158]

[0159]

[0160] Where, the number of predictions F = T clip ·r predict B represents the number of samples in a training batch. and This indicates the embedding of the action units corresponding to the forward and backward predictions.

[0161] Experimental results

[0162] Table 1 shows the comparison results of recognition performance of multiple methods. Here, Sup. indicates supervised training, and Con. indicates the use of contrastive learning. Joint, Motion, and Bone are data representations derived from the original skeleton data. The unit for each performance result is %. The method of this invention achieves optimal performance.

[0163] Table 1

[0164]

[0165]

[0166] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An unsupervised 3D action recognition method based on context-aware topological attention enhancement, characterized in that, The method includes the following steps: Step S1: Obtain the skeleton motion sequence set from the skeleton diagram set. After preprocessing, it is divided into Editing; Step S2: Using an encoder, obtain the preprocessed... Extracting a set of action units with spatiotemporal locality from the edit. ; Step S3: Construct a self-supervised recognition model, perform data augmentation on the action unit set based on the context-aware topological attention mechanism, and aggregate to obtain the context set. Among them, the self-supervised recognition model uses the maximization of the context set. and action unit set Comparison loss of mutual information The training will be conducted as follows: The input feature map and hidden state of the aggregator are fed into the context-aware topological attention module to generate an attention map. Then, a vector filled with 1s is added to the attention map, and then multiplied with the original feature map. The specific expression is as follows: In the formula, concat represents a join operation. It is the code of the t-th action word in the i-th sample; It is the current state from the aggregator GraphGRU, i.e., the encoded context information; This is the corresponding attention map. It is the Tanh activation function. Represents the linear layer weights. It is the deviation of the linear layer. The parameters of the graph convolutional RNN network; The action units enhanced by the context-aware topological attention module are embedded into the aggregator in a loop to obtain the aggregated spatiotemporal context, expressed as: In the formula, This represents the context set obtained by aggregating the skeleton sequence sample set. This represents the number of cells in each layer of the aggregator GraphGRU; The computation process of the context-aware topological attention module, This represents the embedding features of action units. This represents the enhanced action unit embedding features. This represents the first hidden layer features of the aggregator GraphGRU at the previous time step. Indicates in The first skeleton sequence in the _ _ The context set of each sample Indicates the preceding The context information obtained by aggregating individual clips for Quantity, For aggregators, This indicates the number of nodes in the skeletal joint diagram, i.e., the number of joints. Step S4: Use the trained self-supervised recognition model to perform action recognition.

2. The unsupervised 3D action recognition method based on context-aware topological attention enhancement according to claim 1, characterized in that, Step S1 includes the following sub-steps: Step S11: Treat each skeleton graph as an undirected graph. Extract skeleton motion sequences from skeleton diagrams The expression is: In the formula, For the sample set; For the first A skeleton sequence, representing a single sample. The number of skeleton sequences; For the first Joint skeleton diagram in the frame. The number of frames in the skeleton sequence; For joint position dimensions; For the number of joints; Step S12: Perform skeletal motion sequence Perform data augmentation preprocessing, divided into Edited, preprocessed skeleton sequence set The expression is: In the formula, This is a preprocessed skeleton sequence sample. Represents skeleton sequence samples The preprocessing process, The number of skeleton sequences; each clip Including window size skeleton frame, subscript for The quantity.

3. The unsupervised 3D action recognition method based on context-aware topological attention enhancement according to claim 2, characterized in that, The data augmentation preprocessing in step S2 includes random augmentation processing of the original skeleton motion sequence by displacement, rotation, scaling or tilting, as well as preprocessing using bilinear interpolation.

4. The unsupervised 3D action recognition method based on context-aware topological attention enhancement according to claim 2, characterized in that, The encoder in step S2 is a graph convolutional RNN network, which is a set of action units with spatiotemporal locality. The expression is: In the formula, For preprocessed skeleton sequence samples Extracted action units, As an action unit, To edit Extracted action units, This is a graph convolutional RNN network.

5. The unsupervised 3D action recognition method based on context-aware topological attention enhancement according to claim 4, characterized in that, The graph convolutional RNN includes a graph convolutional extended GRU unit, which uses a graph convolution operator. Replace fully connected operator The specific expression is: In the formula, Indicates an update to the door. This indicates that the door is being reset. These are candidate activation vectors; It is a graph convolution operator; operator Represents the Hadamard product; The sigmoid activation function is used. This is the tanh activation function.

6. The unsupervised 3D action recognition method based on context-aware topological attention enhancement according to claim 2, characterized in that, Contrast loss in step S3 The expression is: In the formula, These represent the numbers from the training batch. Predicted embeddings for each sample and actual encoding embedding ; Compute embedding pairs Dot product similarity.

7. The unsupervised 3D action recognition method based on context-aware topological attention enhancement according to claim 6, characterized in that, In step S3, the self-supervised identification model employs the maximization of the context set. and action unit set Comparison loss of mutual information Training is performed specifically by: based on a bidirectional prediction learning mechanism, aggregating context and predicting forward and backward embeddings to maximize the context set. and action unit set Comparison loss of mutual information The self-supervised recognition model is trained.

8. The unsupervised 3D action recognition method based on context-aware topological attention enhancement according to claim 7, characterized in that, The self-supervised recognition model is trained using a bidirectional prediction learning mechanism that aggregates context and predicts forward and backward embeddings, including: Forward prediction: Using a prediction network to process contextual information Make predictions to obtain embedded prediction information. Then, an aggregator is used. Aggregate to obtain the next prediction And so on, the expression is: In the formula, It is the set prediction ratio. This is the initial state of the forward aggregator; Backward prediction: In the formula, This is the initial state of the reverse aggregator; For predictions; Overall loss: In the formula, and Contrastive loss is applied to the contextual stream data based on different aggregation directions. The calculations are performed, and the expressions are as follows: In the formula, the predicted number , This refers to the number of samples in a batch during a single training session. and These represent the action unit embeddings corresponding to forward prediction and backward prediction, respectively.

Images