Dynamic graph optimization and functional adjacency fusion-based electroencephalogram and myoelectricity heterogeneous graph action recognition method

Abstract

The invention discloses an electroencephalogram and myoelectricity heterogeneous graph action recognition method based on dynamic graph optimization and functional adjacency fusion, which comprises the following steps: firstly, constructing a weighted fusion adjacency matrix of electroencephalogram (EEG) and myoelectricity (sEMG) channels by utilizing multiple functional connection indexes (including a phase locking value, a weighted phase lag index and mutual information) to form a heterogeneous graph structure; a learnable channel weight mechanism is introduced, redundant channel pruning is realized through sparse regularization, the expression efficiency of a graph structure is optimized, then local and global features of a fusion graph are extracted based on graph attention, and a comparative learning guide model is utilized to learn consistency and discriminative features between modes. The method can effectively improve the generalization ability and action recognition precision of electroencephalogram and myoelectricity joint modeling, and has good robustness and interpretability in a multi-channel brain-computer interface system.

Description

Technical Field

[0001] The present invention belongs to the field of using graph structures to perform multimodal fusion of brain and myoelectric channels and apply it to upper limb movement decoding, and is used to improve the accuracy of movement decoding. Background Art

[0002] With the rapid development of technologies such as human-computer interaction, neurorehabilitation, and intelligent control, multimodal action recognition methods based on electroencephalogram (EEG) and electromyography (sEMG) signals are gaining widespread attention. Fusion of these two methods facilitates more accurate decoding of human motion intent, thereby driving the development of intelligent control systems towards higher precision and greater robustness. Existing EEG-EMG fusion action recognition methods face several challenges: heterogeneous information between modalities and difficulty aligning them. EEG and sEMG signals differ significantly in physiological sources and frequency characteristics, making it difficult to effectively integrate the complementary information of these two signals through traditional simple feature concatenation. Redundant channels can also affect model performance. When the number of channels is excessive, some EEG or sEMG channels may not contribute significantly to target action recognition or may even introduce noise. Traditional methods often lack adaptive channel screening or pruning mechanisms. Most methods model signals in sequence or image form, neglecting the potential spatial functional connectivity between channels and limiting their ability to exploit interactive information across multiple channels. Summary of the Invention

[0003] To address the shortcomings of the existing technology, the present invention proposes a method for action recognition based on heterogeneous EMG graphs based on dynamic graph optimization and functional adjacency fusion. Each EMG channel is regarded as a graph node, and a connection graph is constructed through adjacency relationships. A EMG modeling method that combines multiple functional connectivity indicators, structural optimization capabilities, and supervised comparative learning is proposed to more fully extract cross-modal information and improve the accuracy and stability of action recognition.

[0004] It includes the following steps:

[0005] 1. A method for brain electromyography heterogeneous graph motion recognition based on dynamic graph optimization and functional adjacency fusion, characterized by comprising the following steps:

[0006] Step 1: Design an experimental paradigm for synchronous EEG and myoelectric acquisition of upper limb movements, collect EEG and EEG signals, and preprocess the collected EEG and sEMG synchronous signals.

[0007] Step 2: Treat each EEG and sEMG channel as a node in the graph, calculate multiple functional adjacency indices (phase locking value PLV, weighted phase lag index WPLI and mutual information MI), and obtain the corresponding adjacency matrix.

[0008] Step 3: Normalize the adjacency matrices of different functions and perform weighted combination to construct an EMG heterogeneous graph that integrates multiple adjacency features, forming a node feature matrix and a weighted adjacency matrix.

[0009] Step 4: Introduce a learnable channel weight mechanism to assign weights to each channel, prune the channels through sparse regularization terms, remove redundant nodes and their adjacencies, and optimize the compactness and representation capabilities of the graph structure.

[0010] Step 5: In the graph structure of heterogeneous EEG and sEMG modalities, multi-channel cross-modal fusion modeling is implemented based on the graph attention mechanism, and a supervised contrastive learning strategy is introduced to further optimize the discriminability and robustness of deep feature representation.

[0011] Step 6: Fuse the final extracted graph features and input them into the classifier to complete the recognition and judgment of action categories and realize the brain-electromyography joint action recognition task.

[0012] 2. The method for action recognition based on EEG and myoelectric graph heterogeneous graphs based on dynamic graph optimization and functional adjacency fusion according to claim 1, characterized by: Step 1: Designing an experimental paradigm for synchronous EEG and myoelectric acquisition for upper limb movements to acquire EMG and EEG signals. The acquired EEG and sEMG synchronous signals are preprocessed. First, a synchronous EEG and myoelectric acquisition experimental paradigm based on upper limb movement tasks is constructed to guide subjects to generate corresponding EEG and EMG signals while completing specific movement tasks. EEG signals are acquired using a standard 32-lead electrode cap according to the 10–20 system, while EMG signals are acquired synchronously using three surface electrodes attached to relevant upper limb muscle groups. The EEG and EMG acquisition systems achieve synchronous acquisition through a unified triggering mechanism, ensuring strict temporal synchronization between the two signals. The collected raw signals are preprocessed to ensure data quality. The preprocessing includes: using independent component analysis to remove artifact interference on EEG signals, and using bandpass filters to remove low-frequency and high-frequency noise on EEG and EMG signals. Finally, time window segmentation is performed according to a fixed window length for subsequent feature extraction and mapping.

[0013] 3. The method for brain electromyogram (EBMG) motion recognition based on dynamic graph optimization and functional adjacency fusion according to claim 1, characterized in that step 2: each EEG and sEMG channel is regarded as a node in the heterogeneous graph, and multiple functional adjacency indices (phase lock value, weighted phase lag index, and mutual information) are calculated to obtain the corresponding adjacency matrix. Specifically, as follows:

[0014] Node definition: Each EEG channel and each sEMG channel are defined as nodes in the graph structure, forming a node set V = {v1, v2, ..., v N}, where N = NEEG +N sEMG ,The attributes of each node are represented by the time domain signal of its corresponding channel in the current time window.

[0015] Functional adjacency calculation: Assume that the synchronous EEG and EMG signals v are acquired i (t), i=1,...,36, where 1-32 are EEG channels, 33-35 are EMG channels, and each node corresponds to the window data of a signal channel, v i (t) represents the time domain signal of the i-th channel in the time window t∈[1,T]. For any two channels (EEG or EMG), use v i (t) and v j (t) indicates that to perform three functional adjacency calculations, it is necessary to calculate the functional adjacency index between each pair of channels (EEG-EEG, EMG-EMG, EEG-EMG) and jointly construct an adjacency matrix.

[0016] (1) Calculate the phase lock value PLV, which is used to measure the phase synchronization between the two channels. First, perform Hilbert transform on the two signals to extract the instantaneous phase:

[0017] φ i (t)=arg(H[v i (t)]),φ j (t)=arg(H[v j (t)])

[0018] Calculate the complex exponential average of the phase difference to get the PLV:

[0019]

[0020] (2) Calculate the weighted phase lag index to solve the EEG volume conduction problem, measure the phase lag of the two signals, and remove zero-phase pseudo-synchronization:

[0021]

[0022] Among them, Im(C ij (t)) represents the imaginary part of the instantaneous cross spectrum of the two signals, sign(·) is the sign function, and E[·] represents the expectation.

[0023] (3) In order to capture the nonlinear relationship between the two channels, calculate the mutual information:

[0024]

[0025] Get three adjacency matrices:

[0026] 4. The method for action recognition based on EMG heterogeneous graphs based on dynamic graph optimization and functional adjacency fusion according to claim 1, characterized in that, in step 3, different functional adjacency matrices are normalized and then weighted combined to construct an EMG heterogeneous graph that integrates multiple adjacency features, thereby forming a node feature matrix and a weighted adjacency matrix. Specifically, due to the different value ranges and distributions of the three indicators, each adjacency matrix needs to be normalized separately, using Min-Max normalization:

[0027]

[0028] Get three normalized adjacency matrices: Construct a weighted fusion adjacency matrix using linear combinations of weight coefficients:

[0029]

[0030] The weight constraint is: PLV +λ WPLI +λ MI =1,λ∈[0,1] is obtained through training. The fusion matrix A∈R N×N At the same time, the structural information of different adjacent features is retained, which enhances the interpretability and generalization ability of brain electromyography.

[0031] Therefore, the graph structure is defined as follows: the node set V = {v1, v2, ..., v N}, including all EEG and sEMG channels, node feature matrix X∈R N×d , the adjacency matrix A, the final brain myoelectric heterogeneous graph is:

[0032] G=(V,A,X)

[0033] This graph not only contains intra-modal connections, but also introduces cross-modal functional connections, which facilitates the subsequent GNN model to jointly model in the structure domain and feature domain, thereby improving action recognition performance.

[0034] 5. The method for brain electromyography heterogeneous graph motion recognition based on dynamic graph optimization and functional adjacency fusion according to claim 1, characterized in that step 4: introduces a learnable channel weight mechanism, assigns weights to each channel, prunes the channels through sparse regularization terms, removes redundant nodes and their adjacencies, and optimizes the compactness and representation ability of the graph structure. For each node v in the graph i , introduce a learnable weight parameter α i ∈[0,1], indicating the importance of the channel to the overall graph modeling. All channel weights can be expressed as vector form:

[0035] α=[α1,α2,…,αN ]∈R N

[0036] These weights are learned adaptively by the network through back-propagation, reflecting the contribution of each channel to the final classification target during training. The node weights are also used to adjust the connection strength of the adjacency matrix:

[0037]

[0038] Weighted adjacency matrix Pay more attention to the connections between important channels.

[0039] Sparse regularization is introduced for channel pruning. In order to push the weights of unimportant channels to zero, the L1 sparse regularization term is introduced as part of the loss function:

[0040]

[0041] Among them: λ is the regularization coefficient, which controls the sparsity; L1 norm can make the network learn the weights of irrelevant channels to be close to 0, thus achieving automatic pruning in the structure. For the pruning condition judgment, by setting the threshold τ, when α i When τ<τ, a pruning strategy is implemented, where τ∈(0,1) is the automatically adjusted pruning threshold. The pruned nodes will be removed from the graph, and the edges connected to them will be All are set to zero. Finally, a subgraph structure G = (V′, A′, X′) with fewer channels but stronger representation ability is obtained.

[0042] 6. The method for action recognition based on heterogeneous EEG graphs based on dynamic graph optimization and functional adjacency fusion according to claim 1 is characterized in that, in step 5, in the graph structure of heterogeneous EEG (EEG) and sEMG (sEMG) modalities, cross-modal fusion modeling between multiple channels is realized based on the graph attention mechanism, and a supervised contrastive learning strategy is introduced to further optimize the discriminability and robustness of deep feature representation. Specifically: in order to fully explore the heterogeneous interaction relationship between nodes in the graph, the graph attention mechanism is introduced to realize neighborhood-aware weight distribution. For any node i, its representation is updated as:

[0043]

[0044] Where: h j represents the feature vector of neighbor node j, represents the attention weight of the k-th head, || represents multi-head output concatenation, σ represents the activation function, W (k)Represents the linear transformation matrix of the kth head. Considering that action category boundaries may have fuzzy regions, relying solely on cross-entropy loss to train a classifier will lead to inter-class confusion. To this end, a supervised contrastive learning strategy is further introduced to enhance the clustering of representations of similar nodes and the separation of heterogeneous ones. This method uses the node embedding vectors extracted by the graph attention mechanism as input and constructs positive and negative sample pairs based on label supervision. This optimizes the feature space structure after modal fusion, making samples of the same category more clustered and samples of different categories more dispersed.

[0045] In a training batch, each sample is assumed to output its final representation vector as a sample embedding z through the graph attention network (such as GAT) i , construct positive sample pairs: with z i Other samples belonging to the same action category; negative sample pairs: with z i All samples belonging to different action categories. To measure the similarity of sample pairs in the embedding space, we use the temperature-scaled cosine similarity contrast loss. Given a sample embedding z i With its positive sample set P(i), its loss function is defined as:

[0046]

[0047] in I Represents the set of all samples in a batch, sim(z i ,z p ) represents the normalized cosine similarity, and τ represents the temperature hyperparameter.

[0048] The cross entropy loss represents the difference between the probability distribution predicted by the model and the true label:

[0049]

[0050] In order to combine classification accuracy and feature separability, we perform a weighted fusion of contrast loss and cross entropy loss to form the final training loss function:

[0051] L=αL1+(1-α)L2

[0052] The α weight coefficient adjusts the balance between the two losses.

[0053] 7. According to the method for action recognition based on EMG heterogeneous graphs based on dynamic graph optimization and functional adjacency fusion as described in claim 1, step 6: fusing the final extracted graph features and inputting them into a classifier to complete action category identification and determination, thereby achieving the task of joint EMG-EMG action recognition. Specifically, all node features are pooled, and the pooled results are then passed through two fully connected layers to output the final action category probability.

[0054] The advantages and beneficial effects of the present invention are as follows:

[0055] This paper proposes a method for action recognition based on EMG heterogeneous graphs based on dynamic graph optimization and functional adjacency fusion. It can effectively solve the problems of heterogeneous fusion of modal information, weak structural modeling capabilities, and channel redundancy in existing EMG fusion action recognition methods. By fusing multiple functional connectivity indicators (such as PLV, WPLI, and MI), a weighted adjacency matrix is ​​constructed to more comprehensively reflect the functional relationship between EEG and EMG channels. A learnable channel weight mechanism is introduced, combined with sparse regularization, to achieve pruning optimization of redundant channels and improve model efficiency. A graph attention mechanism is used to perform feature modeling on EMG heterogeneous graphs to enhance cross-modal fusion capabilities. Combined with supervised contrastive learning and cross-entropy loss, the model's ability to distinguish different action categories and generalization performance are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 It is a method for action recognition based on brain electromyography heterogeneous graphs based on dynamic graph optimization and functional adjacency fusion. DETAILED DESCRIPTION

[0057] The following will describe the technical solutions in the embodiments of the present invention in detail with reference to the accompanying drawings. The described embodiments are only a part of the embodiments of the present invention.

[0058] As shown in the figure, this embodiment provides a method for brain electromyogram action recognition based on dynamic graph optimization and functional adjacency fusion, including the following steps:

[0059] A method for brain and myoelectric heterogeneous graph motion recognition based on dynamic graph optimization and functional adjacency fusion, characterized by comprising the following steps:

[0060] Step 1: Design an experimental paradigm for synchronous EEG and myoelectric acquisition of upper limb movements, collect EEG and EEG signals, and preprocess the collected EEG and sEMG synchronous signals.

[0061] Step 2: Treat each EEG and sEMG channel as a node in the graph, calculate multiple functional adjacency indices (phase locking value PLV, weighted phase lag index WPLI and mutual information MI), and obtain the corresponding adjacency matrix.

[0062] Step 3: Normalize the adjacency matrices of different functions and perform weighted combination to construct an EMG heterogeneous graph that integrates multiple adjacency features, forming a node feature matrix and a weighted adjacency matrix.

[0063] Step 4: Introduce a learnable channel weight mechanism to assign weights to each channel, prune the channels through sparse regularization terms, remove redundant nodes and their adjacencies, and optimize the compactness and representation capabilities of the graph structure.

[0064] Step 5: In the graph structure of heterogeneous EEG and sEMG modalities, multi-channel cross-modal fusion modeling is implemented based on the graph attention mechanism, and a supervised contrastive learning strategy is introduced to further optimize the discriminability and robustness of deep feature representation.

[0065] Step 6: Fuse the final extracted graph features and input them into the classifier to complete the recognition and judgment of action categories and realize the brain-electromyography joint action recognition task.

Claims

1. A method for action recognition based on brain electromyography heterogeneous graphs based on dynamic graph optimization and functional adjacency fusion, characterized in that: The following steps are involved: Step 1: Design an experimental paradigm for synchronous EEG and myoelectric acquisition of upper limb movements, collect EEG and EEG signals, and preprocess the collected EEG and sEMG synchronous signals; Step 2: Treat each EEG and sEMG channel as a node in the graph, calculate multiple functional adjacency indices (phase locking value PLV, weighted phase lag index WPLI and mutual information MI), and obtain the corresponding adjacency matrix; Step 3: Normalize the adjacency matrices of different functions and then perform weighted combination to construct an EMG heterogeneous graph that integrates multiple adjacency features, forming a node feature matrix and a weighted adjacency matrix; Step 4: Introduce a learnable channel weight mechanism to assign weights to each channel. Prune the channels using a sparse regularization term to remove redundant nodes and their adjacencies, thereby optimizing the compactness and representation capabilities of the graph structure. Step 5: In the graph structure of heterogeneous EEG and sEMG modalities, we implement cross-modal fusion modeling between multiple channels based on the graph attention mechanism and introduce a supervised contrastive learning strategy to further optimize the discriminability and robustness of the deep feature representation. Step 6: Fuse the final extracted graph features and input them into the classifier to complete the recognition and judgment of action categories and realize the brain-electromyography joint action recognition task.

2. The method for brain electromyogram motion recognition based on dynamic graph optimization and functional adjacency fusion according to claim 1, characterized in that: The step 1 is as follows: a synchronous EEG and sEMG acquisition experimental paradigm for upper limb movements is designed to acquire EEG and EMG signals, and the acquired EEG and sEMG synchronous signals are preprocessed. First, a synchronous EEG and sEMG acquisition experimental paradigm based on upper limb movement tasks is constructed to guide subjects to generate corresponding EEG and EMG signals while completing specific movement tasks. EEG signals are acquired using a standard 32-lead electrode cap according to the 10–20 system, and EMG signals are acquired synchronously using three surface electrodes attached to relevant upper limb muscle groups. The EEG and EMG acquisition systems are synchronously acquired through a unified trigger mechanism to ensure strict time synchronization between the two signals. The acquired raw signals are preprocessed to ensure data quality. The preprocessing includes: using independent component analysis to remove artifact interference from the EEG signals, and using bandpass filters to remove low-frequency and high-frequency noise from the EEG and EMG signals. Finally, time window segmentation is performed according to a fixed window length for subsequent feature extraction and mapping.

3. The method for brain electromyogram motion recognition based on dynamic graph optimization and functional adjacency fusion according to claim 1, characterized in that: Step 2: Treat each EEG and sEMG channel as a node in a heterogeneous graph, calculate multiple functional adjacency indices (phase locking value, weighted phase lag index, and mutual information), and obtain the corresponding adjacency matrix, specifically as follows: Node definition: Each EEG channel and each sEMG channel are defined as nodes in the graph structure, forming a node set V = {v1, v2, ..., v N }, where N = N EEG +N sEMG ,The attributes of each node are represented by the time domain signal of its corresponding channel in the current time window; Functional adjacency calculation: Assume that the synchronous EEG and EMG signals v are acquired i (t), i=1,...,36, where 1-32 are EEG channels, 33-35 are EMG channels, and each node corresponds to the window data of a signal channel, v i (t) represents the time domain signal of the i-th channel in the time window t∈[1,T], and for any two channels (EEG or EMG), v i (t) and v j (t) indicates that to perform three kinds of functional adjacency calculations, it is necessary to calculate the functional adjacency index between each pair of channels (EEG-EEG, EMG-EMG, EEG-EMG) and jointly construct an adjacency matrix; (1) Calculate the phase lock value PLV, which is used to measure the phase synchronization between the two channels. First, perform Hilbert transform on the two signals to extract the instantaneous phase: ϕ i (t)=arg(H[v i (t)]),φ j (t)=arg(H[v j (t)])? Calculate the complex exponential average of the phase difference to get the PLV: (2) Calculate the weighted phase lag index to solve the EEG volume conduction problem, measure the phase lag of the two signals, and remove zero-phase pseudo-synchronization: Among them, Im(C ij (t)) represents the imaginary part of the instantaneous cross spectrum of the two signals, sign(·) is the sign function, and E[·] represents the expectation; (3) In order to capture the nonlinear relationship between the two channels, calculate the mutual information: Get three adjacency matrices:

4. The method for brain electromyogram motion recognition based on dynamic graph optimization and functional adjacency fusion according to claim 1, characterized in that: Step 3: normalize the adjacency matrices of different functions and then perform weighted combination to construct an EMG heterogeneous graph integrating multiple adjacency features, thereby forming a node feature matrix and a weighted adjacency matrix. Specifically, due to the different value ranges and distributions of the three indicators, each adjacency matrix needs to be normalized separately, using Min-Max normalization: Get three normalized adjacency matrices: Construct a weighted fusion adjacency matrix using linear combinations of weight coefficients: The weight constraint is: PLV +λ WPLI +λ MI =1,λ∈[0,1] is obtained through training, the fusion matrix A∈R N×N At the same time, the structural information of different adjacent features is retained, which enhances the interpretability and generalization ability of EMG. Therefore, the graph structure is defined as follows: the node set V = {v1, v2, ..., v N }, including all EEG and sEMG channels, node feature matrix X∈R N×d , the adjacency matrix A, the final brain myoelectric heterogeneous graph is: G=(V,A,X); This graph not only contains intra-modal connections, but also introduces cross-modal functional connections, which facilitates the subsequent GNN model to jointly model in the structure domain and feature domain, thereby improving action recognition performance.

5. The method for brain electromyogram motion recognition based on dynamic graph optimization and functional adjacency fusion according to claim 1, characterized in that: Step 4: Introduce a learnable channel weight mechanism, assign weights to each channel, prune the channel through the sparse regularization term, remove redundant nodes and their adjacencies, optimize the compactness and representation ability of the graph structure, and for each node v in the graph i , introduce a learnable weight parameter α i ∈[0,1], indicating the importance of the channel to the overall graph modeling, all channel weights can be expressed as vector form: α＝[α1,α2,...,α N ]∈R N ； These weights are learned adaptively by the network through back-propagation, reflecting the contribution of each channel to the final classification target during training. The node weights are also used to adjust the connection strength of the adjacency matrix: Weighted adjacency matrix Pay more attention to the connections between important channels; Sparse regularization is introduced for channel pruning. In order to push the weights of unimportant channels to zero, the L1 sparse regularization term is introduced as part of the loss function: Where: λ is the regularization coefficient, which controls the sparsity; The L1 norm can make the network learn the weights of irrelevant channels close to 0, thereby achieving automatic pruning in the structure. For the pruning condition, by setting the threshold τ, when α i When τ is less than τ, a pruning strategy is implemented, where τ∈(0,1) is the automatically adjusted pruning threshold. The pruned nodes will be removed from the graph, and the edges connected to them will be Set all to zero, and finally obtain a subgraph structure G = (V′, A′, X′) with fewer channels but stronger representation ability.

6. The method for brain electromyography motion recognition based on dynamic graph optimization and functional adjacency fusion according to claim 1, characterized in that: Step 5: In the graph structure of heterogeneous EEG and sEMG modalities, a graph attention mechanism is used to implement cross-modal fusion modeling between multiple channels, and a supervised contrastive learning strategy is introduced to further optimize the discriminability and robustness of deep feature representation. Specifically, in order to fully explore the heterogeneous interaction relationships between nodes in the graph, a graph attention mechanism is introduced to implement neighborhood-aware weight allocation. For any node i, its representation is updated as follows: Where: h j represents the feature vector of neighbor node j, represents the attention weight of the k-th head, || represents multi-head output concatenation, σ represents the activation function, W (k) The linear transformation matrix representing the kth head. Considering that there may be fuzzy areas at the boundaries of action categories, relying solely on cross-entropy loss to train the classifier will lead to confusion between action classes. Therefore, a supervised contrastive learning strategy is further introduced to enhance the aggregation of similar node representations and the separation of different categories. This method uses the node embedding vector extracted by the graph attention mechanism as input and constructs positive and negative sample pairs based on label supervision, thereby optimizing the feature space structure after modal fusion, making samples of the same category more aggregated and samples of different categories more dispersed. In a training batch, each sample is assumed to output its final representation vector as a sample embedding z through the graph attention network (GAT) i , construct positive sample pairs: with z i Other samples belonging to the same action category; negative sample pairs: with z i All samples belonging to different action categories, to measure the similarity of sample pairs in the embedding space, we use the temperature-scaled cosine similarity contrast loss. Given a sample embedding z i With its positive sample set P(i), its loss function is defined as: in I Represents the set of all samples in a batch, sim(z i ,z p ) represents the normalized cosine similarity, and τ represents the temperature hyperparameter; The cross entropy loss represents the difference between the probability distribution predicted by the model and the true label: In order to combine classification accuracy and feature separability, we perform a weighted fusion of contrast loss and cross entropy loss to form the final training loss function: L=αL1+(1-α)L2 The α weight coefficient adjusts the balance between the two losses.

7. According to the method for action recognition based on EMG heterogeneous graphs based on dynamic graph optimization and functional adjacency fusion as described in claim 1, step 6: fusing the finally extracted graph features and inputting them into the classifier to complete the recognition and judgment of the action category, thereby realizing the EMG-EMG combined action recognition task. Specifically, all node features are pooled, and then the pooled results are passed through two fully connected layers to output the final action category probability.

Images