A gait recognition method based on a three-branch input network and channel topology graph convolution.

Abstract

This invention discloses a gait recognition method based on a three-branch input network and channel topology graph convolution. It acquires a human skeletal sequence, constructs a skeletal graph with joints as vertices and bones as edges, and obtains joint movement velocities through temporal relationships within the sequence. The three-branch input network uses joint position, joint velocity, and bone position as three inputs, performs feature extraction on each, and then concatenates the three output features to obtain a fused feature. This fused feature serves as the input to the channel topology module, extracting channel features and sample features and converting them into high-level features. This captures the topological features between joints under different gait correlations. The topological features are then aggregated with the corresponding channel high-level features as the output features for gait recognition. This invention improves the accuracy and stability of gait recognition by considering the correlation between skeletal points.

Description

Technical Field

[0001] This invention belongs to the field of target recognition technology based on artificial intelligence, and specifically relates to a gait recognition method based on a three-branch input network and channel topology graph convolution. Background Technology

[0002] Gait recognition is a technology that identifies human movement and identity information by recognizing gait characteristics such as stride frequency and stride length. Its advantage lies in the fact that it does not require contact sensors, allowing for long-distance, non-invasive human posture identification, and it has many applications in fields such as healthcare, security, and smart homes.

[0003] In recent years, human skeleton modeling methods have demonstrated excellent robustness and efficiency in gait recognition tasks. Emerging technologies, such as graph convolutional networks and deep learning, have also made significant progress in this field, providing better solutions for improving the accuracy and robustness of gait recognition.

[0004] Because the human skeleton is a complex structure with unique characteristics, attention is usually focused on skeletal points with significant displacement, through which basic gait features can be obtained. However, existing techniques all ignore the correlations between skeletal points. In practice, different movement patterns have varying impacts on the spatial correlations of nodes. Ignoring these correlations leads to varying degrees of feature loss, thus limiting recognition accuracy. Summary of the Invention

[0005] In order to overcome the shortcomings of the prior art, the present invention aims to provide a gait recognition method based on a three-branch input network and channel topology graph convolution, so as to improve the accuracy of gait recognition by combining the correlation between skeletal points.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A gait recognition method based on a three-branch input network and channel topology graph convolution includes:

[0008] Step 1: Obtain the human skeleton sequence, construct a skeleton graph G = (V, E, X') with joints as vertices and bones as edges, and obtain the joint movement speed through the sequence time relationship, where V is the vertex set, E is the edge set, and X' is the feature set of the vertex;

[0009] Step 2: Using joint position, joint velocity, and bone position as the three inputs to the three-branch network, respectively, feature extraction is performed on each input, and the three output features are then concatenated to obtain the fused feature X.

[0010] Step 3: Using the fusion feature X as the input of the channel topology module, extract channel features and sample features and convert them into high-level features to capture the topological features between joints under different types of gait correlation. Aggregate the topological features with the high-level features of the corresponding channels as the output feature Z.

[0011] Step 4: After passing the output feature Z through a pooling layer, gait recognition is performed using a fully connected layer.

[0012] Furthermore, in the skeletal diagram, V = {v1, v2, ..., v} N}, where N is the number of joints, and E represents the adjacency matrix I∈R. N×N Its element a ij Reflects the i-th joint v i With the j-th joint v j The correlation strength, v i The neighborhood is represented as N(v i )={v j |a ij ≠0},v i The feature is represented by x i ∈R C , X'∈R N×C , is a matrix containing C-dimensional features of all N joints.

[0013] Furthermore, in the three-branch input network, each branch consists of a BN layer and three convolutional layers M1, M2, and M3 connected sequentially; where M1 is the basic block, including a spatial convolution and a temporal convolution, used to extract preliminary features; M2 adds a bottleneck block to M1 to reduce computational complexity and retain important information; and M3 adds a residual structure and an SC-Attention block to M2 to focus on more important features.

[0014] Furthermore, the channel topology module includes two cascaded units, each of which includes a channel topology graph convolutional network. The two units have different numbers of channels, and the second unit adds an SC-Attention block.

[0015] Furthermore, the SC-Attention block performs channel and spatial weighting on the input features to obtain channel features and spatial features, and adds the spatial features and channel features together as the output features.

[0016] Furthermore, each unit of the channel topology module also includes a temporal convolutional network. The input of the channel topology graph convolutional network is the fused feature X, and the output and the fused feature X are used together as the input of the temporal convolutional network. The input and output of the temporal convolutional network are concatenated to obtain the output feature Z.

[0017] Furthermore, the channel topology graph convolutional network consists of three functional parts: channel topology modeling, feature transformation, and channel feature aggregation.

[0018] Its input is the fused feature X∈R N×C Output feature Z'∈R N×C It is given by the following formula:

[0019] Z'=I(B(X),Y'(G(X),E'))

[0020] Where, I∈R N×N Let E be a fixed adjacency matrix based on the human skeletal structure, which is the matrix representation of the edge set E. B(X) and G(X) both represent feature transformation operations on the input, extracting different feature dimensions from the input respectively. E' is a fixed feature matrix or parameter set, independent of the input fusion feature X. y' is a fusion function. Y'(G(X),E) means combining the result of G(X) and E' to generate a new feature representation.

[0021] Furthermore, the channel topology modeling function is implemented as follows:

[0022] The specific channel correlation K is obtained by performing a series of linear transformations on the input fusion feature X, and then K is introduced into the adjacency matrix I to finally obtain the refined channel topology Y∈R. N×N×C ;

[0023] The feature transformation function is implemented as follows:

[0024] When modeling the channel topology, a feature transformation function is used to convert the input fused feature X into a higher-level feature, as shown in the following expression:

[0025]

[0026] That These are the transformed features, and W is the weight matrix;

[0027] The channel feature aggregation function:

[0028] Based on the refined channel topology Y, obtain the refined topology Y for each channel. c , using Y c and Construct a channel graph for each channel to aggregate features in a channel-based manner.

[0029] Furthermore, the specific channel correlation K is obtained through the following method:

[0030] First, for the corresponding feature (x) i ,xj A pair of vertices (v) i ,v j Perform a linear transformation Γ(x) i ) and Φ(x j );

[0031] Secondly, calculate x i and x j Distance d along the channel dimension ij ;

[0032] Then, d ij Perform a nonlinear transformation to obtain the value w ij As v i With v j The association weights of specific channels between them are used to construct a dynamic topology matrix or represent the topological relationship between vertex pairs;

[0033] Finally, by using a linear transformation ξ to increase the channel dimension, we obtain the specific channel correlation K, expressed as:

[0034] k ij ={ξ(G(Γ(x)} i ),Φ(x j ))}

[0035] Where k ij It is a vector belonging to K, reflecting the vertex pair (v i ,v j The specific topological relationship between channels k ij No forced symmetry, G(Γ(x) i ),Φ(x j ) is the modeling function, G(Γ(x) i ),Φ(x j ))=σ(Γ(x i )-Φ(x j )), σ(·) is the activation function.

[0036] Furthermore, the refined channel topology Y obtained by introducing K into the adjacency matrix I is expressed as: Y = I + a·K; where a is an adjustable scaling factor used to control the degree of influence of K on the final Y.

[0037] Compared with existing technologies, this invention uses joint position, joint velocity, and bone position as inputs to capture the topological features between joints under different types of gait correlations, and aggregates them with the high-level features of the corresponding channels as output features for gait recognition. By considering the correlation between bone points, the accuracy and stability of gait recognition are improved. Attached Figure Description

[0038] Figure 1This is a schematic diagram of the overall process of the present invention.

[0039] Figure 2 This is a schematic diagram of the topological structure of a skeletal point with significant displacement.

[0040] Figure 3 This is a diagram of the three-branch input network structure of the present invention.

[0041] Figure 4 This is the convolution framework for the channel-type topology graph of the present invention. Detailed Implementation

[0042] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings and examples.

[0043] This invention presents a gait recognition method based on a three-branch input network and channel topology map convolution. It uses a three-branch input network and channel topology mapping convolution to capture spatiotemporal and channel-dimensional feature information. Furthermore, to prevent the model from over-focusing on redundant features and reducing efficiency, this invention also designs a spatial channel focusing mechanism to help better capture key features. The overall framework of this invention is as follows: Figure 1 As shown. A detailed description follows.

[0044] Step 1: Obtain the human skeleton sequence, construct a skeleton map, and obtain the joint movement velocity through the temporal relationship of the sequence.

[0045] This invention uses human joint information as input features. Specifically, in one embodiment, HRNet is used to extract human joints from the CASIA-B dataset, resulting in a human skeleton sequence composed of 17 joints. This skeleton information is then used for gait recognition. In the skeleton diagram, joints are used as vertices and bones as edges to obtain the skeletal representation of the human body. The skeleton diagram is represented as G = (V, E, X'), as shown below. Figure 2 As shown. Where V is the set of vertices, it can be represented as V = {v1, v2, ..., v...} N}, where N is the number of joints. E is the set of edges, which can be represented as the adjacency matrix I∈R. N×N Its element a ij Reflects the i-th joint v i With the j-th joint v j The correlation strength, v i The neighborhood is represented as N(v i )={v j |a ij ≠0},v i The feature is represented by x i ∈R C X' is the feature set of vertices, X'∈R N×C , is a matrix containing C-dimensional features of all N joints.

[0046] In a specific embodiment, the human posture is first estimated, and then the human skeleton is topologically optimized. Using the optimized skeleton topology, combined with association weights and feature extraction mechanisms, a more reliable feature representation is provided for downstream pattern analysis and task applications. Figure 2 The different colors of the dashed lines are used to distinguish different topological structures, while the thickness of the solid lines represents different degrees of correlation.

[0047] Step 2: Construct the recognition model.

[0048] The identification model of this invention mainly consists of a three-branch input network and a channel topology module. Figure 2 The green section shows the internal structure of the channel topology module, including the CTCN, TCN, and remaining structures. The following sections explain the functions of the three-branch input network and the channel topology module.

[0049] Step 21: Using joint position, joint velocity, and bone position as the three inputs to the three-branch input network, respectively, feature extraction is performed on each input. The three output features are then concatenated to obtain the fused feature X.

[0050] It is worth noting that current model-based methods should not focus on only one type of input feature. In model-based gait recognition, this invention uses an improved three-branch input network as the first part of the model, such as... Figure 3 As shown.

[0051] The input features of the model are divided into three categories: 1) joint position; 2) joint velocity; 3) bone position.

[0052] The three-branch input network of this invention consists of three branches. Each branch is composed of a batch normalization (BN) layer and three convolutional layers (M1, M2, M3) connected in sequence.

[0053] To facilitate raw data processing, a basic block M1 is used in the first part of each branch, containing a spatial convolution and a temporal convolution to extract preliminary features. Then, a bottleneck block and a residual structure are added to the basic block to obtain module M2, which reduces computational complexity while preserving important information. A residual structure and an SC-Attention block are added to M2 to obtain module M3, which focuses on more important features. Finally, the features from these three branches are concatenated to obtain the fused feature X, which serves as the input for the next part.

[0054] This invention, through a three-branch input network, captures skeletal positions, enabling the model to capture spatial layout information between joints and more accurately reflect an individual's walking characteristics. By considering joint velocities, dynamic changes during gait can be captured, helping to distinguish different gait stages and increasing sensitivity to subtle gait differences. In contrast, comprehensive consideration of skeletal positions allows the model to integrate static and dynamic information, enhancing the overall understanding of gait patterns.

[0055] Step 22: Using the fusion feature X as the input to the channel topology module, extract channel features and sample features and convert them into high-level features to capture the topological features between joints under different types of gait correlation. Aggregate the topological features with the high-level features of the corresponding channels as the output feature Z.

[0056] The structure of the channel topology module is as follows: Figure 2 and 4 The algorithm comprises two cascaded units, C1 and C2, each containing a channel topology graph convolutional network (CTCN module). The two units have different numbers of channels, and C2 adds an SC-Attention block to C1. First, the extracted channel and sample features are converted into high-level features. Static channel topology of the sample features is achieved using an adjacency matrix, capturing the topological features between skeletal points related to different types of gait features. Then, these topological features are aggregated with the corresponding channel high-level features to form the output feature Z.

[0057] Furthermore, each unit (C1, C2) of the channel topology module of the present invention also includes a temporal convolutional network (TCN module). The input of the channel topology graph convolutional network (TCN module) is the fused feature X, and the output and the fused feature X are used together as the input of the temporal convolutional network (TCN module). The input and output of the temporal convolutional network (TCN module) are concatenated to obtain the output feature Z.

[0058] The Channel Topology Graph Convolutional Network (CTCN module) is a key component of this invention, mainly composed of three functional parts: channel topology modeling, feature transformation, and channel feature aggregation; given its input as fused features X∈R... M ×C Output feature Z'∈R N×C It is given by the following formula:

[0059] Z'=I(B(X),Y'(G(X),E'))

[0060] Where, I∈R N×NThis refers to the adjacency matrix mentioned earlier, which is the matrix representation of the edge set E. It records the strength or weight of node connections numerically. Therefore, I is a fixed adjacency matrix based on the human skeletal structure, defining the fixed connections between joints in the skeletal graph. This adjacency matrix is ​​"static" because it does not change with samples or dynamic features, but is a fixed encoding of predefined skeleton topological relationships. Static shared topology refers to using a fixed adjacency matrix I to represent the relationships between nodes. This topological structure remains unchanged throughout the training and inference process. It defines a fixed connection pattern for channels or nodes in the network, meaning that it does not dynamically change with input samples when calculating channel topological graph convolutions. This topological structure can be understood as a basic framework that characterizes the interrelationships between channels.

[0061] Both B(X) and G(X) represent feature transformation operations on the input, serving as basic convolutional layers or feature extraction modules responsible for generating preliminary feature representations. Their function is to extract different feature dimensions from the input, providing a foundation for subsequent topology modeling and feature fusion. G(X) represents an alternative feature transformation method to B(X), complementing B(X). It can be used to capture different feature dimensions.

[0062] E' is a fixed feature matrix or parameter set, independent of the input fusion features X, and may contain graph structure properties or other inherent information of the model. Y' is a fusion function, Y'(G(X),E') representing the fusion of G(X) and E', meaning that E' is no longer a simple set of edges, but an additional, static feature or parameter used to combine with G(X) to generate the final result. By integrating the output features of G(X) and the external information contained in E', Y'(G(X),E) generates a new feature representation. This representation is high-dimensional and contains joint information from both, used for further topology modeling or feature classification.

[0063] The functions are described in detail below.

[0064] Channel topology modeling function:

[0065] Channel topology modeling is the most important part of graph convolutional networks. This invention uses a static adjacency matrix I to realize the topology of all channels. Specifically, a series of linear transformations are performed on the input fused features X to obtain the specific channel correlations K, and then K is introduced into the static adjacency matrix I to finally obtain the refined channel topology Y∈R. N×N×C Here, the channel topology Y is an actual tensor representing the topological relationship between channels, which is a specific feature matrix generated through specific calculations.

[0066] Before calculating the correlation K of a specific channel, two linear transformation operations Γ and Φ are performed first to reduce the complexity of the model and save computational costs. Furthermore, since there is no fixed expression for the correlation between channels in existing studies, the correlation modeling function G() is used to simulate the channel correlation between vertices. Assume there exists a pair of vertices (v... i ,v j Their corresponding features are (x) i ,x j If ), then the modeling function can be expressed as:

[0067] G(Γ(x i ),Φ(x j ))=σ(Γ(x i )-Φ(x j ))

[0068] Where σ(·) is the activation function, Γ(x) i ) represents the vertex v i Feature x i The output after applying the linear transformation, Φ(x) j ) represents the vertex v j Feature x j The output after applying the linear transformation; Γ(x) i ) and Φ(x j Affine transformations of the vertex features are performed using different weight matrices and biases, as follows:

[0069] Γ(x i ) = W Γ x i +b Γ

[0070] Φ(x j ) = W Φ x j +b Φ

[0071] Among them, W Φ and W Γ Both are weight matrices, b Φ and b Γ All are bias vectors.

[0072] Essentially, x can be calculated when a pair of vertex features are linearly transformed twice by Γ and φ. i and x j The distance along the channel dimension. Then, the distance is non-linearly transformed, and the results of these transformations can be used to characterize vertex pairs (v...). i ,v j The channel-specific topological relationships between them.

[0073] In the process of nonlinear transformation, distance is not used directly for representation. Instead, a mapping function converts distance values ​​into weights or representations with certain nonlinear relationships. This nonlinear transformation can capture the complex patterns in the distance and map them to a space that is more suitable for describing the topological relationships between vertices.

[0074] The method for nonlinearly transforming distance according to the present invention is as follows:

[0075] First, calculate the Euclidean distance between vertex pairs:

[0076] d ij =||x i -x j ||2

[0077] Then, d is activated by the activation function σ(). ij Perform nonlinear mapping

[0078] w ij =σ(d ij )

[0079] The purpose of the nonlinear transformation here is to amplify or suppress certain distance patterns, mapping distances from the linear space to the nonlinear space, making the weights of the approximate relationships more consistent with the actual topological relationships. The value w after the nonlinear transformation... ij As v i With v j The correlation weights of specific channels between them are used to construct a dynamic topology matrix or represent the topological relationships of these vertex pairs. After obtaining the topological relationships based on the modeling function, the channel dimension is increased using a linear transformation ξ to obtain the correlation K∈Y of specific channels. N×N×C' , can be represented as:

[0080] k ij ={ξ(G(Γ(x)} i ),Φ(x j ))} i,j∈{1,2,...,N}

[0081] Where j ij ∈R C′ It is a vector belonging to K. ij Its function is to reflect the vertex pairs (v i ,v j The channel-specific topological relationship between vectors k. ij This can be viewed as a representation of the correlation between vertex pairs. By calculating k... ij Able to capture v i and v jThe channel correlation between them. ξ is an operation used to perform a linear transformation, the main purpose of which is to map the result of the nonlinear transformation to a higher channel dimension. ξ receives the correlation value calculated after the transformation by Γ and Φ, and then performs a linear mapping on the input correlation value, which can be written in the form of:

[0082] ξ(G(Γ(x i ),Φ(x j )))=W·G(Γ(x i ),Φ(x j ))+b

[0083] Where W is the weight matrix, which determines how to extend the low-dimensional relation values ​​to the high-dimensional channel space. b is the bias vector used to adjust the transformed values. Through linear transformation ξ, the values ​​based on Γ(x) are transformed... i ) and Φ(x j The relational results (obtained through the modeling function G) are further mapped to higher channel dimensions. Together, they complete the feature transformation process from the original vertex features to channel-specific topological relationships, providing richer information representation for subsequent graph convolutions.

[0084] Note that the vector k in K is not required to be symmetric, so k ij ≠k ji This property increases the flexibility of related modeling and allows for handling a wider range of topological relationships. By allowing asymmetry, it is possible to more accurately depict the connection and association patterns between different channels, thus better modeling the channel correlations between vertex pairs. Finally, the adjacency matrix I is refined using the channel correlation K, resulting in the shared channel topology Y as follows:

[0085] Y = γ(K,I) = I + a·K

[0086] Here, γ(K,I) represents a combination operation function that combines the static adjacency matrix I and the specific channel correlation K. This function adjusts the original static adjacency matrix I to not only consider the static topology but also dynamically incorporate channel correlation information, thus obtaining a new channel topology Y. 'a' is an adjustable scaling factor used to control the degree of influence of the channel correlation matrix K on the final topology Y. The larger 'a' is, the greater the contribution of channel correlation to the final result; the smaller 'a' is, the more the result depends on the static adjacency matrix I.

[0087] Feature conversion function:

[0088] When modeling the channel topology, a feature transformation function is used to transform the fused feature X into a higher-level feature. Its expression is:

[0089]

[0090] That These are the transformed features, W∈R N×C′ This is the weight matrix. Other transformations can also be used here, such as a multilayer perceptron.

[0091] Channel feature aggregation function:

[0092] In obtaining refined channel topology Y and high-level features Next, feature aggregation is achieved through channels. That is, based on the channel topology and features, feature summarization and representation enhancement are performed at each channel dimension. Specifically, the channel topology graph convolution constructs a channel graph for each channel, and the channel topology graph convolution uses the refined topology Y of each channel. c ∈R N×N and advanced features Construct a channel graph for each channel, where Y c This corresponds to the c-th channel, where c ∈ {1,...,C′}. Each channel map reflects the vertex relationships under a specific type of motion feature. Therefore, feature aggregation is performed on each channel map, and the final output Z is obtained by concatenating the output features of all channel maps, which is formulated as:

[0093]

[0094] Here, || represents the concatenation operation. Throughout the process, the inference of the channel-specific correlation K depends on the input samples. Therefore, the proposed channel topology graph convolution is a dynamic graph convolution that adapts to different input samples. For each channel c, the channel-specific correlation component can be extracted from the channel correlation matrix and combined with the static topology to obtain the refined topology Y of that channel. c This means that the edge weights (representing the strength of the topological relationship) between vertex pairs have been determined. The graph structure of each channel can be simply understood as a "weighted adjacency matrix", i.e., Y. c This matrix directly defines the graph structure under this channel; the relationships and weights between vertices are all determined by Y. c Characterization. Through Y c The set of vertices V and the set of edges E are defined for each channel. The vertices are fixed. The weights of the edges come from Y. c Element value: If Y c If [i,j]>0, then there exists a path from v i to v j The weighted edge with weight Y c [i,j].

[0095] The calculation formula for Z does not directly show that the inference of channel-specific correlation K depends on the input sample. However, the calculation process of channel-specific correlation K reveals that different input samples lead to different K values, thus affecting the entire inference process. Therefore, the proposed channel topology graph convolution is a dynamic graph convolution that adapts to different input samples.

[0096] Furthermore, to aid in learning features in both spatial location and channel dimensions, this invention designs an SC-Attention mechanism based on spatial and channel dimensions to improve the model's expressive and generalization abilities. In this invention, the main function of the SC-Attention block is to perform channel and spatial weighting on the input features respectively, obtaining channel features and spatial features, and then adding the spatial features to the channel features as the output features. The specific implementation principle and method are as follows:

[0097] The SC-Attention block multiplies channels containing important information by larger weights to obtain channel features. Specifically, channel information is extracted from the input features, and channel weights are adaptively generated through an attention mechanism. Each channel weight represents the importance of each channel position in the feature map. The generated channel weights are multiplied by the input features, i.e., attention is applied to the input features. The resulting channel features have the same number of input channels, each with only one pixel, whose pixel value represents the weight, i.e., the importance of each channel. Due to the broadcast mechanism, all elements on the same input channel (one element representing one position) are multiplied by the same channel weight, ultimately resulting in a set of channel feature maps adjusted for channel importance. These maps retain information from key channels while reducing the influence of redundant features.

[0098] Similarly, the SC-Attention block multiplies spatial features containing important information by larger weights to obtain spatial features. Specifically, it extracts spatial location information from the input features and generates spatial weights with dimensions N×1×H×W through an attention mechanism. The spatial weights represent the importance of each spatial location in the feature map. Multiplying the generated spatial weights by the input features, i.e., performing attention calculations on the input features, results in spatial features with the same dimension. Since the dimension of the spatial weights matches the height and width of the input features, the broadcast mechanism weights the feature values ​​at each spatial location, thus highlighting information from certain important locations and weakening the influence of unimportant regions. After weighting, a new set of feature maps is obtained, where the weights of each spatial location have been adjusted according to their importance. This set of weighted feature maps reflects the importance distribution of the input features in the spatial dimension, providing a more discriminative spatial information representation for subsequent feature fusion.

[0099] Then, the spatial features are added to the channel features to obtain the final output features, as shown in the following expression:

[0100] X out =W channel ⊙X in +W pixel ⊙X in

[0101] Where X in ∈R N×C×H×W W represents the input features, specifically the feature map before channel weighting and spatial weighting. chann ∈R N ×C×1×1 W represents the channel weight. pixel ∈R N×1×H×W ⊙ represents spatial weights, and ⊙ represents tensor broadcast multiplication.

[0102] It's worth noting that the SC-Attention block can adaptively adjust the importance of different channels, highlighting key features and suppressing redundant information, thereby enhancing the model's focus on important features. On the other hand, the spatial attention mechanism helps the model assign different weights to different spatial locations in the feature map to capture important spatial structure and contextual information. This integrated attention mechanism enables the SC-Attention block to extract more discriminative and discriminative features, thus improving the model's performance in gait recognition tasks. With fewer parameters, it effectively reduces the model's computational complexity and storage requirements.

[0103] Step 4: After passing the output feature Z through a pooling layer, gait recognition is performed using a fully connected layer.

[0104] In summary, this invention first uses a three-branch input network to process the gait sequence, and then performs non-shared topology modeling. This modeling parameterizes the adjacency matrix I as the topology of the graph, while providing general correlations between joints and inferring the correlations of channels from each sample, representing the degree of correlation between skeletal points within each channel. In addition, this invention introduces an SC-Attention block to calculate the importance of each spatial location and channel to better capture key features, thereby improving the accuracy and robustness of the model.

Claims

1. A gait recognition method based on a three-branch input network and channel topology graph convolution, characterized in that, include: Step 1: Obtain the human skeleton sequence and construct a skeleton map with joints as vertices and bones as edges. The joint movement speed is obtained through the sequential time relationship, where For the set of vertices, Let be the set of edges. The feature set of the vertices; Step 2: Using joint position, joint velocity, and bone position as the three inputs to the three-branch network, feature extraction is performed separately. The three output features are then concatenated to obtain the fused features. In the three-branch input network, each branch consists of a BN layer and three convolutional layers M1, M2, and M3 connected in sequence. M1 is a basic block, including a spatial convolution and a temporal convolution, used to extract preliminary features. M2 adds a bottleneck block to M1 to reduce computational complexity and retain important information. M3 adds a residual structure and an SC-Attention block to M2 to focus on more important features. Step 3, using the fusion features As input to the channel topology module, channel features and spatial features are extracted and transformed into high-level features. This captures the topological features between joints under different types of gait correlations. The topological features are then aggregated with the corresponding channel high-level features to obtain the output features. ; Step 4, output features After the pooling layer, a fully connected layer is used for gait recognition.

2. The gait recognition method based on a three-branch input network and channel topology graph convolution as described in claim 1, characterized in that, The skeletal diagram , N For the number of joints, E Represented as an adjacency matrix Its elements Reflects the first Joint With the Joint The correlation strength, The neighborhood is represented as , The features are represented as , The form is to include all N Each joint C A matrix of dimensional features.

3. The gait recognition method based on a three-branch input network and channel topology graph convolution as described in claim 1, characterized in that, The channel topology module includes two cascaded units, each of which includes a channel topology graph convolutional network. The two units have different numbers of channels, and the second unit adds an SC-Attention block.

4. The gait recognition method based on a three-branch input network and channel topology graph convolution as described in claim 1 or 3, characterized in that, The SC-Attention block performs channel and spatial weighting on the input features to obtain channel features and spatial features, and adds the spatial features and channel features as the output features.

5. The gait recognition method based on a three-branch input network and channel topology graph convolution as described in claim 3, characterized in that, Each unit of the channel topology module also includes a temporal convolutional network, the input of which is the fused feature. Output features With fusion features Both are used as input to the temporal convolutional network, and the input and output of the temporal convolutional network are concatenated to obtain the output features. .

6. The gait recognition method based on a three-branch input network and channel topology graph convolution as described in claim 5, characterized in that, The channel topology graph convolutional network consists of three functional parts: channel topology modeling, feature transformation, and channel feature aggregation. Its input is fused features Output features It is given by the following formula: in, Represents a fixed adjacency matrix based on the human skeletal structure, which is a set of edges. E The matrix representation of, and Both represent performing feature transformation operations on the input, extracting different feature dimensions from the input respectively; It is a fixed feature matrix or parameter set, independent of the input fused features. , It is a fusion function. Indicates will The results and These features, when combined, generate a new feature representation.

7. The gait recognition method based on a three-branch input network and channel topology graph convolution as described in claim 6, characterized in that, The channel topology modeling function is implemented as follows: By fusing features of the input Obtain the correlation of a specific channel by performing a series of linear transformations. Then Introduced into the adjacency matrix In the end, a refined channel topology is obtained. ; The feature transformation function is implemented as follows: When modeling the channel topology, a feature transformation function is used to transform the input fused features. To convert to a higher-level feature, the expression is as follows: That These are the transformed features. It is a weight matrix; The channel feature aggregation function: Based on the refined channel topology Obtain the refined topology for each channel. ,use and Construct a channel graph for each channel to aggregate features in a channel-based manner.

8. The gait recognition method based on a three-branch input network and channel topology graph convolution as described in claim 7, characterized in that, The specific channel correlation It can be obtained through the following method: First, for the corresponding feature is ( A pair of vertices () , Perform linear transformation and ; Secondly, calculation and Distance along the channel dimension ; Then, Perform a nonlinear transformation to obtain the value As and The association weights of specific channels between them are used to construct a dynamic topology matrix or represent the topological relationship between vertex pairs; Finally, using linear transformation Increasing the channel dimension yields the correlation of specific channels. K , is represented as: in It belongs to The vector reflects the vertex pair ( , The specific topological relationship between the channels. No forced symmetry, For modeling functions, , It is an activation function.

9. The gait recognition method based on a three-branch input network and channel topology graph convolution as described in claim 8, characterized in that, The Introduced into the adjacency matrix The refined channel topology is obtained in , is represented as: ;in, a It is an adjustable scaling factor used to control For the final result The extent of the impact.

Images