Gait recognition method based on GaitSTGAN

Abstract

The invention relates to the technical field of gait recognition, in particular to a gait recognition method based on a GaitSTGAN, and the method comprises the steps: obtaining a human body posture sequence; performing feature extraction on the human body posture sequence by using an IDE module; the EGAIM module is used for carrying out gait space feature extraction on features output by the IDE module; the stacked MTGAIM module is used for carrying out gait space-time feature extraction on the output features of the EGAIM module under different time scales; and inputting the output feature vectors of the plurality of stacked MTGAIM modules into the GAP and the FC, and then outputting an embedded vector, thereby completing the construction of the GaitSTGAN model. The problem that the existing model gait recognition precision is insufficient is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of gait recognition, and in particular to a gait recognition method based on GaitSTGAN. Background Art

[0002] Gait recognition is an important task in the field of human recognition. Compared with biometric technologies such as face, fingerprint and iris, gait recognition does not require the active cooperation of the target and can capture its characteristics at a long distance.

[0003] Currently, the methods for gait recognition through video are mainly divided into two categories: appearance-based method and model-based method; the appearance-based method uses video image processing technology to extract the human body contour from the video to learn gait characteristics; the model-based method recognizes by estimating the human body structure (such as key points or 3D skeleton).

[0004] The GaitTR model introduced Transformer into the field of gait recognition for the first time. By constructing the overall spatial relationship between key points and combining the temporal convolutional network to extract the spatiotemporal features of gait, the model achieved higher accuracy than previous model-based methods on the CASIA-B dataset, and even surpassed some appearance-based methods when walking in a coat. The GPGait model proposed a unified posture representation (HOD) and used PAGCN to extract the local-global relationship between key points, thereby improving the gait recognition ability. However, the gait recognition accuracy of these two models needs to be further improved. Summary of the invention

[0005] In view of the shortcomings of the existing methods, the present invention constructs the GaitSTGAN model, which improves the recognition ability of the model under complex conditions and scenarios by mining the long- and short-time features of time series signals and the local and global topological features of spatial signals; constructs the GAIM fusion graph convolution local structure extraction capability and the Transformer global feature modeling capability, effectively captures the long-distance relationship and local structure information in space, enhances the model's ability to characterize the topological relationship between key points, and improves the model's ability to recognize complex gait patterns; constructs EGAIM and MTGAIM modules. Based on the GAIM module, the EGAIM module adds the ability to efficiently fuse information between channels, further optimizing the integration and expression of features; the MTGAIM module enhances the analysis ability of data at different time scales, thereby being able to identify short-term and long-term dynamic changes in gait data.

[0006] The technical solution adopted by the present invention is: a gait recognition method based on GaitSTGAN comprises the following steps:

[0007] Step 1: Obtain human posture sequence;

[0008] As a preferred implementation of the present invention, HRNet is used to estimate the human body posture sequence.

[0009] Step 2: Use the IDE module to extract features from human posture sequences;

[0010] As a preferred implementation of the present invention, feature extraction includes: joint feature vectors, bone feature vectors and speed feature vectors.

[0011] Step 3: Use the EGAIM module to extract the spatial features of gait from the output features of the IDE module;

[0012] As a preferred embodiment of the present invention, the EGAIM module includes:

[0013] The feature vector X1 is input into Dropout, 1x1 Conv and BatchNormal&Mish operations in sequence, and then added to X1 to obtain the feature vector X 11 ;X 11 After the GAIM module, the feature vector X is obtained 12 ; Eigenvector X1 and eigenvector X 12 Perform the addition operation to obtain the feature vector X2.

[0014] Step 4: Use the stacked MTGAIM modules to extract the spatiotemporal features of gait at different time scales using the output features of the EGAIM module;

[0015] As a preferred embodiment of the present invention, the MTGAIM module includes: the feature vector X2 is sequentially input into four convolution channels, the first convolution channel is 1x1conv and 5x1 Temporal Conv with a dilation rate of 1, the second convolution channel is 1x1conv and 5x1 Temporal Conv with a dilation rate of 2, the third convolution channel is 1x1conv, Maxpool and 3x1conv, and the fourth convolution channel is 1x1conv, the feature vectors of the four convolution channels are concat, and then added to the feature vector X2 to obtain the feature vector X 21 ;X 21 Input GAIM module to get the feature vector X 22 ;X 22 Add it to X2 to output the feature vector X3.

[0016] As a preferred embodiment of the present invention, the GAIM module includes three branches. The first branch is composed of BatchNormal, Multi-headAttention, Concat&Mish and BatchNormal in series, the second branch is composed of GCLayer, BatchNormal&ReLU and Max Pooling in series, and the third branch is the input feature vector. The three branch vectors are added to output the feature vector.

[0017] Step 5: Input the output feature vectors of several stacked MTGAIM modules into GAP and FC and then output the embedding vector, thus completing the construction of the GaitSTGAN model;

[0018] As a preferred embodiment of the present invention, a triplet loss function is used to guide the parameter optimization of the GaitSTGAN model.

[0019] As a preferred embodiment of the present invention, a gait recognition system based on GaitSTGAN includes: a memory for storing instructions executable by a processor; and a processor for executing the instructions to implement a gait recognition method based on GaitSTGAN.

[0020] As a preferred embodiment of the present invention, a computer-readable medium stores a computer program code, and the computer program code implements a gait recognition method based on GaitSTGAN when executed by a processor.

[0021] Beneficial effects of the present invention:

[0022] 1. The present invention constructs the GaitSTGAN model of EGAIM and MTGAIM. The core of the model is to construct GAIM. GAIM can capture both long-distance relationships and local structural information in space by enhancing the representation of topological relationships between key points, thereby significantly improving the ability to recognize complex spatial patterns in gait data;

[0023] 2. Based on GAIM, the EGAIM module further improves the efficient fusion capability of information between channels and optimizes the integration and expression of features;

[0024] 3. The MTGAIM module successfully identified short-term and long-term dynamic changes in gait data by analyzing data at different time scales, ensuring a comprehensive understanding of temporal information;

[0025] 4. Experimental results verify the superior performance of GaitSTGAN on multiple datasets. Through the GAIM module and its extended EGAIM and MTGAIM modules, GaitSTGAN effectively improves the ability to capture and analyze the spatiotemporal features of gait data, demonstrating strong adaptability and generalization. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a structural diagram of the GaitSTGAN model based on the present invention;

[0027] Figure 2 This is a visual comparison chart of GaitSTGAN and GaitTR on the CASIA-B dataset;

[0028] Figure 3 It is a box plot of the original bone data of the dataset CASIA-B and GREW. DETAILED DESCRIPTION

[0029] The present invention is further described below in conjunction with the accompanying drawings and embodiments. This figure is a simplified schematic diagram, which only illustrates the basic structure of the present invention in a schematic manner, and therefore it only shows the components related to the present invention.

[0030] like Figure 1 As shown, a gait recognition method based on GaitSTGAN includes the following steps:

[0031] GaitSTGAN includes: IDE (Input data Extraction, input data extraction module), EGAIM (Enhanced Graph Attention Integration Module, enhanced graph attention aggregation module) and MTGAIM (Multi-scale Temporal Graph Attention Integration Module, multi-scale temporal graph attention aggregation module);

[0032] Figure 1 In the figure, GAIM is the graph attention aggregation module (light green part), A is the adjacency matrix, GC Layer is the graph convolution layer, GAP is the global average pooling, and FC is the fully connected layer.

[0033] Step 1: Obtain human posture sequence;

[0034] The human posture sequence can be estimated by HRNet. The HRNet network significantly improves the accuracy of key point detection in high-precision posture estimation by maintaining high-resolution feature maps. The dataset obtained by HRNet for 2D human posture estimation is used.

[0035] HRNet can obtain human pose sequences by referring to the literature Deep high-resolution representation learning for human pose estimation by Sun et al.

[0036] The extracted data is input into the GaitSTGAN model (the part in the lavender box in the figure); the GaitSTGAN model is composed of the EGAIM module and the stacked MTGAIM module.

[0037] Step 2: Use the IDE module to extract features from human posture sequences;

[0038] Specifically include:

[0039] Step 11, extracting three bone features, namely joint feature, bone feature and velocity feature, as input;

[0040] The specific operations are as follows:

[0041] Given the original skeleton joint X of the human body, the joint features include the original joint coordinate information X[:,:,i] and the relative joint position X[:,:,i]-X[:,:,i center ], that is, the position difference of each joint relative to the specified center joint.

[0042] For bone features, the bone features are described by calculating the position difference between each pair of connected joints, X[:,:,i]-X[:,:,i adj ]; where i adj represents the adjacent joints of the i-th joint.

[0043] The speed feature includes two aspects: one is the speed feature X[,t+1,]-X[,t,] between each time step and the previous time step, that is, the speed of the joint moving in time; the other is the speed change feature X[,t+2,]-X[,t,] between each time step and the previous time step, that is, the acceleration change of the joint in time.

[0044] Step 3: Use the EGAIM module to extract the spatial features of gait from the output features of the IDE module;

[0045] By joint feature X J , bone features X B and speed characteristic X V , let the input feature vector of the EGAIM module be X1;

[0046] The EGAIM module includes: the feature vector X1 is sequentially input into Dropout, 1x1 Conv and BatchNormal&Mish operations, and then added to X1 to obtain the feature vector X 11 ;X 11 After the GAIM module, the feature vector X is obtained 12 ; Eigenvector X1 and eigenvector X12 Perform addition operation to obtain feature vector X2;

[0047] The GAIM module consists of three branches. The first branch is composed of BatchNormal, Multi-headAttention, Concat&Mish and BatchNormal in series. The second branch is composed of GC Layer, BatchNormal&ReLU and Max Pooling in series. The third branch is the input feature vector. The three branch vectors are added together to output the feature vector. For example, the third branch is the input feature vector X 11 , the output characteristic vector of the three branch addition operation is X 12 .

[0048] The EGAIM module first fuses the information between channels of the input data through convolution to optimize the feature representation of the input data, and then further extracts the local-global spatial features of the gait through the GAIM module. The construction of GAIM includes two important components: multi-head self-attention mechanism (Multi-headAttention) and graph convolution (GC Layer). The multi-head self-attention mechanism is used in deep learning to dynamically adjust the weights of features to generate more flexible and dynamic feature representations. This mechanism significantly enhances the model's ability to focus on important features and improves its ability to capture global information to a certain extent. However, the design of the attention mechanism usually relies on the calculation of global information, and may not be as effective as dedicated local convolution operations when processing local information. Graph convolution is known for its ability to efficiently capture local graph structural information and can provide a more stable local connection feature representation. By combining the multi-head self-attention mechanism with graph convolution, the GAIM module can achieve effective fusion of global and local information while modeling global features, taking advantage of the advantages of graph convolution in local information processing. This combination strategy provides a balance between flexibility and stability for the model's expressiveness, thereby promoting more comprehensive feature representation learning.

[0049] The effectiveness of the GAIM module was verified in the experimental results, showing that the model combined with graph convolution is significantly better than the model that only uses multi-head self-attention. These experimental results fully demonstrate the advantages of the strategy of fusing global and local information in improving the expressiveness of the model.

[0050] The multi-head self-attention mechanism is described as follows:

[0051] Given an input x∈R n , C channels, T frames, and V nodes, first calculate the query vector for each head i of the total head H Key vector Value vector The method is to separate x and the parameter matrix and Then apply the scaled dot product attention function to each q, k, v:

[0052]

[0053] Finally, the results of the h heads are connected in parallel and linearly projected to get the final output in, is the projection matrix;

[0054] z=Concat(head1,head2,...,head H )W (2)

[0055] In this work, the spatial self-attention module of SpatialTransformer is used, and the input is a skeleton sequence C channel, T frame and V joint; in the spatial self-attention module, the attention function contains the correlation between different nodes, namely:

[0056]

[0057] For the dimensions of query, key, and value vectors, d q =d k =f k ×d model , d v =f v ×d model , where d model is the number of output channels of the spatial self-attention module, f k and f v is a fixed factor.

[0058] The description of graph convolution is as follows:

[0059] Given an input X and an adjacency matrix A, first calculate the output of the graph convolution, the formula is as follows:

[0060] x′=X·A i (4)

[0061] Then for each graph convolution output x i ′Apply the convolution operation:

[0062] y i =Conv i (x i ′) (5)

[0063] Among them, Conv i is the i-th convolution operation in the graph convolution layer; finally, the results of all graph convolution operations are accumulated:

[0064]

[0065] Where num_A is the number of key points.

[0066] Step 4: Use the stacked MTGAIM modules to extract the spatiotemporal features of gait at different time scales using the output features of the EGAIM module;

[0067] The number of stacked MTGAIM modules is N, which can be customized.

[0068] The input of the MTGAIM module is the feature vector X2; the MTGAIM module includes: the feature vector X2 is sequentially input into four convolution channels, the first convolution channel is 1x1conv and 5x1 Temporal Conv with a dilation rate of 1, the second convolution channel is 1x1conv and 5x1 Temporal Conv with a dilation rate of 2, the third convolution channel is 1x1conv, Maxpool and 3x1conv, and the fourth convolution channel is 1x1conv. The feature vectors of the four convolution channels are concat, and then added to the feature vector X2 to obtain the feature vector X 21 ;X 21 Input GAIM module to get the feature vector X 22 ;X 22 Add it to X2 to output the feature vector X3.

[0069] The MTGAIM module includes multi-scale temporal convolution and GAIM module; multi-scale temporal convolution can capture feature information of different time scales through multi-branch temporal convolution, maximum pooling and 1x1 convolution technology;

[0070] The specific process is as follows:

[0071] Given an input tensor X∈R C×T×V (i.e., eigenvector X2), the tensor will be processed by three different branches;

[0072] (1) Multi-branch temporal convolution: It contains two temporal convolution branches. Each temporal convolution branch first passes through a 1x1 convolution layer, then goes through batch normalization and ReLU activation function, and finally passes through a TemporalConv layer with a specific expansion rate. For the i-th temporal convolution branch, the processing flow can be expressed by the following formula:

[0073] Xi=TemporalConvd i (ReLU(BN(Conv1×1(X)))) (7)

[0074] Among them, di is the expansion rate corresponding to the i-th branch.

[0075] (2) Max pooling branch: This branch first processes the input through a 1x1 convolutional layer and a batch normalization layer, and then applies the ReLU activation function; then it performs a max pooling operation and performs batch normalization on the pooled result. The formula is as follows:

[0076] X max =BN(MaxPool2d(ReLU(BN(Conv 1×1 (X))))) (8)

[0077] (3) 1x1 convolution branch: The formula is as follows:

[0078] X 1×1 =BN(Conv 1×1 (X)) (9)

[0079] Finally, the output results of these three branches are concatenated in the channel dimension to obtain the final result of multi-scale temporal convolution.

[0080] Step 5: Input the output feature vectors of several stacked MTGAIM modules into GAP and FC and then output the embedding vector, thus completing the construction of the GaitSTGAN model;

[0081] The GaitSTGAN model outputs embedding, which is an embedding vector. The embedding vector is the key information predicted by the GaitSTGAN model and is used for similarity calculation during GaitSTGAN model evaluation; that is, in the inference stage, the Euclidean distance between the two input embedding vectors is used to distinguish different objects.

[0082] The triplet loss function is used to guide the parameter optimization of the GaitSTGAN model.

[0083] Loss function: Using the triplet loss function, for the sample triplet (a, p, n), a represents the anchor point, p is the positive object of the same class as the anchor point, and n is the negative object. The triplet loss is defined as:

[0084] L triple =max(d(f a ,f p )-d(f a ,f n )+margin,0)

[0085] Among them, fa represents the feature vector of the anchor point, d(f a ,f p) represents the Euclidean distance between the feature vectors of a and p; the loss margin is set to 0.3, and the batch-hard method means that for each sample a, the positive sample p with the largest distance d(a,p) and the negative sample n with the smallest distance d(a,n) in the batch are selected.

[0086] Dataset

[0087] The CASIA-B dataset is a large multi-view database established by the State Key Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences. The present invention uses HRNet to extract posture data from RGB videos according to the CASIA-B release protocol. The dataset contains 124 subjects, capturing gait data from 11 different viewpoints. It considers three types of changes, namely viewpoint changes (0°-180°), clothing changes, and changes in the items carried by the subjects. Each view of each person has 10 gait sequences under three different walking conditions: 6 sequences for normal walking (NM), 2 sequences for carrying a bag (BG), and 3 sequences for walking with a bag (BG). Two experimental conditions were set, namely single view and cross view. Following the settings of most studies, in the cross view experiment, the first 24, 62 and 74 objects were used as training sets, denoted as small sample (ST), medium sample (MT) and large sample (LT). In the testing phase, the first four sequences under the NM condition were used as the gallery set (NM#1-4), the last two sequences under the NM condition (NM#5-6), the two sequences under the BG condition (BG#1-2), and the two sequences under the CL condition (CL#1-2) to make three probe subsets.

[0088] GREW is a wild dataset containing 3,500 hours of data from 882 cameras; all data are collected in natural environments, covering various challenging factors of actual scenes, such as different perspectives, occlusions, lighting changes, etc.; the GREW dataset is divided into training set, validation set and test set, containing 20,000, 345 and 6,000 subjects respectively; during testing, two sequences of each subject are regarded as probes, and the other two sequences are regarded as library sets.

[0089] The details of the two data sets are shown in Table 1:

[0090] Table 1: Data types, perspective changes, and challenging factors of different datasets

[0091]

[0092] Among them, #Id., #Seq. and #Cam. represent the number of subjects, the number of sequences and the number of cameras respectively; Sil, Flow represent silhouette and optical flow; VI, DIS, BA, CA, DR, OCC, ILL, and SU are abbreviations of view, distractor, background, carrying, dressing, occlusion, illumination and surface.

[0093] Experimental process:

[0094] The GaitSTGAN of the present invention is composed of EGAIM and MTGIM stacked together. The configuration of GaitSTGAN is shown in Table 2, including the output dimension and the number of parameters; Block0 is an EGAIM block, and Block2-Block3 are four stacked MTGAIM blocks with different channels.

[0095] Table 2: Configuration overview of GaitSTGAN

[0096]

[0097] Enhancement: For the CASIA-B dataset, we first apply an inverse operation by swapping the coordinates of the left and right parts of the skeleton, e.g. Lnose and x Rnose ; Then, the skeleton data is mirrored left and right to increase the diversity of the data; after that, Gaussian noise is added to each joint, and the same Gaussian noise is added to all joints in a gait sequence; finally, a continuous joint sequence of length 60 is randomly selected; for the GREW dataset, the data sequence is flipped, and the probability of flipping the sequence is 0.5.

[0098] The Adam optimizer is used, and the weight decay is set to 2e-5. The training data is sampled in batches. The batch size of the CASIA-B dataset is (4,64), which means that each batch contains 4 people and each person has 64 gait sequences. The batch size of the GREW dataset is (32,8). The OneCycleLR learning rate scheduling strategy is adopted. The initial, maximum and final learning rates are set to 1e-5, 1e-3 and 1e-6 respectively.

[0099] Results and Analysis:

[0100] Compared with the skeleton-based model: (1) Single view:

[0101] In Table 3, the Rank-1 accuracy of GaitSTGAN in a single view of the CASIA-B dataset is shown, as well as a comparison with existing skeleton-based gait recognition methods, including GaitGraph, GaitGraph2, GaitTR, and GPGait. The results show that the GaitSTGAN model achieves advanced performance, outperforming recent skeleton-based methods in almost all conditions, and even outperforming in some cases. In particular, under the condition of walking in a coat, the accuracy of the GaitSTGAN model is 24.71% higher than that of the GaitGraph2 model. Table 3 clearly shows the excellent performance of the GaitSTGAN model in a single view. The UMAP technique is also used to visualize the results of GaitSTGAN and GaitTR. Figure 3 It can be clearly seen that the model of the present invention performs well in distinguishing individual features; specifically, the feature point groups of most people are clearly separated, which shows that the model of the present invention has high accuracy and robustness in extracting and distinguishing individual gait features; in addition, the points under different conditions have a higher degree of overlap, which shows that the model of the present invention is more inclined to extract the common features of the model; in contrast, the feature point groups of different people in the GaitTR model have a certain degree of overlap and confusion, showing that the feature distinction between individuals is unclear, and the points under different conditions of the same person have more obvious differences.

[0102] In general, the model of the present invention shows better performance in the visualization results, proving its advantages in the gait recognition task. In order to further explore the generalization of the model of the present invention, the GREW dataset, which is quite different from the CASIA-B dataset, is used for experiments. The difference between the two datasets is tested by the KS statistic on a few samples in the two datasets. Among them, KS Statistic: 0.175, P-Value: 8.68e-14 (α = 0.05), and box plots of the two datasets are also drawn. Figure 3 As shown, through these two methods, it can be clearly seen that there are significant differences between the two data sets. In this case, the model of the present invention still achieves good performance on the GREW data set, which is 1.42% higher than the second best model GPGait, indicating that the model of the present invention has good generalization.

[0103] Table 3: Performance comparison between the proposed method and existing methods on the GREW dataset and CASIA-B

[0104]

[0105] Table 3 shows the Rank-1 accuracy comparison of the skeleton-based methods on the GREW dataset and the CASIA-B dataset, excluding the case of the same view; the results of the GaitGraph, GaitGraph2, GaitPT and other models are also shown for comparison.

[0106] Figure 2 The visualization results of GaitSTGAN and GaitTR on the CASIA-B dataset are shown below. There are three conditions, each of which contains two cases. 1 means the first element is predicted correctly, and 0 means it is predicted incorrectly. Figure 3 : Box plots drawn from the original skeleton data of the datasets CASIA-B and GREW.

[0107] (2) Cross-perspective:

[0108] In Table 4, the Rank-1 accuracy and mAP values ​​of the GaitSTGAN model and other models compared in single-view conditions in cross-view conditions are shown; unlike the Rank-1 accuracy that reflects the performance of the model in the most accurate prediction, the mAP value reflects the overall performance of the model in the entire task; Table 3 clearly shows that the GaitSTGAN model achieves advanced performance under most cross-view and detection conditions; first, in the case of LT, except for the NM condition, the Rank-1 accuracy and mAP of the model of the present invention are 1.09% and 0.13% lower than the best performing models GPGait and GaitTR, and are the highest in the other two conditions, 2.36% and 1.76%, 1.91% and 0.4% higher than the next model; then in the MT case, in terms of Rank-1 accuracy, compared with GPGait, although GaitSTGAN is 1.69% lower in the NM condition, it is better in the BG and CL conditions in the NM condition, but are 2.35% and 13.71% higher than GaitTR respectively; compared with GaitTR, GaitSTGAN is 0.22% lower under NM condition, but 1.83% and 2.78% higher under BG and CL conditions respectively; in terms of mAP value, GaitSTGAN is 0.54% and 1.04% lower than GaitTR under NM and CL conditions respectively, but performs best under BG condition, surpassing all other models; indicating that GaitSTGAN shows significant advantages under complex background and occlusion conditions, although it is slightly inferior under normal walking conditions; finally, in the ST condition, although the Rank-1 accuracy of the model of the present invention under the three conditions did not reach the best state, being 7.5%, 5.78% and 1.68% lower than the best performing model respectively, its mAP value is the highest among all models; indicating that although it is slightly inferior in Rank-1 accuracy, the model of the present invention is still superior to other models in overall performance.

[0109] Table 4: Rank-1 accuracy of the present invention and the existing methods on the CASIA-B dataset

[0110]

[0111] Comparison with silhouette-based models: We compare the best results achieved under LT conditions with those of the most advanced silhouette-based gait models, as shown in Table 5.

[0112] Table 5: Comparison of GaitSTGAN and silhouette-based models

[0113]

[0114] Ablation experiment: (1) Effectiveness of combining graph convolution:

[0115] In order to demonstrate the effectiveness of the GAIM module combining graph convolution and multi-head self-attention mechanisms, the experiments shown in Table 6 were conducted on the dataset CASIA-B to verify this effectiveness. The experiments show that a single graph convolution or multi-head self-attention mechanism cannot express the spatial characteristics of gait well. The accuracy of the combination of the two is 2.31% and 1.8% higher than that of a single graph convolution and a single multi-head self-attention mechanism on average.

[0116] Table 6: Performance comparison between GCN and MHSA

[0117]

[0118] (2) Effectiveness of the EGAIM module:

[0119] In order to prove whether it is necessary to integrate and optimize the input data and to verify its effectiveness, the experiment shown in Table 7 was conducted on the dataset CASIA-B. The experiment shows that directly using the MTGAIM module to extract the spatiotemporal features of gait and directly using the EGAIM module are not effective. The combination of the two has a higher accuracy, which is 5.15% and 1.6% higher on average, respectively.

[0120] Table 7: Performance comparison between EGAIM and MTGAIM

[0121]

[0122] The present invention constructs the GaitSTGAN model of EGAIM and MTGAIM. The core of the model is to construct GAIM. GAIM can simultaneously capture long-distance relationships and local structural information in space by enhancing the topological relationship representation between key points, thereby significantly improving the ability to recognize complex spatial patterns in gait data.

[0123] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, and its technical scope must be determined according to the scope of the claims.

Claims

1. A gait recognition method based on GaitSTGAN, characterized in that: The following steps are involved: Step 1: Obtain human posture sequence; Step 2: Use the IDE module to extract features from human posture sequences; Step 3: Use the EGAIM module to extract the spatial features of gait from the output features of the IDE module; Step 4: Use the stacked MTGAIM modules to extract the spatiotemporal features of gait at different time scales using the output features of the EGAIM module; Step 5: Input the output feature vectors of several stacked MTGAIM modules into GAP and FC and output the embedding vector to complete the construction of the GaitSTGAN model.

2. The gait recognition method based on GaitSTGAN according to claim 1, characterized in that: The EGAIM module includes: The feature vector X1 is sequentially input into Dropout, 1x1 Conv, and BatchNormal&Mish operations and then added to X1 to obtain the feature vector X 11 ;X 11 After the GAIM module, the feature vector X is obtained 12 ; Eigenvector X1 and eigenvector X 12 Perform the addition operation to obtain the eigenvector X2.

3. The gait recognition method based on GaitSTGAN according to claim 1, characterized in that: The MTGAIM module includes: the feature vector X2 is sequentially input into four convolution channels, the first convolution channel is 1x1conv and 5x1 Temporal Conv with a dilation rate of 1; the second convolution channel is 1x1conv and 5x1 Temporal Conv with a dilation rate of 2; the third convolution channel is 1x1conv, Maxpool and 3x1conv; the fourth convolution channel is 1x1conv, the feature vectors of the four convolution channels are concat, and then added with X2 to obtain the feature vector X 21 ;X 21 Input GAIM module to get the feature vector X 22 ;X 22 Add it to X2 and output the feature vector X3.

4. The gait recognition method based on GaitSTGAN according to claims 2 and 3, characterized in that: The GAIM module consists of three branches. The first branch is composed of BatchNormal, Multi-headAttention, Concat&Mish and BatchNormal in series. The second branch is composed of GC Layer, BatchNormal&ReLU and Max Pooling in series. The third branch is the input feature vector, which adds the three branch vectors and outputs the feature vector.

5. The gait recognition method based on GaitSTGAN according to claim 1, characterized in that: The triplet loss function is used to guide the parameter optimization of the GaitSTGAN model.

6. The gait recognition method based on GaitSTGAN according to claim 1, characterized in that: The human body posture sequence is estimated using HRNet.

7. The gait recognition method based on GaitSTGAN according to claim 1, characterized in that: Feature extraction includes: joint feature vector, bone feature vector and velocity feature vector.

8. The gait recognition system based on GaitSTGAN is characterized by: include: a memory for storing instructions executable by a processor; A processor, configured to execute instructions to implement the GaitSTGAN-based gait recognition method as described in any one of claims 1 to 7.

9. A computer readable medium storing a computer program code, characterized in that: When the computer program code is executed by a processor, the computer program code implements the GaitSTGAN-based gait recognition method as described in any one of claims 1 to 7.

Images