Gait recognition method fusing depth map derived from RGB image and contour sequence

Abstract

The invention belongs to the technical field of gait recognition, and provides a gait recognition method fusing a depth map derived from an RGB image and a contour sequence, and the method fuses a depth map sequence based on the RGB image and a traditional contour map sequence. An estimation depth map is displayed from a given RGB image sequence by using an existing RGB video data set and a latest depth estimation model, and the estimation depth map is used as a new mode to capture inherent distinctive features in human motion; compared with a traditional input mode, the depth map provides clearer 3D geometric information about the human body and the movement of the human body, and gait characterization is enriched; the invention provides a more comprehensive and accurate gait recognition solution; and a relatively good recognition effect is achieved on the basis of a widely used gait recognition reference.

Description

Technical Field

[0001] The present invention belongs to the technical field of gait recognition, and in particular relates to a gait recognition method that fuses a depth map derived from an RGB image and a contour sequence. Background Art

[0002] Gait recognition is an identity verification technology that confirms an individual's identity by analyzing their unique movement patterns while walking. It is often used in public security fields, such as crime investigation and suspect tracking. Compared with traditional biometric technologies such as face, fingerprint and iris recognition, gait recognition has the advantages of being contactless, highly private, difficult to forge, and able to achieve long-distance recognition, so it is widely used in various scenarios.

[0003] At present, there are two main input modalities for gait recognition methods, namely silhouette sequences and skeleton sequences. Silhouette sequences distinguish individuals by retaining the apparent appearance information, while skeleton sequences retain the internal structural information of the human body. Skeleton sequences remain robust when the appearance changes drastically. However, both modalities have certain limitations. Silhouettes are susceptible to significant changes in external body shape caused by changes in clothing, and although skeletons are effective in solving clothing occlusions, they completely ignore the highly discriminative body shape information, resulting in poor performance. At the same time, in order to address the limitations of single modality, recent studies have also explored the possibility of multimodal fusion, thereby improving performance. However, existing solutions still cannot effectively solve the complex problems of real-world scenarios.

[0004] When the research scene shifts from the traditional laboratory scene to the real real-world scene, the traditional gait recognition method is no longer applicable. In recent years, researchers have gradually tried to use new gait input modalities to achieve robust gait recognition effects. The SMPLGait model proposes to use the features extracted from SMPL parameters as the three-dimensional representation of the human body. It uses a dual-branch structure based on a deep neural network. One branch learns appearance features from the human body contour, and the other branch learns 3D viewpoints and shape knowledge from 3D SMPL parameters, thereby using the 3D geometric information of the SMPL model to enhance gait appearance feature learning. However, SMPLGait simply connects the final global features of the two modalities and cannot effectively capture fine-grained spatial information. How to effectively fuse their features and capture and integrate the complex relationship between different gait modalities remains a problem. At the same time, although the SMPL model is usually regarded as a dense grid, its feature vector has only dozens of dimensions, showing a relatively sparse representation of body shape and posture, and the description of gait patterns is not fine-grained enough. The SkeletonGait++ model innovatively introduces the concept of skeleton graph. Unlike the traditional skeleton modality, the skeleton graph is not just a simple representation of the joints, but generates a heat map for each joint point through Gaussian approximation, and the structure is more intuitive. At the same time, the skeleton graph is closer to the traditional image modality in data format, so that the skeleton graph can better combine the advantages of the contour graph modality and has stronger compatibility in the multimodal fusion of gait recognition. Despite the good performance, the skeleton graph modality still only contains 2D gait information, and its performance is still poor when facing challenges such as viewpoint changes. The LidarGait model first proposed the use of 3D point clouds captured by LIDAR sensors for gait recognition. By projecting sparse point cloud data into a depth map and combining it with a deep neural network, the fine-grained features required for gait recognition are extracted from 3D geometric information. However, the mainstream sensors for scenes that require gait recognition are still mainly low-cost cameras, and expensive lidars are not yet popular. In addition, the current gait datasets are mainly RGB video datasets, and point cloud datasets are relatively small.

[0005] In addition, with the development of depth estimation technology, the depth information estimated from RGB images is becoming more and more accurate. Compared with traditional input modalities, depth maps provide clearer 3D geometric information about the human body and its movement, which is not available in contour and skeleton input modalities. Compared with 2D planar information, this additional dimension enriches the expression of gait features. Secondly, depth maps can more accurately analyze subtle gait motion changes, which is crucial for capturing the unique characteristics of an individual's gait. In addition, depth information can help solve the challenges of viewpoint changes because it provides information about body structure at different angles and a more consistent expression of motion information.

[0006] The existing gait recognition method based on fusion contour sequence and SMPL parameter sequence can complete personnel recognition in complex field scenes. However, it does not give full play to the potential of three-dimensional information in gait recognition. After replacing a more powerful backbone network, the 3D branch does not significantly enhance the overall gait recognition performance. This is because in terms of feature fusion, SMPLGait only uses simple element-by-element multiplication and addition operations, which cannot effectively bridge the gap between the two modal features. In addition, the feature vector of the SMPL model has only dozens of dimensions, the representation of body shape and posture is relatively sparse, and the description of gait features is not fine-grained enough.

[0007] The existing gait recognition methods based on skeleton graph sequences and contour graph sequences innovatively introduce the concept of skeleton graphs, and generate a heat map for each joint point through Gaussian approximation, making the skeleton structure more intuitive. And the data format of the traditional image modality is retained. However, the skeleton graph modality only contains 2D planar structure information, which is not enough to express more subtle gait movement changes, and still performs poorly when facing challenges such as viewpoint changes. At the same time, SkeletonGait++ only uses a simple attention fusion operation when performing feature fusion. This is not enough to effectively capture and integrate the complex relationship between modalities.

[0008] The existing gait recognition method based on 3D radar point cloud projects sparse point cloud data into a depth map, and then combines it with a deep neural network to extract the fine-grained features required for gait recognition from 3D geometric information. Although good results have been achieved, the sensors currently used in the main scenarios where gait recognition is required are still mainly low-cost cameras. In addition, the current data sets are mainly RGB video data sets, and point cloud data sets are relatively small.

[0009] In summary, although gait recognition methods based on new input modalities such as SMPL parameters and skeleton graphs have shown certain potential, it is still necessary to comprehensively consider various factors and continuously improve relevant technologies and methods when solving practical scene problems. In particular, how to construct an input modality that can contain more gait information, so as to extract more distinguishing gait features, deeply explore the potential of depth information in gait recognition tasks, and how to effectively integrate the information of two different modalities, depth map and contour map, to obtain a more powerful gait representation, thereby improving the accuracy of gait recognition. Summary of the invention

[0010] In order to solve the above technical problems, the present invention provides a gait recognition method that integrates a depth map derived from an RGB image and a contour sequence to solve the problems in the prior art. The technical solution adopted by the present invention is: A gait recognition method fusing a depth map derived from an RGB image and a contour sequence comprises the following steps: S1: Contour map sequences and depth map sequences are derived from original video clips in public datasets; S2: Perform cropping and normalization operations on the contour map sequence and the depth map sequence; S3: Feed the obtained contour image sequence and depth image sequence to the contour feature extractor and depth image feature extractor in the feature extraction module respectively; extract features from the contour image sequence and the depth image sequence respectively to obtain the contour feature image and deep feature maps , represents the stage of encoding; S4: Feature fusion through cross-level and multi-scale fusion modules; S5: Send the fused features to the feature aggregation module, and generate gait recognition features through feature aggregation through time pooling and horizontal pyramid pooling; S6: Use the features obtained in S5 for training and reasoning; S7: When finally making inferences, the cosine similarity of the features of the probe set and gallery set is compared, and the most similar one is considered as the prediction object.

[0011] Further, step S1 includes: S11: Through the pedestrian segmentation algorithm, the background is set to black, the human body contour is set to white, the contour sequence is represented as S, and the size is , Indicates the number of channels, represents the length of the contour sequence, Indicates the height of each frame image. Indicates the width of each frame image; S12: Use the Depth Anything base model to estimate the depth map from the RGB image.

[0012] Further, step S2 includes: S21: determining the top and bottom positions of non-zero elements in the contour image sequence and the depth image sequence, and cropping the image to delete the background; S22: setting the height of the contour input to 64 pixels, and adjusting the widths of the contour image sequence and the depth image sequence according to the aspect ratio of the human body, so as to keep the aspect ratio of the human body unchanged; S23: Calculate the total number of pixels in the contour image, then calculate the cumulative number of pixels in each column, determine the position where the cumulative number of pixels exceeds half of the total pixels, and designate it as the vertical center of the image; the center of the contour image is set to the center of the depth image; the depth map sequence is denoted as D, and the size is ,in, , , , They represent the number of channels of the depth image, sequence length, image frame height, and image frame width respectively.

[0013] Further, step S4 includes: S41: The multi-scale spatial extraction module connects the contour feature map and the channel feature map through the channel dimension to obtain a unified feature vector , the formula is as follows: ; S42: Perform multi-scale space extraction on the concatenated feature vectors. The formula is as follows: ; in, , Represents a 1×1 convolution kernel; is the local score, Score for the global score; is the relu activation function, It is the BatchNorm2d batch normalization layer; S43: Through attention fusion and The attention weight is calculated by adding them together. The formula is as follows: ; in, express Activation function; S44: Using cross-level fusion to obtain the final gait representation , while obtaining higher-level semantic information, retaining the spatial information contained in the shallow feature map. The formula is as follows: ; in It represents the features extracted at each encoding stage before the cross-level and multi-scale fusion module; the feature fusion process of step S41 to step S44 is applied at each encoding stage.

[0014] Further, step S5 includes: S51: The fused features are subjected to a time pooling operation to aggregate the feature map sequence by maximizing along the time dimension and output a global understanding; S52: Perform horizontal pyramid pooling on the temporally pooled features to obtain multi-scale information of the input features. Specify the number of blocks in the horizontal pyramid as 16. Perform horizontal segmentation on the width dimension of the input features. Then perform average pooling and maximum pooling operations on each block. Concatenate the two pooling results along the last dimension to form the final output features.

[0015] Further, step S6 includes: S61: Linearly map the output features obtained by S5 through 16 independent fully connected layers. Each fully connected layer maps the input features to a new feature space with 256 output channels. The linearly transformed output features are used to train the triplet loss. S62: Perform batch normalization on the features obtained in S61, divide the obtained features into multiple parts, perform batch normalization on each part, and finally use the features obtained to train the cross entropy loss and perform inference; S63: During the training process, the weighted sum of triplet loss and cross entropy loss is used as the loss function. The triplet loss function The formula is as follows: ; in, represents the set of positive sample pairs, represents the set of negative sample pairs, represents the distance between the anchor point and the positive sample, represents the distance between the anchor point and the negative sample pair, It means the boundary value is set to 0.2; Cross Entropy Loss Function The formula is as follows: ; in, is the number of categories, is the one-hot encoding of the true category label, is the class probability predicted by the model; The overall loss function formula is as follows: ; in is the triplet loss, is the cross entropy loss, and is the weighting parameter; S64: The similarity is measured by calculating the cosine similarity between the query feature vector and the library feature vector.

[0016] The present invention has the following beneficial effects: The present invention combines a depth map sequence based on RGB images with a traditional silhouette map sequence. The estimated depth map displayed from a given RGB image sequence is used using an existing RGB video dataset and the latest depth estimation model, and is used as a new modality to capture the distinctive features inherent in human motion. Compared with traditional input modalities, depth maps provide clearer 3D geometric information about the human body and its motion, enriching the gait representation. Secondly, depth maps can more accurately analyze subtle gait motion changes, which is crucial for capturing the unique characteristics of an individual's gait. In addition, depth information can help solve the challenge of viewpoint changes because it provides a more consistent expression of body structure information and motion information from different angles.

[0017] At the same time, in order to facilitate the feature fusion of the two modalities, the present invention also proposes a new cross-level, multi-stage, multi-scale feature fusion network. Through cross-level, multi-scale, multi-stage attention fusion, the interaction between modalities is enhanced, and the gait information between different modalities is captured in a more fine-grained manner, so as to better bridge the gap between the two modalities and achieve more robust gait recognition, thereby providing a more comprehensive and accurate gait recognition solution. And it has achieved good recognition results on the widely used gait recognition benchmark. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is the overall flow chart of the present invention; Figure 2 Flowchart for cropping and normalization of depth and contour maps; Figure 3 This is the feature fusion flow chart. DETAILED DESCRIPTION

[0019] The following will be combined with the embodiments of the present invention Figure 1-Figure 3 , the technical solutions in the embodiments of the present invention are described clearly and completely. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Unless otherwise specified, the technical means used in the embodiments are conventional means well known to those skilled in the art.

[0020] The present invention combines a depth map sequence based on RGB images with a traditional contour map sequence. The estimated depth map displayed from a given RGB image sequence is used as a new modality to capture the distinctive features inherent in human motion using an existing RGB video dataset and the latest depth estimation model. Compared with traditional input modalities, the depth map provides more explicit 3D geometric information about the human body and its motion, enriching the gait representation. Secondly, the depth map can more accurately analyze subtle gait motion changes, which is crucial for capturing the unique characteristics of an individual's gait. In addition, depth information can help solve the challenge of viewpoint changes because it provides a more consistent expression of body structure information and motion information at different angles. At the same time, in order to facilitate the feature fusion of the two modalities, the present invention also proposes a new cross-level, multi-stage, multi-scale feature fusion network. Through cross-level, multi-scale, and multi-stage attention fusion, the interaction between modalities is enhanced, and the gait information between different modalities is captured in a more fine-grained manner, so as to better bridge the gap between the two modalities and achieve more robust gait recognition, thereby providing a more comprehensive and accurate gait recognition solution. And it has achieved good recognition results on the widely used gait recognition benchmark.

[0021] The present invention proposes a gait recognition method that integrates a depth map derived from an RGB image and a contour sequence. The flowchart is as follows: Figure 1 As shown. First, in the data preprocessing stage, depth map sequences and contour map sequences are derived from the sampling frames of the RGB motion video clips. Then, these sequences are fed into the corresponding feature extractors in the encoding module to generate corresponding feature maps. In the cross-level multi-scale fusion module, a multi-scale spatial feature extraction module is designed to extract multi-scale features from the feature map. The obtained multi-scale features are fused with the original feature map in the previous module through attention, and the cross-level fusion module is used to generate the final gait representation. Finally, feature aggregation is performed through pooling operations, and the model is trained using cross entropy loss and triplet loss, with cosine similarity as an indicator for evaluating gait similarity. Specifically including: S1: In the preprocessing stage, a contour map sequence and a depth map sequence are derived from the original video clip in the public dataset. The contour map sequence includes multiple contour maps, and the depth map sequence includes multiple depth maps. The public dataset can be obtained from the Internet.

[0022] S11: In general, existing gait recognition datasets usually have directly processed contour image sequences. Through the classic pedestrian segmentation algorithm, the background is set to black and the human body contour is set to white. The contour sequence is represented as S and has a size of . Indicates the number of channels, represents the length of the contour sequence, Indicates the height of each frame image. Indicates the width of each frame image.

[0023] S12: Use the existing Depth Anything basic model to estimate the depth map from the RGB image. The RGB image is the video frame in the original dataset. Specifically, in order to eliminate the interference caused by background factors, we use the contour as a mask to generate a depth map with only the area of ​​interest. Other depth estimation models can also be used. Then, use the contour as a mask to completely set the background to black to eliminate the interference caused by background factors.

[0024] S2: At the same time, in order to make the obtained contour image sequence and depth image sequence contain more abundant and accurate gait information, the contour image sequence and the depth image sequence are cropped and normalized. Figure 2 shown.

[0025] S21: First, we determine the top and bottom positions of non-zero elements in the contour image sequence and the depth image sequence, and crop the image to remove irrelevant background.

[0026] S22: Then, according to the standard practice of gait recognition, the height of the contour input is set to 64 pixels, and the widths of the contour image sequence and the depth image sequence are adjusted accordingly according to the aspect ratio of the human body, keeping the aspect ratio of the human body unchanged.

[0027] S23: The above process ensures that the horizontal center is located at half the image height, thereby providing a consistent reference frame for all samples. When looking for the vertical center, because the vertical center of the contour image is easier to determine, the vertical center of the contour image is used as the vertical center of the depth image. First, calculate the total number of pixels in the contour image, and then calculate the cumulative number of pixels in each column from left to right. By traversing this cumulative pixel list, determine the position where the cumulative number of pixels exceeds half of the total pixels, and designate it as the vertical center of the image. The center of the contour image found is also set as the center of the depth image. Denote the depth map sequence as D, with a size of .in, , , , They represent the number of channels of the depth map, sequence length, image frame height, and image frame width respectively.

[0028] S3: Then, the obtained contour map sequence S and depth map sequence D are fed into the contour feature extractor and depth map feature extractor in the feature extraction module respectively. We use the feature extractor introduced in DeepGaitV2 as the encoder because it achieves the best performance on common gait recognition datasets. Feature extraction is performed from S and D to obtain feature maps and , Represents the stage of encoding.

[0029] S4: After obtaining the feature maps of the two modalities, feature fusion is performed through the cross-level and multi-scale fusion module. Figure 3 shown.

[0030] S41: First, the multi-scale spatial extraction module obtains a unified feature vector by connecting the contour feature map and the channel feature map along the channel dimension , the formula is as follows: ; This connection enables the subsequent convolution operation to process these features simultaneously, thereby leveraging complementary information from the contour and depth modalities.

[0031] S42: Then, multi-scale space extraction is performed on the concatenated feature vectors. The formula is as follows: ; in, , Represents a 1×1 convolution kernel. is the local score, Score for the whole world. is the relu activation function, The BatchNorm2d batch normalization layer uses convolution kernels of different sizes to capture both fine-grained features and global information. By using this multi-scale approach, the comprehensiveness of feature expression is enhanced to ensure that local details and global context information are preserved. Thus, the local score is obtained. And the global score .

[0032] S43: Finally, through attention fusion and The attention weight is calculated by adding them together. and Fusion is performed to obtain the fusion output of this stage , so that the model can selectively focus on the important parts of the input, thereby improving the effectiveness of gait fusion features. The formula is as follows: ; in, express Activation function.

[0033] S44: In order to enhance the feature expression capability, cross-level fusion is used to obtain the final gait representation While obtaining higher-level semantic information, the rich spatial information contained in the shallow feature map is retained. The formula is as follows: ; in It represents the features extracted at each encoding stage before the cross-level, multi-scale fusion module. At the same time, in order to solve the problem of insufficient information interaction between modalities, the feature fusion process of step S41-step S44 is applied at each stage of encoding. By fusing feature information of different depths layer by layer, the feature representation is gradually optimized, thereby improving the overall performance of the model.

[0034] S5: The fused final features are sent to the feature aggregation module, and feature aggregation is performed through time pooling and horizontal pyramid pooling to generate the final gait recognition features.

[0035] S51: The fused features are first subjected to a time pooling operation to aggregate the feature map sequence by maximizing along the time dimension and output a global understanding.

[0036] S52: Perform horizontal pyramid pooling on the time-pooled features to obtain multi-scale information of the input features. Specify the number of blocks in the horizontal pyramid as 16, perform horizontal segmentation on the width dimension of the input features, and then perform average pooling and maximum pooling operations on each block. By combining average pooling and maximum pooling, the details of local features are retained (maximum pooling) and the statistical information of global features is obtained (average pooling). Finally, the two pooling results are spliced ​​together along the last dimension to form the final output features.

[0037] S6: Use the features obtained in S5 for training and inference.

[0038] S61: The output features obtained by S5 are linearly mapped through 16 independent fully connected layers. Each fully connected layer maps the input features to a new feature space, and the number of output channels is 256. The linearly transformed output features are used to train the triplet loss.

[0039] S62: Perform batch normalization on the features obtained in S61, divide the obtained features into multiple parts, perform batch normalization on each part, and finally use the features obtained to train the cross entropy loss and perform inference.

[0040] S63: During the training process, the weighted sum of triplet loss and cross entropy loss is used as the loss function. The triplet loss function The formula is as follows: ; in, represents the set of positive sample pairs, Represents the set of negative sample pairs. represents the distance between the anchor point and the positive sample, represents the distance between the anchor point and the negative sample pair, Indicates that the boundary value is set to 0.2.

[0041] Cross Entropy Loss Function The formula is as follows: ; in, is the number of categories, that is, the number of identity IDs in the data set, is the one-hot encoding of the true category label, is the class probability predicted by the model, calculated by softmax.

[0042] The overall loss function formula is as follows: ; in is the triplet loss, is the cross entropy loss, and is the weighting parameter.

[0043] S64: Measure the similarity by calculating the cosine similarity between the query feature vector and the library feature vector. Cosine similarity: used to measure how similar the directions of the two vectors are. The closer the value is to 1, the more similar they are; the closer it is to 0, the more different they are.

[0044] S7: When finally making inferences, by comparing the cosine similarity of the features of the probe set and the gallery set, the most similar one is considered the prediction object; for example, if a photo of a person being identified or a walking video is obtained, the image features of the person are extracted, compared with the image features in the gallery set, and the cosine similarity is calculated, and the similar one is considered the prediction object.

[0045] The embodiments described above are only descriptions of the preferred modes of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various deformations, modifications, and substitutions made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A gait recognition method that fuses a depth map derived from an RGB image and a contour sequence, characterized in that: The following steps are involved: S1: Contour map sequences and depth map sequences are derived from original video clips in public datasets; S2: Perform cropping and normalization operations on the contour map sequence and the depth map sequence; S3: Feed the obtained contour image sequence and depth image sequence into the contour feature extractor and depth image feature extractor in the feature extraction module respectively; Feature extraction is performed from the contour map sequence and the depth map sequence to obtain the contour feature map and deep feature maps , represents the stage of encoding; S4: Feature fusion through cross-level and multi-scale fusion modules; S5: Send the fused features to the feature aggregation module, and generate gait recognition features through feature aggregation through time pooling and horizontal pyramid pooling; S6: Use the features obtained in S5 for training and reasoning; S7: When finally making inferences, the cosine similarity of the features of the probe set and gallery set is compared, and the most similar one is considered as the prediction object.

2. The gait recognition method of claim 1 that fuses a depth map derived from an RGB image and a contour sequence, characterized in that: Step S1 includes: S11: Through the pedestrian segmentation algorithm, the background is set to black, the human body contour is set to white, the contour sequence is represented as S, and the size is , Indicates the number of channels, represents the length of the contour sequence, Indicates the height of each frame image. Indicates the width of each frame image; S12: Use the Depth Anything base model to estimate the depth map from the RGB image.

3. The gait recognition method of claim 1 that fuses a depth map derived from an RGB image and a contour sequence, characterized in that: Step S2 includes: S21: determining the top and bottom positions of non-zero elements in the contour image sequence and the depth image sequence, and cropping the image to delete the background; S22: setting the height of the contour input to 64 pixels, and adjusting the widths of the contour image sequence and the depth image sequence according to the aspect ratio of the human body, so as to keep the aspect ratio of the human body unchanged; S23: Calculate the total number of pixels in the contour image, then calculate the cumulative number of pixels in each column, determine the position where the cumulative number of pixels exceeds half of the total pixels, and designate it as the vertical center of the image; the center of the contour image is set to the center of the depth image; the depth map sequence is denoted as D, and the size is ,in, , , , They represent the number of channels of the depth image, sequence length, image frame height, and image frame width respectively.

4. The gait recognition method of claim 1 that fuses a depth map derived from an RGB image and a contour sequence, characterized in that: Step S4 includes: S41: The multi-scale spatial extraction module connects the contour feature map and the channel feature map through the channel dimension to obtain a unified feature vector , the formula is as follows: ； S42: Perform multi-scale space extraction on the concatenated feature vectors. The formula is as follows: ； in, , Represents a 1×1 convolution kernel; is the local score, Score for the global score; is the relu activation function, It is the BatchNorm2d batch normalization layer; S43: Through attention fusion and The attention weight is calculated by adding them together. The formula is as follows: ； in, express Activation function; S44: Using cross-level fusion to obtain the final gait representation , while obtaining higher-level semantic information, retaining the spatial information contained in the shallow feature map. The formula is as follows: ； in It represents the features extracted at each encoding stage before the cross-level and multi-scale fusion module; the feature fusion process of step S41 to step S44 is applied at each encoding stage.

5. The gait recognition method of claim 1 that fuses a depth map derived from an RGB image and a contour sequence, characterized in that: Step S5 includes: S51: The fused features are subjected to a time pooling operation to aggregate the feature map sequence by maximizing along the time dimension and output a global understanding; S52: Perform horizontal pyramid pooling on the temporally pooled features to obtain multi-scale information of the input features. Specify the number of blocks in the horizontal pyramid as 16. Perform horizontal segmentation on the width dimension of the input features. Then perform average pooling and maximum pooling operations on each block. Concatenate the two pooling results along the last dimension to form the final output features.

6. The gait recognition method of claim 1 that fuses a depth map derived from an RGB image and a contour sequence, characterized in that: Step S6 includes: S61: Linearly map the output features obtained by S5 through 16 independent fully connected layers. Each fully connected layer maps the input features to a new feature space with 256 output channels. The linearly transformed output features are used to train the triplet loss. S62: Perform batch normalization on the features obtained in S61, divide the obtained features into multiple parts, perform batch normalization on each part, and finally use the features obtained to train the cross entropy loss and perform inference; S63: During the training process, the weighted sum of triplet loss and cross entropy loss is used as the loss function. The triplet loss function The formula is as follows: ； in, represents the set of positive sample pairs, represents the set of negative sample pairs, represents the distance between the anchor point and the positive sample, represents the distance between the anchor point and the negative sample pair, It means the boundary value is set to 0.2; Cross Entropy Loss Function The formula is as follows: ； in, is the number of categories, is the one-hot encoding of the true category label, is the class probability predicted by the model; The overall loss function formula is as follows: ； in is the triplet loss, is the cross entropy loss, and is the weighting parameter; S64: The similarity is measured by calculating the cosine similarity between the query feature vector and the library feature vector.

Images