Infrared image human gait recognition method in complex scene based on improved ViT

Abstract

The application discloses an infrared image human gait recognition method in a complex scene based on an improved ViT, relates to the technical field of image recognition, and comprises the following steps: a data preprocessing step, a heterogeneous transfer learning weight preparation step, a model training step and a model testing step. The application can recognize the infrared human image gait in a complex scene and can solve the influence of factors such as rain, snow, fog or insufficient visible light sources on gait recognition.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method for recognizing human gait in complex scenes using infrared images based on an improved ViT (SD Vision Transformer). Background Technology

[0002] Human gait recognition, as one of the most promising long-range biometric identification technologies, can identify pedestrians' identity information using low-resolution medium-to-long-range gait video images without requiring the cooperation of the person being identified. Compared with biometric identification technologies such as face and fingerprint recognition, which have relatively strict identification conditions, it has many advantages such as not requiring the cooperation of the person being identified, non-intrusive identification, and being difficult to hide and forge, and has emerged as a dark horse in the field of identity recognition.

[0003] Currently, most gait recognition methods operate under visible light conditions, but they struggle to perform effectively in special environments such as insufficient visible light, rain, snow, and heavy fog. Current gait recognition algorithms primarily utilize Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs). The former, when learning gait features, does not differentiate between posture features at different body positions, treating them as a whole and failing to consider that the human walking process is a feature set encompassing both temporal and spatial domains, resulting in low recognition accuracy. The latter, while incorporating temporal features and using correlation coefficient functions to calculate gait cycle size with gait cycle sets as input, still treats the human gait as a whole for feature learning, limiting its recognition performance.

[0004] Therefore, current gait recognition methods suffer from drawbacks such as overly idealized dataset selection, algorithm models that do not closely reflect human thought processes, and a failure to effectively learn pose features at different locations. Thus, proposing a human gait recognition method based on improved ViT in complex scenarios using infrared images to address these limitations is a problem urgently requiring solutions from those skilled in the art. Summary of the Invention

[0005] In view of this, the present invention provides a method for human gait recognition in complex scenes using infrared images based on an improved ViT model, wherein the improved ViT model is the SD Vision Transformer model, in order to solve the above-mentioned technical problems.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] The method for human gait recognition in complex scenes based on improved ViT includes the following steps:

[0008] S101. Data preprocessing steps: preprocess the CASIAC infrared human gait dataset, estimate the capacity of gait cycle groups, and segment the data according to gait cycle groups to obtain the test dataset and training dataset;

[0009] S201. Steps for preparing weights for heterogeneous transfer learning: Train the Vision Transformer model on the ImageNet-21K dataset, and then freeze and match the weight parameters of each layer obtained from the training.

[0010] S301. Model training steps: Initialize the constructed SD Vision Transformer model, introduce the weights obtained in S201, load the training dataset to train the model, and obtain the trained SD Vision Transformer model.

[0011] S401. Model testing steps: Input the test dataset into the trained SD Vision Transformer model to perform gait recognition of infrared human images in complex scenes.

[0012] Optionally, S101. Data preprocessing specifically includes the following steps:

[0013] S1011. Obtain the CASIAC infrared gait database;

[0014] S1012. Perform mean model background subtraction on infrared human images, extract human contour features using background subtraction, and display the gait information of the subject in the center to obtain gait images;

[0015] S1013. The zero-mean normalized cross-correlation coefficient is used to estimate the similarity of the gait images in S1012 in order to measure the capacity of the gait cycle group.

[0016] S1014. Use a sliding window method to divide the gait images to obtain gait cycle groups, and divide all gait cycle groups into training datasets and test datasets according to a certain segmentation ratio.

[0017] Optional, the specific content of S1012 is as follows:

[0018] First, the mean background subtraction method is used. The mean value of the same position is calculated for N consecutive infrared gait images in a continuous image sequence, and the obtained pixel mean value is used as the background model. Then, the difference operation is performed one by one. Finally, the moving target and the background are segmented by threshold comparison. Then, the human gait features are displayed by binarization operation and redundant background cropping operation.

[0019] Optionally, the background image at time t is as follows:

[0020]

[0021] The binary image of the moving target extracted at time t is as follows:

[0022]

[0023] In the formula, I t (i,j) represents the video frame at time t, T represents the segmentation threshold, and (i,j) represents the coordinates of each pixel in the image of that frame.

[0024] Optionally, the formula for calculating the zero-mean normalized cross-correlation coefficient in S1013 is as follows:

[0025]

[0026] In equation (3), (x,y) are the pixel coordinates in the image, f(x,y) are the pixel values ​​of the original image, t(x,y) are the pixel values ​​of the template image, n is the number of pixels (elements) in the template, and μ f μ t σf and σt are the pixel mean values ​​of the original image and the template image, respectively, and σf and σt are the pixel standard deviations of the original image and the template image, respectively.

[0027] Optionally, S301. Model training includes the following steps:

[0028] S3011. Initialize the constructed SD Vision Transformer model, introduce the weights obtained in S201, load the training dataset, and input the images of each time step in the gait cycle group in sequence;

[0029] S3012. Divide the image at a certain time into equal-sized segments. Cut the entire image into equal-sized image blocks according to the grid. Reconstruct each segmented image block into a one-dimensional tensor and add position embedding. The position order is from left to right and from top to bottom.

[0030] S3013. Input the one-dimensional tensor with added position embedding into the multi-head attention mechanism module, and then fit the feature weights through the feature averaging fusion module;

[0031] S3014. Update the weights of the SD Vision Transformer model;

[0032] S3015. Determine whether the training dataset has been loaded. If yes, proceed to S3016. If no, return to S3011 to continue loading the training dataset.

[0033] S3016. Obtain the trained SD Vision Transformer model.

[0034] Optionally, S401. Model testing includes the following steps:

[0035] S4011. Input the test dataset into the trained SDVision Transformer model in a way that preserves the information in the time domain and the spatial domain, and extract the gait features in the time domain and the spatial domain.

[0036] S4012. Compare the obtained gait features in the time domain and spatial domain to perform gait recognition of infrared human images in complex scenes, and obtain the recognition test results.

[0037] As can be seen from the above technical solution, compared with the prior art, the present invention provides a method for human gait recognition in complex scenarios based on improved ViT using infrared images: 1) Infrared video gait images are used as the data research background, and gait feature recognition is performed by utilizing the imaging characteristics of the human body using an infrared camera, without over-reliance on visible light sources. To a certain extent, this solves the problem of the impact of insufficient visible light intensity, rain, snow, and fog on the gait feature recognition effect.

[0038] 2) Regarding gait period estimation, the traditional method of calculating gait period using normalized cross-correlation coefficients and absolute differences is abandoned. Instead, zero-mean normalized cross-correlation coefficients are used as the basis for gait period estimation. This method is more sensitive to infrared gait profile features and has a moderate computational cost, representing a trade-off between performance and computational complexity. Furthermore, using gait period groups as input blocks preserves the temporal characteristics of gait, making the gait model more robust.

[0039] 3) An independent feature subspace and a symmetric dual attention mechanism are used to fit gait features, and heterogeneous transfer learning is employed to accelerate the fitting of small sample gait datasets:

[0040] First, unlike previous gait CNN and RNN algorithms that learn gait features holistically, this approach involves dividing gait images into grids, adding corresponding positional embeddings, and then using independent feature subspaces to learn pose features for different body positions. This makes gait feature learning more targeted while preserving the positional relationships between individual parts and the whole to the greatest extent possible.

[0041] Secondly, a symmetrical dual attention mechanism is used to fit gait data. This method can enhance the attention effect on dense parts of human posture and thus weaken the negative impact of irrelevant feature regions on feature fitting, resulting in more human-like and efficient learning of gait features.

[0042] Finally, transfer learning is used to improve feature fitting efficiency, which can significantly reduce model training time even on small gait datasets. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0044] Figure 1 A flowchart of the infrared image human gait recognition method in complex scenes based on improved ViT provided by the present invention;

[0045] Figure 2 The flowchart of the data preprocessing step S101 provided by the present invention;

[0046] Figure 3 S301. Flowchart of model training steps provided by the present invention;

[0047] Figure 4 S401. Flowchart of model testing steps provided by the present invention;

[0048] Figure 5 A flowchart illustrating the infrared image human gait recognition method for complex scenes based on improved ViT provided by this invention.

[0049] Figure 6 A symmetric dual attention mechanism gait model structure diagram provided by the present invention;

[0050] Figure 7 A diagram illustrating the comparison results of different processing methods;

[0051] Figure 8 The block diagram of the infrared image human gait recognition system in complex scenarios based on the improved ViT provided by the present invention is shown. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] Reference Figure 1 and Figure 5As shown, this invention discloses a method for human gait recognition in complex scenes using infrared images based on an improved ViT, comprising the following steps:

[0054] S101. Data preprocessing steps: preprocess the CASIAC infrared human gait dataset, estimate the capacity of gait cycle groups, and segment the data according to gait cycle groups to obtain the test dataset and training dataset;

[0055] S201. Steps for preparing weights for heterogeneous transfer learning: Train the Vision Transformer model on the ImageNet-21K dataset, and then freeze and match the weight parameters of each layer obtained from the training.

[0056] S301. Model training steps: Initialize the constructed SD Vision Transformer model, introduce the weights obtained in S201, load the training dataset to train the model, and obtain the trained SD Vision Transformer model.

[0057] S401. Model testing steps: Input the test dataset into the trained SD Vision Transformer model to perform gait recognition of infrared human images in complex scenes.

[0058] Furthermore, refer to Figure 2 As shown, S101. Data preprocessing specifically includes the following steps:

[0059] S1011. Obtain the CASIAC infrared gait database;

[0060] S1012. Perform mean background subtraction on infrared human images, extract human contour features using background subtraction, and display the gait information of the subject in the center to obtain gait images;

[0061] S1013. The zero-mean normalized cross-correlation coefficient is used to estimate the similarity of the gait images in S1012 in order to measure the capacity of the gait cycle group.

[0062] S1014. Use a sliding window method to divide the gait images to obtain gait cycle groups, and divide all gait cycle groups into training datasets and test datasets according to a certain segmentation ratio.

[0063] Furthermore, the specific content of S1012 is as follows:

[0064] First, the mean background subtraction method is used. The mean value of the same position is calculated for N consecutive infrared gait images in a continuous image sequence, and the obtained pixel mean value is used as the background model. Then, the difference operation is performed one by one. Finally, the moving target and the background are segmented by threshold comparison. Then, the human gait features are displayed by binarization operation and redundant background cropping operation.

[0065] Furthermore, the background image at time t is as follows:

[0066]

[0067] The binary image of the moving target extracted at time t is as follows:

[0068]

[0069] In the formula, I t (i,j) represents the video frame at time t, T represents the segmentation threshold, and (i,j) represents the coordinates of each pixel in the image of that frame.

[0070] Furthermore, in S1013, the zero-mean normalized crosscorrelation (ZNCC) is used as the basis for gait cycle estimation. That is, a certain human gait is first selected as the starting state of walking, and then the ZNCC function is used to calculate the correlation coefficient between each picture and the starting state in turn. From the periodic change pattern of the correlation coefficient, the change pattern of gait cycle can be calculated, and then the average value of the cycle is used as the gait cycle group capacity.

[0071] The gait cycle is defined as follows: taking a certain moment of walking posture as the initial state, the next moment when the correlation coefficient with the initial state is highest is determined as the initial state of the next cycle. Therefore, the image between two initial states is considered as one gait cycle.

[0072] The gait cycle referred to here is, in actual operation, a single-step gait cycle. That is, if the left foot steps out as the initial state, when calculating the correlation coefficient, it is found that the zero-mean normalized cross-correlation coefficient between the right foot step out time and the left foot step out time is very high, so it is regarded as a single-step gait cycle.

[0073] The formula for calculating the zero-mean normalized cross-correlation coefficient is as follows:

[0074]

[0075] In equation (3), (x,y) are the pixel coordinates in the image, f(x,y) are the pixel values ​​of the original image, t(x,y) are the pixel values ​​of the template image, n is the number of pixels (elements) in the template, and μ f μt σf and σt are the pixel mean values ​​of the original image and the template image, respectively, and σf and σt are the pixel standard deviations of the original image and the template image, respectively.

[0076] Optionally, in S1014, the dataset is divided according to gait cycle groups. Since the gait dataset is relatively small, a sliding window method is used to divide the gait dataset to obtain more gait cycle groups, thereby expanding the dataset. The traditional single-foot step is no longer used as the starting state of the gait cycle, making the division of gait cycle groups richer and more diverse.

[0077] The sliding window method refers to defining a window size and considering the area within the window as a gait cycle. As the window slides forward frame by frame, different gait cycle groups are obtained. Specifically, in continuous gait images of the same subject, the posture at each moment can serve as the starting state of a gait cycle, and the moment most recently this state is reached is taken as the ending state. Note that the sliding window method always slides within the same subject's data. It stops sliding when the window reaches the last image of that subject's data, ensuring that the gait cycle group contains only the walking posture of the same subject and preventing the mixing of postures from other subjects, which would lead to data saturation.

[0078] After segmenting the gait cycle groups, all gait cycle groups are divided into training and testing datasets according to a 7:3 segmentation ratio.

[0079] Optionally, S301. Model training includes the following steps:

[0080] S3011. Initialize the constructed SD Vision Transformer model, introduce the weights obtained in S201, load the training dataset, and input the images of each time step in the gait cycle group in sequence;

[0081] S3012. Divide the image at a certain time into equal-sized segments. Cut the entire image into equal-sized image blocks according to the grid. Reconstruct each segmented image block into a one-dimensional tensor and add position embedding.

[0082] S3013. Input the one-dimensional tensor with added position embedding into the multi-head attention mechanism module, and then fit the feature weights through the feature averaging fusion module;

[0083] S3014. Update the weights of the SD Vision Transformer model;

[0084] S3015. Determine whether the training dataset has been loaded. If yes, proceed to S3016. If no, return to S3011 to continue loading the training dataset.

[0085] S3016. Obtain the trained SD Vision Transformer model.

[0086] Furthermore, the specific training process for the S301 model is as follows: First, the images at each moment in the gait cycle group are input sequentially to preserve the spatiotemporal pose features. Then, each moment image is segmented into equal-sized patches, dividing the entire image into gridded blocks to segment the poses at different positions during walking (e.g., head, forward arm swing, backward arm swing, leg step, and leg retraction). Each segmented image patch is then reconstructed into a one-dimensional tensor and a position embedding is added to preserve the spatial pose features.

[0087] Subsequently, the one-dimensional tensor with added position embedding is input into the multi-head attention mechanism module. This part adopts a symmetrical architecture, and then the feature weights are fitted through a feature averaging fusion module. The purpose is to enhance the model's "attention" effect on rich pose features by increasing attention weights. The key is to use independent feature subspaces to fit pose features, matching a feature weight fitting matrix for each different human position. This allows each independent feature subspace to specifically learn the pose features of the same spatial domain in each time domain, making feature fitting more targeted and significantly improving both accuracy and fitting speed.

[0088] Finally, the information is processed by the MLP layer, which summarizes the information from each layer to obtain the final classification result.

[0089] Attention mechanisms refer to structures that mimic human visual focus and capture key features of input information. This paper constructs a scaled dot-product attention mechanism, which assigns different attention to different features by building "query vectors," "value vectors," and "key vectors," as expressed by formulas (4), (5), and (6):

[0090]

[0091] X = softmax(f(Q,K)) (5)

[0092] Attention(X,V)=X×V (6)

[0093] In the above formula, Q, K, and V are the constructed query vector matrix, key vector matrix, and value vector matrix, respectively, and K... T Let d be the transpose of matrix K. k This refers to the dimension of K. Here, it is used as a scaling factor to control the excessive influence of the result; Equations (4), (5), and (6) are the calculation steps for scaling the dot product.

[0094] Multi-head attention, building upon the basic attention mechanism, further refines self-attention, enhancing the model's ability to focus on different positions. Each position in the computation process overwrites positions from the previous layer, ensuring the model doesn't only focus on the current input during learning; inputs from other times also influence the current result, thus creating temporal correlations between data. Then, the values ​​of each attention channel are concatenated and connected with W. O The dot product can be expressed by the following formulas (7) and (8):

[0095]

[0096] MultiHead(Q,K,V)=Concat(head1,head2,...,head n W O (8)

[0097] In equation (7) above, This means that the head at the current time step considers not only Q, K, and V at this time step, but also the weight matrices from other time steps. It will have an impact on it; W O This indicates that multiple heads are concatenated and then normalized according to the function's output size.

[0098] Position embedding is added using Positional Encoding. This invention uses fixed-size segmentation blocks for image segmentation. The following sine and cosine methods have similar effects and can both be used. This invention chooses the sine position embedding method, which can better adapt to segmentation blocks of different sizes. As shown in formula (9):

[0099]

[0100] In the formula, pos represents the position sequence number of the token in the global array; i represents the dimension, which is [0,...,d]. model / 2],d model Take 512.

[0101] Alternatively, the cosine position embedding method can be selected, as shown in formula (10):

[0102]

[0103] Furthermore, the S401 model testing includes the following steps:

[0104] S4011. Input the test dataset into the trained SDVision Transformer model in a way that preserves the information in the time domain and the spatial domain, and extract the gait features in the time domain and the spatial domain.

[0105] S4012. Compare the obtained gait features in the time domain and spatial domain to perform gait recognition of infrared human images in complex scenes, and obtain the recognition test results.

[0106] Furthermore, refer to Figure 6 As shown, this invention discloses a specific symmetric dual attention mechanism gait model structure diagram: First, the images at each time step in the gait cycle group are sequentially input into the Embedding layer. Then, the gait image with a size of 128×128 pixels is divided into 32×32 pixel image blocks. Each image block is then reconstructed into a one-dimensional tensor, and after adding positional embeddings, it is input into the EncoderBlock. After passing through a parallel multi-head attention module, pose detail features are extracted from the input images of different human body positions and poses (e.g., arms, legs). Feature weights are then fitted through a feature averaging fusion module, and after LayerNormalization, the model enters the MLP module. Finally, the MLP_Head block is used to obtain the classification result. To prevent the model from overfitting small sample data, DropPath is used instead of traditional Dropout in the symmetric multi-head attention module and MLP module, randomly deleting multi-branch structures in the network. Since ViT is a model structure that processes temporally sequential data, it's crucial to maintain the sequential nature of human walking postures. Therefore, predefined gait cycle groups are used as the model's input data. The image is then segmented into multiple parts according to the corresponding embedding size and input into the encoder in a left-to-right, top-to-bottom order. An ablation experiment is employed, with an initial learning rate of 1×10⁻⁶. -3 The number of Multi_Head Attention points is 12, Adam is used as the optimizer, and the loss is calculated using the categorical_crossentropy multi-class cross-entropy loss function.

[0107] Furthermore, refer to Figure 7 As shown, the recognition performance of the SD Vision Transformer model is significantly improved after introducing transfer learning.

[0108] Reference Figure 8 As shown, this invention discloses an infrared image human gait recognition system for complex scenes based on improved ViT. The system applies the above-mentioned infrared image human gait recognition method for complex scenes based on improved ViT and includes a data preprocessing module, a heterogeneous transfer learning weight preparation module, a model training module, and a model testing module connected in sequence.

[0109] The data preprocessing module is used to preprocess the CASIAC infrared human gait dataset, estimate the capacity of gait cycle groups, and segment the data according to gait cycle groups to obtain the test dataset and the training dataset.

[0110] The heterogeneous transfer learning weight preparation module is used to train the Vision Transformer model on the ImageNet-21K dataset, and then freeze and match the weight parameters of each layer obtained from the training.

[0111] The model training module is used to build the SD Vision Transformer model, introduce the weights obtained from S201, load the training dataset to train the model, and obtain the trained SD Vision Transformer model.

[0112] The model testing module is used to input the test dataset into the trained SD Vision Transformer model to perform gait recognition of infrared human images in complex scenes.

[0113] Furthermore, the present invention also discloses a computer-readable storage medium storing computer instructions for causing a computer to execute the above-described method for human gait recognition in complex scenarios based on improved ViT infrared images.

[0114] Furthermore, a computer-readable storage medium is specifically disclosed.

[0115] The computer-readable storage medium stores computer instructions that cause the computer to execute a human gait recognition method based on improved ViT in complex scenarios using infrared images. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium may also include combinations of the above types of memory.

[0116] The above description of the disclosed embodiments is presented in a progressive manner to enable those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for human gait recognition in complex scenes using infrared images based on improved ViT, characterized in that, Includes the following steps: S101. Data preprocessing steps: Preprocess the CASIAC infrared human gait dataset, estimate the capacity of gait cycle groups, and segment the data according to gait cycle groups to obtain the test dataset and training dataset; S201. Steps for preparing weights for heterogeneous transfer learning: Train the Vision Transformer model using the ImageNet-21K dataset, and then freeze and match the weight parameters of each layer obtained from the training. S301. Model training steps: Initialize the constructed SD VisionTransformer model with symmetric dual attention mechanism architecture, introduce the weights obtained in S201, load the training dataset to train the model, and obtain the trained SD Vision Transformer model. The SD Vision Transformer model is a symmetric dual attention mechanism gait model, including: an embedding layer, an encoder block, a feature averaging and fusion module, a layer normalization layer, a multilayer perceptron module, and a multilayer perceptron head; the encoder block includes a parallel multi-head attention module with a symmetric architecture. The SD Vision Transformer model processing flow is as follows: Images from each moment in the gait cycle group are sequentially input into the embedding layer; the embedding layer divides the input current moment image into equal-sized image blocks according to a preset size, reconstructs each image block into a one-dimensional tensor, and adds position embeddings to each one-dimensional tensor; the one-dimensional tensors with added position embeddings are input into a parallel multi-head attention module to extract detailed features of different human body positions and poses in parallel; the output data of the parallel multi-head attention module is then fitted with feature weights by a feature averaging fusion module; the fitted data is then processed sequentially through a layer normalization layer, a multilayer perceptron module, and a multilayer perceptron head, and the final classification result is output; S401. Model testing steps: Input the test dataset into the trained SD Vision Transformer model to perform gait recognition of infrared human images in complex scenes.

2. The method for human gait recognition in complex scenes based on improved ViT according to claim 1, characterized in that, S101. Data preprocessing specifically includes the following steps: S1011. Obtain the CASIAC infrared gait database; S1012. Perform mean background subtraction on infrared human images, extract human contour features using background subtraction, and display the gait information of the subject in the center to obtain gait images; S1013. The zero-mean normalized cross-correlation coefficient is used to estimate the similarity of the gait images in S1012 in order to measure the capacity of the gait cycle group. S1014. Use a sliding window method to divide the gait images to obtain gait cycle groups, and divide all gait cycle groups into training datasets and test datasets according to a certain segmentation ratio.

3. The method for human gait recognition in complex scenes based on improved ViT according to claim 2, characterized in that, The specific content of S1012 is as follows: First, the mean background subtraction method is used. The mean value of the same position is calculated for N consecutive infrared gait images in a continuous image sequence, and the obtained pixel mean value is used as the background model. Then, the difference operation is performed one by one. Finally, the moving target and the background are segmented by threshold comparison. Then, the human gait features are displayed by binarization operation and redundant background cropping operation.

4. The method for human gait recognition in complex scenes based on improved ViT according to claim 3, characterized in that, The background image at time t is as follows: （1） The binary image of the moving target extracted at time t is as follows: （2） In the formula, This represents the video frame at time t, where T represents the segmentation threshold. This represents the coordinates of each pixel in the image frame.

5. The method for human gait recognition in complex scenes based on improved ViT according to claim 2, characterized in that, The formula for calculating the zero-mean normalized cross-correlation coefficient in S1013 is as follows: （3） In equation (3), These are the pixel coordinates in the image. These are the pixel values ​​of the original image. The pixel values ​​of the template image. This represents the number of pixels (elements) in the template. , These are the pixel mean values ​​of the original image and the template image, respectively. These represent the standard deviations of pixels in the original image and the template image, respectively.

6. The method for human gait recognition in complex scenes based on improved ViT according to claim 1, characterized in that, S301. Model training includes the following steps: S3011. Initialize the constructed SD Vision Transformer model, introduce the weights obtained in S201, load the training dataset, and input the images of each time step in the gait cycle group in sequence; S3012. Divide the image at a certain time into equal-sized segments. Cut the entire image into equal-sized image blocks according to the grid. Reconstruct each segmented image block into a one-dimensional tensor and add position embedding. The position order is from left to right and from top to bottom. S3013. A one-dimensional tensor input multi-head attention mechanism module with added position embedding is added, and then the feature weights are fitted through the feature averaging fusion module; S3014. Update the weights of the SD Vision Transformer model; S3015. Determine whether the training dataset has been loaded. If yes, proceed to S3016. If no, return to S3011 to continue loading the training dataset. S3016. Obtain the trained SD Vision Transformer model.

7. The method for human gait recognition in complex scenes based on improved ViT according to claim 1, characterized in that, S401. Model testing includes the following steps: S4011. Input the test dataset into the trained SD VisionTransformer model in a way that preserves the information in the time domain and the spatial domain, and extract the gait features in the time domain and the spatial domain. S4012. Compare the obtained gait features in the time domain and spatial domain to perform gait recognition of infrared human images in complex scenes, and obtain the recognition test results.

Images