Gait recognition method and system based on local and global difference modeling network

Abstract

The present invention discloses a gait recognition method and system based on a local and global difference modeling network. First, a preprocessing operation is performed based on the collected video data to extract the gait profile sequence of the pedestrian target from the video data; then the extracted gait profile sequence is input into the local and global difference modeling network to perform identity recognition based on gait; the local and global difference modeling network comprises: an initial convolution block, a local and global difference modeling unit, a spatiotemporal pooling module, and a classification module. The present invention proposes a feature learning module for local and global difference modeling of gait features, which strengthens the modeling of gait dynamic changes by focusing on the differences between the profiles in the gait sequence, and significantly improves the accuracy and robustness of gait recognition.

Description

Technical Field

[0001] The present invention belongs to the technical field of biometric recognition, and relates to a gait recognition method and system, and specifically to a gait recognition method and system based on a local and global difference modeling network. Background Art

[0002] As an advanced biometric recognition technology, gait recognition has received widespread attention in recent years. It recognizes individual identities by analyzing the body's motion posture during walking. Compared with traditional biometric recognition methods such as fingerprint, iris, and facial recognition, gait recognition has unique advantages. An individual's walking posture is usually difficult to disguise or hide, and gait recognition does not require the cooperation of the person being identified. Therefore, gait recognition has important application value in non-contact, long-distance monitoring and identity authentication. However, the development of gait recognition technology still faces many challenges, especially in complex environments. Since gait characteristics are affected by multiple external conditions such as environmental factors, clothing, carried objects, and walking speed, how to improve the accuracy and robustness of gait recognition under these conditions has become an urgent problem to be solved.

[0003] At present, gait recognition methods are mainly divided into two categories: model-based methods and appearance-based methods. Model-based methods construct gait models by analyzing the human skeleton structure or kinematic features. Such methods can provide relatively accurate recognition results under ideal conditions, especially with good robustness in multi-view and changing environments. However, these methods have high requirements on the quality of input data and require clear images to ensure accurate estimation of human posture. Therefore, in practical applications, model-based methods are usually affected by factors such as background complexity, illumination changes, and noise, resulting in poor quality of input images and thus unsatisfactory recognition results. In addition, model-based methods have high computational complexity and poor real-time performance, which also limits their application in large-scale application scenarios.

[0004] On the other hand, appearance-based methods mainly use the contour information formed by the human body during the gait cycle for recognition. Most early studies used template methods such as Gait Energy Image (GEI) to compress the motion features of the entire gait cycle into a static image. This type of method is simple to calculate and can extract the overall characteristics of the gait. However, the template-based method loses a lot of spatiotemporal detail information during the compression process, and has insufficient recognition ability when dealing with small differences between different individuals. In addition, the template method is sensitive to changes in perspective, occlusion, and changes in wearing objects, which can easily lead to a decrease in recognition accuracy.

[0005] With the rise of deep learning, especially the development of convolutional neural networks, appearance-based gait recognition methods have been significantly improved. Researchers have designed a series of network structures for gait recognition by introducing two-dimensional and three-dimensional convolutions, such as GaitSet, GaitPart, and GaitGL. These methods have better solved the problem of losing detail information in traditional methods by extracting the spatial and temporal features of gait contours. However, despite the excellent performance of deep learning models in extracting gait features, most existing methods still have the following shortcomings:

[0006] (1) Ignoring minor differences between frames: Most existing methods focus on extracting global spatiotemporal features, ignoring the subtle changes between adjacent frames in the gait sequence. In fact, these subtle motion differences are often important clues to distinguish different individuals. These subtle differences become even more critical when dealing with situations where the details vary less, such as when different individuals have similar postures or clothing. Existing methods fail to effectively capture these differences between frames, resulting in insufficient accuracy and robustness in gait recognition.

[0007] (2) Insufficient temporal dependency: Many deep learning-based gait recognition methods fail to fully consider the temporal dependency between frames when processing gait sequences. Although 3D convolutional networks can capture temporal information to a certain extent, they are more sensitive to short-term dynamic changes and cannot accurately reflect the long-term dependencies in the entire gait sequence. As a result, in long-term gait recognition, the model may not be able to effectively aggregate and utilize global gait information, affecting the accuracy of recognition.

[0008] In summary, although existing gait recognition techniques have made significant progress in accuracy and robustness, they still have many shortcomings. In particular, existing methods have not yet provided a satisfactory solution on how to effectively capture and utilize local and global dynamic differences in gait sequences. Summary of the invention

[0009] In order to solve the technical problem of how to effectively capture and utilize local and global dynamic difference features in gait sequences in gait recognition technology, the present invention provides a gait recognition method and system based on a local and global difference modeling network.

[0010] The technical solution adopted by the method of the present invention is: a gait recognition method of a local and global difference modeling network, comprising the following steps:

[0011] Step 1: Perform pedestrian detection, tracking and segmentation on the collected video data to extract the gait profile sequence;

[0012] Step 2: Input the gait profile sequence extracted in step 1 into the local and global difference modeling network to perform local and global difference feature modeling and extract robust gait features;

[0013] The local and global difference modeling network comprises a sequentially connected initial convolution module, N local and global difference modeling units connected in series, a spatiotemporal pooling module and a feature-based classification module; the local and global difference modeling unit comprises a first global sequence modeling module and a local difference modeling module arranged in parallel, a first spatiotemporal alignment module, a second global sequence modeling module and a global difference modeling module arranged in parallel, and a second spatiotemporal alignment module; wherein N is a preset value;

[0014] Step 3: Match the gait features obtained in step 2 with the gait features of known identities in the database to determine the identity information of the gait profile sequence extracted in step 1.

[0015] Preferably, the specific implementation of step 1 includes the following sub-steps:

[0016] Step 1.1: Preprocess the video data collected by the camera, including pedestrian detection, target tracking and pedestrian segmentation, and extract the gait profile sequence A1 of the target pedestrian from the video;

[0017] Step 1.2: crop and align the gait profile sequence A1, and perform sampling in time series to obtain the gait profile sequence A2;

[0018] Step 1.3: Perform spatial data enhancement on the gait profile sequence A2.

[0019] Preferably, in step 2, the initial convolution module is used to transform the gait profile sequence obtained in step 1 Send it to an initialized convolution layer to obtain the initial spatiotemporal feature map represents the preliminary dynamic information of each frame in the gait sequence; where B is the batch size when the local and global difference modeling network is trained, C in is the number of channels of the input contour map, C is the number of channels of the initial feature, T is the number of frames of the feature map, and H and W are the height and width of the feature map, respectively.

[0020] Preferably, in step 2, the local and global difference modeling unit first performs a spatial-temporal feature map output by the initial convolution module. Through the first global sequence modeling module, 3D convolution operation is used to extract global motion features in both spatial and temporal dimensions. ; Through the local difference modeling module, the local difference features between adjacent frames are calculated using the sliding window mechanism ; Then the global motion feature and local difference characteristics Input the first time sequence alignment module to align and fuse the local difference features and the global motion features in the time dimension to generate the aligned and fused gait features ; Then fuse the gait features Input the second global sequence modeling module to calculate the global spatiotemporal features on the entire gait sequence to obtain the global motion features ; The gait features will be integrated The global difference feature modeling module is input to obtain the global difference feature by comparing the difference between each frame and the temporal average representation of the entire sequence. ; Finally, the global motion features and global difference features Input the second temporal alignment module to align and fuse the global difference features and the global gait features in the time dimension to generate the aligned and fused gait features ; Wherein, B is the batch size during the training of the local and global difference modeling network, C is the number of channels of the initial feature, T is the number of frames of the feature map, and H and W are the height and width of the feature map, respectively.

[0021] Preferably, the first global sequence modeling module is composed of a 3D pooling layer, a 3D convolution layer, a BN layer and a ReLU activation function; its working principle is:

[0022] ;

[0023] ;

[0024] in, It is the difference between the current frame and the average timing characteristics of the window in the local sliding window.

[0025] Preferably, the second global sequence modeling module is composed of a 3D convolution layer, a BN layer and a ReLU activation function, and its working principle is:

[0026] .

[0027] Preferably, the first timing alignment module and the second timing alignment module are composed of a feature splicing layer, a convolution layer, a timing convolution layer and a feature addition layer connected in sequence; the feature splicing layer is used to splice the input difference features and sequence features and input them into the convolution layer; the feature addition layer is used to add the output of the timing convolution layer to the input sequence features and then output them; the timing convolution layer includes a 3D convolution layer, a batch normalization layer and a ReLU nonlinear activation function.

[0028] Preferably, in step 2, the input of the first unit of the N local and global difference modeling units connected in series is the output of the initial convolution module, and the input of the remaining units are the output of the previous unit; the spatiotemporal feature map of the gait sequence is obtained. ;in, and are the number of channels, height and width of the spatiotemporal feature map respectively; B is the batch size during training of the local and global difference modeling network, and T is the number of frames of the feature map.

[0029] Preferably, in step 2, the spatiotemporal pooling module processes the spatiotemporal feature graph of the gait sequence. , perform temporal pooling and spatial pooling respectively to obtain the final gait feature representation ; Wherein, B is the batch size when the local and global difference modeling network is trained, is the number of channels of the spatiotemporal feature map.

[0030] Preferably, in step 3, the Euclidean distance is calculated based on the gait features obtained in step 2 and the gait features of known identities in the database, and the identity information of the contour sequence extracted in step 1 is determined based on the Euclidean distance and a preset threshold.

[0031] Preferably, the local and global difference modeling networks are trained networks; a joint loss function is used in the training, and parameters are updated by an SGD optimizer and a gradient descent algorithm;

[0032] The joint loss function is the triple loss and cross entropy loss The joint loss L is:

[0033] ;

[0034] ;

[0035] ;

[0036] in, and Represent the feature vectors of anchor samples, positive samples and negative samples respectively, is a preset interval; is with the correct category y The corresponding weight vector, Yes Category k The corresponding weight vector, K is the total number of categories and z is the input feature vector.

[0037] The technical solution adopted by the system of the present invention is: a gait recognition system based on a local and global difference modeling network, comprising:

[0038] one or more processors;

[0039] A storage device is used to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the gait recognition method based on the local and global difference modeling network.

[0040] Compared with the existing gait recognition methods, the invention has the following advantages and positive effects:

[0041] (1) The present invention effectively captures subtle changes and overall motion patterns in gait sequences by combining local and global difference modeling. The local difference module focuses on extracting small differences between adjacent frames, while the global difference module focuses on the global differences of the entire gait sequence, so that the extracted gait features have both fine local motion information and can reflect the overall gait trend.

[0042] (2) The temporal alignment module in the present invention ensures the consistency of features in the temporal sequence by aligning the difference features and the global sequence features in the temporal dimension. This module not only aligns the feature expressions at different time points, but also combines the difference features and the global sequence features through feature fusion, thereby correcting the errors caused by the motion changes of different gaits in the gait sequence.

[0043] (3) The present invention pays more attention to the motion information in gait through difference modeling, and shows better robustness in complex environments. In particular, the recognition accuracy is significantly improved when the perspective changes and the influence of the worn objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The technical solution of the present invention is further described below using embodiments and specific implementation methods. In addition, in the process of describing the technical solution, some drawings are also used. For those skilled in the art, other drawings and the intention of the present invention can also be obtained based on these drawings without paying creative work.

[0045] Figure 1 A network structure diagram for modeling local and global differences in an embodiment of the present invention;

[0046] Figure 2 A network structure diagram of a local and global difference modeling unit according to an embodiment of the present invention;

[0047] Figure 3 This is a structural diagram of a first timing alignment module and a second timing alignment module according to an embodiment of the present invention;

[0048] Figure 4 Comparison of attention heat maps between the network model of the embodiment of the present invention and other models. DETAILED DESCRIPTION

[0049] In order to facilitate ordinary technicians in the field to understand and implement the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the implementation examples described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0050] Please see Figure 1 , this embodiment provides a gait recognition method based on a local and global difference modeling network, comprising the following steps:

[0051] Step 1: Perform preprocessing operations such as pedestrian detection, tracking and segmentation on the video data collected by the camera to extract the gait profile sequence;

[0052] In one embodiment, the specific implementation of step 1 includes the following sub-steps:

[0053] Step 1.1: Preprocess the video data collected by the camera, including pedestrian detection, target tracking, and pedestrian segmentation, etc., in order to extract the gait profile sequence of the target pedestrian from the video; in this embodiment, the pedestrian detection algorithm uses YOLOX, the pedestrian tracking algorithm uses ByteTrack, and the pedestrian segmentation algorithm uses HumanSegV2. The gait profile sequence is stored in binary form, that is, the background area is 0 and the pedestrian area is 1;

[0054] Step 1.2: The gait profile sequence extracted in step 1.2 is cropped and aligned, and sampled in time series; in this embodiment, all profiles are cropped to a size of 64×44, and the time series sampling method is fixed order sampling;

[0055] Step 1.3: Perform spatial data enhancement on the gait profile sequence obtained in step 1.2, specifically including: random rotation, random erasing, horizontal flipping, perspective transformation, etc. In this embodiment, the probability of various data enhancement operations is set to 0.2.

[0056] Step 2: Input the gait profile sequence extracted in step 1 into the local and global difference modeling network to perform local and global difference feature modeling to extract robust gait features;

[0057] The local and global difference modeling network comprises a sequentially connected initial convolution module, N local and global difference modeling units connected in series, a spatiotemporal pooling module and a feature-based classification module; the local and global difference modeling unit comprises a first global sequence modeling module and a local difference modeling module arranged in parallel, a first spatiotemporal alignment module, a second global sequence modeling module and a global difference modeling module arranged in parallel, and a second spatiotemporal alignment module; wherein N is a preset value;

[0058] In this embodiment, the initial convolution module consists of a 3D convolution layer with a kernel length of 3×3×3 and a step length of 1×1×1; the convolution module converts the gait profile sequence obtained in step 1.2 into As input, used to obtain the initial spatiotemporal feature map , the feature map represents the preliminary dynamic information of each frame in the gait sequence; where B is the batch size when the local and global difference modeling network is trained, C in is the number of channels of the input contour map, C is the number of channels of the initial feature, T is the number of frames of the feature map, H and W are the height and width of the feature map respectively; R is an identifier used to indicate the feature dimension, which is a common parameter in this field; in this embodiment, C is set to 64, H and W are set to 64 and 44 respectively.

[0059] Please see Figure 2 In one embodiment, the local and global difference modeling unit works as follows:

[0060] Step 2.1: The initial spatiotemporal feature map The input is sent to the first global sequence modeling module, which extracts global motion features in both spatial and temporal dimensions through 3D convolution operations. ; The first global sequence modeling module consists of a 3D convolutional layer, a BN layer, and a ReLU activation function. The convolutional layer kernel length is 3×3×3 and the step size is 1×1×1. The calculation formula is as follows:

[0061] ;

[0062] Step 2.2: The initial spatiotemporal feature map The input is sent to the local difference modeling module, which calculates the local difference features between adjacent frames through a sliding window mechanism. , to capture small motion changes within a short-term time series window; the first global sequence modeling module consists of a 3D pooling layer, a 3D convolution layer, a BN layer, and a ReLU activation function. The pooling layer has a kernel length of 3×1×1 and a step size of 3×1×1. The convolution layer has a kernel length of 3×3×3 and a step size of 1×1×1. The calculation formula is as follows:

[0063] ;

[0064] ;

[0065] in, It is the difference between the current frame and the average timing characteristics of the window in the local sliding window.

[0066] Step 2.3: The global motion features obtained in step 2.1 and the local difference features obtained in step 2.2 As the input of the first temporal alignment module, the temporal alignment module aligns and fuses the local difference and global gait features in the temporal dimension to generate the aligned and fused gait features. ;

[0067] Please see Figure 3 In one embodiment, the first time series alignment module is composed of a sequentially connected feature concatenation layer, a convolution layer, a time series convolution layer and a feature addition layer; the feature concatenation layer is used to concatenate the input difference features and sequence features and then input them into the convolution layer; the feature addition layer is used to add the output of the time series convolution layer to the input sequence features and then output them; the time series convolution layer includes a 3D convolution layer, a batch normalization layer and a ReLU nonlinear activation function, and the convolution layer kernel length is 3×3×1 and the step length is 3×1×1. The calculation formula is as follows:

[0068] ;

[0069] Among them, Cat represents the concatenation of features in the channel dimension.

[0070] Step 2.4: The fused gait features obtained in step 2.3 The data is input to the second global sequence modeling module to calculate the global spatiotemporal features on the entire gait sequence to capture the overall gait sequence characteristics. ;

[0071] In one embodiment, the second global sequence modeling module consists of a 3D convolutional layer, a BN layer, and a ReLU activation function, and the convolutional layer kernel length is 3×3×3 and the step length is 1×1×1. The calculation formula is as follows:

[0072] ;

[0073] Step 2.5: The fused gait features obtained in step 2.3 The module is fed into the global difference feature modeling module, which obtains the global difference feature by comparing the difference between each frame and the temporal average representation of the entire sequence. , highlighting the global significant changes in the gait sequence; the first global sequence modeling module consists of a 3D pooling layer, a 3D convolution layer, a BN layer, and a ReLU activation function. The pooling layer has a kernel length of 3×1×1 and a step size of 3×1×1. The convolution layer has a kernel length of 3×3×3 and a step size of 1×1×1. The calculation formula is as follows:

[0074] ;

[0075] ;

[0076] in, is the difference between each frame in the gait sequence and the temporal average representation of the entire sequence.

[0077] Step 2.6: The global motion features obtained in step 2.4 are and the global difference features obtained in step 2.5 As the input of the second temporal alignment module, the temporal alignment module aligns and fuses the global difference features and the global gait features in the temporal dimension to generate the aligned and fused gait features. .

[0078] Please see Figure 3 In one embodiment, the second temporal alignment module is composed of a sequentially connected feature concatenation layer, a convolution layer, a temporal convolution layer, and a feature addition layer; the feature concatenation layer is used to concatenate the input difference features and sequence features and then input them into the convolution layer; the feature addition layer is used to add the output of the temporal convolution layer to the input sequence features and then output them; the temporal convolution layer includes a 3D convolution layer, a batch normalization layer, and a ReLU nonlinear activation function, and the convolution layer kernel length is 3×3×1 and the step length is 3×1×1. The calculation formula is as follows:

[0079] ;

[0080] Among them, Cat represents the concatenation of features in the channel dimension.

[0081] Step 2.7: Repeat N local and global difference modeling units, that is, repeat steps 2.1-2.6 N times; except for the first unit whose input is the output of the initial convolutional block, the inputs of the remaining units are the outputs of the previous unit; obtain the spatiotemporal feature map of the gait sequence ;in, and are the number of channels, height and width of the spatiotemporal feature map, respectively; B is the batch size when the local and global difference modeling network is trained, and T is the number of frames of the feature map. In this embodiment, the number of local and global difference modeling units N is set to 4, and each unit doubles the number of channels of the feature map and halves the spatial size. and They are 512, 16 and 11 respectively.

[0082] Step 2.8: The gait features described in step 2.7 Input the spatiotemporal pooling module to perform temporal pooling and spatial pooling respectively to obtain the final gait feature representation In this embodiment, temporal pooling adopts maximum pooling, and spatial pooling adds the average pooling and maximum pooling results.

[0083] Step 3: Calculate the Euclidean distance between the gait features obtained in step 2 and the gait features of known identities in the database, and determine the identity information of the gait profile sequence extracted in step 1 based on the Euclidean distance and a preset threshold.

[0084] In one embodiment, the local and global difference modeling network is a trained network; during the training process, the final gait feature representation in step 2.8 is It is used to calculate the triplet loss to enhance the discriminative power of the features. Subsequently, these features are sent to the BN layer for normalization and the probability of each ID is calculated by the FC layer for calculating the cross entropy loss.

[0085] The triplet loss function is defined as follows:

[0086] ;

[0087] in, and Represent the feature vectors of anchor samples, positive samples and negative samples respectively, is a preset interval; in this embodiment, Set to 0.2.

[0088] The cross entropy loss is defined as follows:

[0089] ;

[0090] in, is with the correct category y The corresponding weight vector, Yes Category k The corresponding weight vector, K is the total number of categories and z is the input feature vector.

[0091] The final loss function is defined as the joint loss of triplet loss and cross entropy loss:

[0092] ;

[0093] In this embodiment, during the training process, the algorithm model uses the joint loss function and updates parameters through the SGD optimizer and the gradient descent algorithm. The initial learning rate is set to 0.1, and the learning rate gradually decays during the training process.

[0094] This embodiment also provides a gait recognition system based on a local and global difference modeling network, which is characterized by comprising:

[0095] one or more processors;

[0096] A storage device is used to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the gait recognition method based on the local and global difference modeling network.

[0097] The present invention is further described below through experiments:

[0098] The gait recognition method based on dynamic difference modeling proposed in this invention evaluates the performance of the model on the CASIA-B dataset. The CASIA-B dataset contains three scenes: normal walking (NM), backpack (BG) and cross-dressing (CL). The dataset contains a total of 124 pedestrians, each of which has 11 shooting angles (0°, 18°, 36°, ..., 180°). The gait sequence data of the first 74 pedestrians are used as the training set, and the gait sequence data of the last 50 pedestrians are used as the test set. During the test, the gait data with sequence labels NM#01-04 are used as the registration set, and the remaining gait data with labels NM#05-06, BG#01-02 and CL#01-02 are used as the query set.

[0099] Experimental settings: All gait profiles were uniformly cropped to a resolution of 64×44. During training, 8 pedestrians were sampled in each batch, and 16 gait sequences were sampled for each pedestrian, for a total of 128 gait sequences. The optimizer used SGD, the initial learning rate was set to 0.1, and the weight decay was set to 0.0005. The loss function used triple loss and cross entropy loss. A total of 80K batches were trained, and the learning rate was reduced by 10 times when the training batch reached 20K, 40K, and 60K.

[0100] Experimental results: In three different scenarios, namely normal walking (NM), backpack (BG) and cross-dressing (CL), the Rank-1 accuracy rates are shown in the following table:

[0101]

[0102] Figure 4 The comparison of the model focus area between the proposed method and other methods is given. (a) is the input gait sequence data, (b) is the model focus area of ​​other methods, and (c) is the model focus area of ​​the proposed method. It can be seen that the focus areas of other methods are relatively scattered, while the proposed method mainly focuses on the boundary areas of pedestrians. The changes in these areas can reflect the motion model of pedestrians.

[0103] Experiments have shown that the proposed method has achieved excellent recognition results in three gait walking scenarios of the cross-view gait recognition dataset CASIA-B. Specifically, under normal walking conditions, the accuracy of this method is 98.8%; in complex backpack and cross-dressing scenarios, this method still achieved accuracies of 96.7% and 90.2%; the overall average accuracy is: 95.2%. These results verify the robustness and effectiveness of the present invention for pedestrian gait recognition.

[0104] It should be understood that the embodiments described above are part of the embodiments of the present invention, rather than all of the embodiments. In addition, the technical features in the various embodiments or single embodiments provided by the present invention can be combined with each other arbitrarily to form a feasible technical solution. Such combination is not restricted by the sequence of steps and / or the structural composition mode, but must be based on the ability of ordinary technicians in the field to implement. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the protection scope required by the present invention.

[0105] It should be understood that the above description of the preferred embodiment is relatively detailed and cannot be regarded as limiting the scope of patent protection of the present invention. Under the enlightenment of the present invention, ordinary technicians in this field can also make substitutions or modifications without departing from the scope of protection of the claims of the present invention, which all fall within the scope of protection of the present invention. The scope of protection requested for the present invention shall be based on the attached claims.

Claims

1. A gait recognition method based on a local and global difference modeling network, characterized in that: The following steps are involved: Step 1: Perform pedestrian detection, tracking and segmentation on the collected video data to extract the gait profile sequence; Step 2: Input the gait profile sequence extracted in step 1 into the local and global difference modeling network to perform local and global difference feature modeling and extract robust gait features; The local and global difference modeling network comprises a sequentially connected initial convolution module, N local and global difference modeling units connected in series, a spatiotemporal pooling module and a feature-based classification module; the local and global difference modeling unit comprises a first global sequence modeling module and a local difference modeling module arranged in parallel, a first spatiotemporal alignment module, a second global sequence modeling module and a global difference modeling module arranged in parallel, and a second spatiotemporal alignment module; wherein N is a preset value; The local and global difference modeling unit first performs an initial spatiotemporal feature map output by the initial convolution module. Through the first global sequence modeling module, 3D convolution operation is used to extract global motion features in both spatial and temporal dimensions. ; Through the local difference modeling module, the local difference features between adjacent frames are calculated using the sliding window mechanism ; Then the global motion feature and local difference characteristics Input the first time sequence alignment module to align and fuse the local difference features and the global motion features in the time dimension to generate the aligned and fused gait features ; Then fuse the gait features Input the second global sequence modeling module to calculate the global spatiotemporal features on the entire gait sequence to obtain the global motion features ; The gait features will be integrated The global difference feature modeling module is input to obtain the global difference feature by comparing the difference between each frame and the temporal average representation of the entire sequence. ; Finally, the global motion features and global difference features Input the second temporal alignment module to align and fuse the global difference features and the global gait features in the time dimension to generate the aligned and fused gait features ; Wherein, B is the batch size when the local and global difference modeling network is trained, C is the number of channels of the initial feature, T is the number of frames of the feature map, and H and W are the height and width of the feature map respectively; The first global sequence modeling module consists of a 3D pooling layer, a 3D convolution layer, a BN layer and a ReLU activation function; its working principle is: ； ； in, is the difference between the current frame and the average temporal characteristics of the window in the local sliding window; The second global sequence modeling module consists of a 3D convolutional layer, a BN layer and a ReLU activation function, and its working principle is: ； The first time series alignment module and the second time series alignment module are composed of a feature splicing layer, a convolution layer, a time series convolution layer and a feature addition layer connected in sequence; the feature splicing layer is used to splice the input difference features and sequence features and input them into the convolution layer; the feature addition layer is used to add the output of the time series convolution layer to the input sequence features and then output them; the time series convolution layer includes a 3D convolution layer, a batch normalization layer and a ReLU nonlinear activation function; Step 3: Calculate the Euclidean distance between the gait features obtained in step 2 and the gait features of known identities in the database, and determine the identity information of the gait profile sequence extracted in step 1 based on the Euclidean distance and a preset threshold.

2. The gait recognition method based on local and global difference modeling network according to claim 1, characterized in that: The specific implementation of step 1 includes the following sub-steps: Step 1.1: Preprocess the video data collected by the camera, including pedestrian detection, target tracking and pedestrian segmentation, and extract the gait profile sequence A1 of the target pedestrian from the video; Step 1.2: crop and align the gait profile sequence A1, and perform sampling in time series to obtain the gait profile sequence A2; Step 1.3: Perform spatial data enhancement on the gait profile sequence A2.

3. The gait recognition method based on local and global difference modeling network according to claim 1, characterized in that: In step 2, the initial convolution module is used to transform the gait profile sequence obtained in step 1 Send it to an initialized convolution layer to obtain the initial spatiotemporal feature map represents the preliminary dynamic information of each frame in the gait sequence; where B is the batch size when the local and global difference modeling network is trained, C in is the number of channels of the input contour map, C is the number of channels of the initial feature, T is the number of frames of the feature map, and H and W are the height and width of the feature map, respectively.

4. The gait recognition method based on local and global difference modeling network according to claim 1, characterized in that: In step 2, the input of the first unit of the N local and global difference modeling units connected in series is the output of the initial convolution module, and the input of the remaining units are the output of the previous unit; Get the spatiotemporal feature map of the gait sequence ;in, and are the number of channels, height and width of the spatiotemporal feature map respectively; B is the batch size during training of the local and global difference modeling network, and T is the number of frames of the feature map.

5. The gait recognition method based on local and global difference modeling network according to any one of claims 1 to 4, characterized in that: The local and global difference modeling network is a trained network; a joint loss function is used in the training, and parameters are updated through an SGD optimizer and a gradient descent algorithm; The joint loss function is the triple loss and cross entropy loss The joint loss L is: ； ； ； in, and Represent the feature vectors of anchor samples, positive samples and negative samples respectively, is a preset interval; Is the correct category y The corresponding weight vector, Yes Category k The corresponding weight vector, K is the total number of categories and z is the input feature vector.

6. A gait recognition system based on a local and global difference modeling network, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the one or more processors to implement the gait recognition method based on the local and global difference modeling network according to any one of claims 1 to 5.

Images