A gait recognition method and system based on a dynamic aggregation network
By extracting local and global motion patterns from gait sequences using a dynamic aggregation network, the problem that fixed-parameter neural networks cannot model the features of dynamic gait sequences is solved, achieving higher recognition accuracy and robustness.
Patent Information
- Application Number
- CN202211682281.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-27
- Publication Date
- 2025-11-04
- Estimated Expiration
- 2042-12-27
AI Technical Summary
In existing gait recognition methods, neural networks with fixed parameters cannot effectively model the feature relationships in dynamic gait sequences, and global feature extraction is insufficient, resulting in low recognition accuracy.
A gait recognition method based on dynamic aggregation network is adopted. Local motion patterns are extracted through three-dimensional convolution, the backbone of dynamic aggregation network is used to establish the correlation between local features, and discriminative global motion patterns are extracted through horizontal pooling and global motion pattern aggregation.
It effectively extracts distinctive local and global motion patterns, improves the accuracy of gait recognition, reduces the impact of noise, and solves the problem of insufficient feature representation in existing methods.
Smart Images
Figure CN116229564B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to a gait recognition method and system based on a dynamic aggregation network, specifically to a method and system for identifying identity by establishing local and global motion patterns from gait silhouette data, belonging to the fields of computer vision and biometrics. Background Technology
[0002] Gait recognition is an emerging biometric identification method designed to overcome the challenge of identifying the same pedestrian in real-world scenarios, regardless of clothing or viewing angle. Compared to action recognition and person re-identification, gait recognition is one of the most challenging fine-grained label classification problems. Gait silhouette data is constrained by segmentation algorithms, often exhibiting gaps and broken edges. Furthermore, gait recognition is influenced by various external factors in real-world scenarios, such as carrying conditions, clothing, and different viewing angles. Different angles and clothing conditions can drastically alter the silhouette appearance of the same person, resulting in intra-class variance being much greater than inter-class variance.
[0003] Currently, gait recognition methods are mainly divided into two categories: model-based methods and image-based methods. Model-based methods utilize the structural relationships of the human body to model gait features, attempting to fit the human skeleton into an image and then extract gait features from the skeleton sequence. However, these methods are limited by the quality of skeleton recognition and gait silhouette images, resulting in lower recognition performance compared to image-based methods. Image-based methods directly extract features from gait sequences, primarily involving two steps: local motion modeling and global motion modeling. Local motion modeling refers to modeling the local movements of the gait sequence within the backbone network, extracting local motion features from key areas. Global motion modeling refers to modeling the global features of the gait sequence at the neck or head of the network, extracting global motion features from key areas. Local motion modeling provides many stable local gait features for global motion modeling, which then selects representative features from these local features for further learning, ultimately obtaining the global gait features.
[0004] Gait, or the act of walking, is essentially the coordinated movement of various body parts, each with its own unique and representative movement pattern, some of which are complementary. For example, a pedestrian often extends their right hand when stepping with their left leg. To obtain diverse and discriminative movement patterns, many methods have proposed effective solutions, namely, dividing gait features into a fixed number of horizontal features. Some methods utilize convolutional networks to adaptively locate the coordinates, size, and temporal length of body parts. Once the neural network is trained, the parameters of its convolutional and nonlinear mapping layers become fixed, causing the network to recognize only a limited number of movement patterns. However, gait recognition scenarios are far more complex in reality, and neural networks with fixed parameters cannot extract constantly changing gait movement patterns. To obtain more discriminative features, many methods propose using 3D convolutional neural networks to learn local movement patterns in gait sequences, and then directly applying max pooling or average pooling to extract a global movement pattern from the extracted features. These methods focus only on one global movement pattern in the global spatiotemporal dimension. Therefore, focusing on gait features in key regions and extracting the complete global movement pattern is crucial for gait recognition tasks.
[0005] Therefore, in order to apply gait recognition technology in real-world scenarios to extract discriminative global features, or to obtain a gait recognition system with higher recognition accuracy, there is an urgent need for a gait recognition neural network method and system with higher recognition accuracy and the ability to extract more robust global features. Summary of the Invention
[0006] The purpose of this invention is to address the shortcomings and deficiencies of existing technologies. In order to solve the problems that existing methods cannot model the relationship between features in dynamic gait sequences through neural networks with fixed parameters, and lack the technical problem of multi-action fusion due to directly taking the maximum value of global features, a gait recognition method and system based on dynamic aggregation network is creatively proposed.
[0007] To achieve the above objectives, the present invention employs the following technical solutions.
[0008] On the one hand, this invention proposes a gait recognition method based on a dynamic aggregation network, comprising the following steps:
[0009] Step 1: Obtain gait silhouette sequences from the acquired video data through pedestrian detection, pedestrian segmentation, and quality assessment.
[0010] Step 2: Preprocess the gait silhouette sequence to obtain input data of uniform size.
[0011] During preprocessing, the center of each person is located using the edge information of the human figure outline in the image. Then, interpolation methods are used to scale images of different sizes to a uniform size, thereby achieving uniformity in the position and size of each person.
[0012] Step 3: Train a gait recognition network based on dynamic aggregation.
[0013] Includes the following steps:
[0014] Step 3.1: Using a gait sequence of uniform size as input, extract local motion patterns from the gait sequence using a dynamic aggregation network.
[0015] Gait sequences consist of a variable number of consecutive frames, with each movement constantly changing over time. Using the backbone of a dynamic aggregation network, local motion features between gait sequences are first extracted using 3D convolution. Then, each feature point in the gait sequence is mapped to a vector containing amplitude and phase components. The amplitude represents the contextual information of each feature point, while the phase establishes the relationships between feature points. Finally, the backbone of the dynamic aggregation network obtains discriminative local motion patterns by focusing on the relationships between key features.
[0016] Step 3.2: Utilize the horizontal pooling of the dynamic aggregation network to horizontally divide the local features into a fixed number of features, each of which represents a key part.
[0017] Horizontal pooling involves horizontally segmenting the height dimension of the gait feature map. This means that for the same person, the map is segmented from head to toe into a fixed number of features and constrained separately, with the aim of obtaining features that are distinctive for each part.
[0018] Step 3.3: For the features of each local region, use global motion pattern aggregation to select representative local motion patterns and establish the correlation between local features to obtain robust global motion patterns.
[0019] Among them, global motion pattern aggregation does not share parameters for the features of each local region, that is, each key region has a corresponding global motion pattern aggregation module.
[0020] Step 3.4: Map the global motion pattern of each key region to obtain discriminative global gait features. Then, use triplet loss and cross-entropy loss to constrain the global motion pattern of each key region, and calculate the final loss function using a weighted summation method. Finally, use the backpropagation algorithm to iteratively update the parameters of the gait neural network.
[0021] Step 4: Based on the trained gait recognition network, compare the gait features of unknown identities with the gait feature database of known identities, and match the identity information from the gait feature database of known identities.
[0022] Furthermore, to achieve the objectives of this invention, based on the above method, this invention proposes a gait recognition system based on a dynamic aggregation network, including an image acquisition module, an image preprocessing module, a feature extraction module, and a gait recognition module.
[0023] The image acquisition module includes a human detection submodule, a human segmentation submodule, and a quality assessment submodule. The human detection submodule detects human bodies from the original video and segments them (e.g., using bounding boxes). The human segmentation submodule segments each human body image to obtain a silhouette of each image. The quality assessment submodule evaluates the quality of the segmented data, filtering out images with holes or missing limbs.
[0024] The image preprocessing module includes a human body localization submodule and a human body alignment submodule. This module is used to scale data of inconsistent sizes, extract the center of gravity of the human body as the center position of the image, and scale it to a uniform size. The feature extraction module includes a local feature extraction submodule and a global feature extraction submodule. The local feature extraction module dynamically extracts local feature relationships from the input silhouette sequence to obtain local gait features at each pixel location; the global feature extraction module dynamically selects discriminative local features and further maps them to obtain robust global features.
[0025] The gait recognition module is used to match the gait sequences of unknown individuals with a gait feature database of known individuals to obtain the identity information of the human body.
[0026] The connection relationships between the above modules are as follows:
[0027] The output of the image acquisition module is connected to the input of the image preprocessing module. The output of the image preprocessing module is connected to the input of the feature extraction module. The output of the feature extraction module is connected to the input of the gait recognition module.
[0028] Beneficial effects
[0029] Compared with existing technologies, the present invention has the following advantages:
[0030] 1. This invention can effectively establish the relationship between each feature point in a gait sequence, which is beneficial for extracting distinctive local motion patterns while reducing the influence of noise.
[0031] 2. This invention can effectively select representative local features from local motion patterns, which is beneficial for extracting distinctive global motion patterns, while solving the technical problem of insufficient gait feature expression in existing methods. Attached Figure Description
[0032] Figure 1 This is a flowchart illustrating the overall process of the method of the present invention.
[0033] Figure 2 This is a flowchart of the algorithm of the method of the present invention;
[0034] Figure 3 This is a schematic diagram of the system of the present invention. Detailed Implementation
[0035] To better illustrate the purpose and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.
[0036] Example
[0037] like Figure 1 As shown, a gait recognition method based on a dynamic aggregation network includes the following steps:
[0038] Step S10: For the acquired raw gait video data, firstly, the human detection algorithm is used to obtain the detection box of the pedestrian, and the original image of the human body is obtained based on the detection box. Then, the human segmentation algorithm is used to obtain the silhouette data of the pedestrian. Finally, the quality assessment algorithm is used to filter out data with holes and missing limbs in the segmented image.
[0039] Step S20: Data preprocessing.
[0040] This includes aligning pedestrian sequences from silhouette images and scaling pedestrian images of different sizes to a uniform size.
[0041] In this embodiment of the invention, the size of the gait images is uniformly scaled to 64×64.
[0042] Step S30: The data is fed into a gait recognition network based on dynamic aggregation, and after processing, representative global gait features are obtained.
[0043] like Figure 2 As shown, step S30 includes the following sub-steps:
[0044] Step S31: Use local feature extraction to obtain the local motion patterns of the gait sequence.
[0045] Specifically, the gait sequence input image uses express, Let T represent the real number field, H represent the number of frames in the input sequence, H represent the height dimension of the gait silhouette image, and W represent the width dimension of the gait silhouette image.
[0046] Furthermore, local motion patterns are obtained by local feature extraction, and relationships between the features of surrounding pixels are established for each feature point in the gait sequence;
[0047] Furthermore, lateral segmentation refers to dividing the gait feature map laterally along the height dimension, dividing the global gait features into P key regions of global features.
[0048] P i,j =MaxPool2d(V i,j )+AvgPool2d(V i,j (1)
[0049] Among them, V i,j P represents the local motion pattern obtained through local feature extraction. i,j This represents the local gait features obtained through lateral segmentation, where i and j represent the features of the i-th frame and the j-th region, respectively.
[0050] In this embodiment of the invention, the value of P is 64.
[0051] Step S32: For the features of each part, use global feature extraction to select local features with discriminative power, and further establish the relationship between the selected features.
[0052] Furthermore, the maximum value is used to pool the temporal dimension of the gait features, thereby obtaining globally stable gait features for any frame of data input.
[0053] Step S33: Combine triplet loss and cross-entropy loss to constrain the features of each region, and calculate the final loss using a weighted summation method. Wherein, the triplet loss L... tp for:
[0054] L tp =max(D(a,p)-D(a,n)+m,0) (2)
[0055] Where D(a,p) represents the Euclidean distance between anchor point a and positive sample p, D(a,n) represents the Euclidean distance between anchor point a and negative sample n, and m represents the interval of the triplet loss. The triplet loss calculates the Euclidean distances between all positive and negative samples, and then scales the Euclidean distances between the anchor point and positive samples and between the anchor point and negative samples to a size of m. The cross-entropy loss L... ce for:
[0056] L ce =∑-qi,j logP i,j (3)
[0057] Where, q i,j The label of the sample, P i,j Labels representing network predictions.
[0058] Step S40: Compare the gait features of the unknown identity with the gait feature database of the known identity, and match the identity information from the gait feature database of the known identity.
[0059] Figure 3 Here is a schematic diagram of the system structure of the present invention:
[0060] like Figure 3 As shown, a gait recognition method based on a dynamic aggregation network includes an image acquisition module M10, an image preprocessing module M20, a feature extraction module M30, and a gait recognition module M40. Wherein:
[0061] The image acquisition module M10 is used to acquire raw gait video data. The image acquisition module M10 includes a human detection submodule M11, a human segmentation submodule M12, and a quality assessment submodule M13. First, the human detection submodule M11 acquires the raw human images from the video. Then, the human segmentation submodule M12 acquires segmented images of the human body. Finally, the quality assessment submodule M13 filters out data with holes or missing limbs from the segmented images.
[0062] Image preprocessing module M20 is used to obtain a uniform pedestrian sequence. Image preprocessing module M20 includes a human body localization submodule M21 and a human body alignment submodule M22. The human body localization submodule M21 extracts the center point of the human body, and the human body alignment submodule M22 scales the image to a uniform size.
[0063] The feature extraction module M30 is used to obtain gait features of gait sequences of unknown identities. Gait extraction module M30 includes a local feature extraction submodule M31 and a global feature extraction submodule M32. First, the local feature extraction submodule M31 is used to obtain local gait features, and lateral pooling is used to divide the local features into multiple key regions. Then, the global feature extraction submodule M32 is used to obtain global gait features, and max pooling is used to obtain robust global features.
[0064] The gait recognition module M40 is used to match the gait sequences of unknown individuals with the gait feature database of known individuals, thereby obtaining the identity information of the human body.
[0065] The connection relationship of the above modules is as follows:
[0066] The output of image acquisition module M10 is connected to the input of image preprocessing module M20. The output of image preprocessing module M20 is connected to the input of feature extraction module M30. The output of feature extraction module M30 is connected to the input of gait recognition module M40.
[0067] Inside the image acquisition module M10, the output of the human body detection submodule M11 is connected to the input of the human body segmentation submodule M12, and the output of the human body segmentation submodule M12 is connected to the input of the quality assessment submodule M13.
[0068] Inside the image preprocessing module M20, the output of the image delocalization submodule M21 is connected to the input of the image alignment submodule M22.
[0069] Inside the feature extraction module M30, the output of the local feature extraction submodule M31 is connected to the input of the global feature extraction submodule M32.
[0070] Therefore, it is possible to effectively establish the relationship between each feature point in the gait sequence, which is beneficial for extracting discriminative local motion patterns while reducing the influence of noise. This invention can effectively select representative local features from local motion patterns, which is beneficial for extracting discriminative global motion patterns, while solving the technical problem of insufficient gait feature representation in existing methods.
Claims
1. A gait recognition method based on a dynamic aggregation network, characterized in that, Includes the following steps: Step 1: Obtain the gait silhouette sequence from the acquired video data; Step 2: Preprocess the gait silhouette sequence to obtain input data of uniform size; Step 3: Train a gait recognition network based on dynamic aggregation; Step 3.1: Using a gait sequence of uniform size as input, extract local motion patterns from the gait sequence using a dynamic aggregation network; Gait sequences consist of a variable number of consecutive frames, with each action constantly changing over time. Using the backbone of a dynamic aggregation network, local motion features between gait sequences are first extracted using 3D convolution. Then, each feature point in the gait sequence is mapped to a vector containing amplitude and phase. The amplitude represents the contextual information of each feature point, and the phase is used to establish the relationship between feature points. Finally, the backbone of the dynamic aggregation network obtains discriminative local motion patterns by focusing on the relationships between key feature features. Step 3.2: Utilize the horizontal pooling of the dynamic aggregation network to horizontally segment the local features into a fixed number of features, with each feature representing a key part; Horizontal pooling involves horizontally segmenting the height dimension of the gait feature map, that is, segmenting the same person from head to toe into a fixed number of features and constraining them separately. Step 3.3: For the features of each local region, use global motion pattern aggregation to select representative local motion patterns and establish the correlation between local features to obtain robust global motion patterns; Among them, global motion pattern aggregation does not share parameters for the features of each local region, that is, each key region has a corresponding global motion pattern aggregation module. Step 3.4: Map the global motion pattern of each key region to obtain discriminative gait global features; then, use triplet loss and cross-entropy loss to constrain the global motion pattern of each key region, and calculate the final loss function using a weighted summation method; finally, use the backpropagation algorithm to iteratively update the parameters of the gait neural network. Step 4: Based on the trained gait recognition network, compare the gait features of unknown identities with the gait feature database of known identities, and match the identity information from the gait feature database of known identities.
2. The gait recognition method based on a dynamic aggregation network as described in claim 1, characterized in that, In step 2, during preprocessing, the center of each person is located using the edge information of the human figure outline in the image. Then, interpolation methods are used to scale images of different sizes to a uniform size, so that the position and size of each person are uniform.
3. The gait recognition method based on a dynamic aggregation network as described in claim 1, characterized in that, In step 3, the gait sequence input image is used express, Let T represent the real number field, H represent the number of frames in the input sequence, H represent the height dimension of the gait silhouette image, and W represent the width dimension of the gait silhouette image. Local motion patterns are obtained by local feature extraction, and relationships between the features of surrounding pixels are established for each feature point in the gait sequence. Lateral segmentation refers to dividing the gait feature map laterally along the height dimension, dividing the global gait features into P key regions of global features: P i,j =MaxPool2d(V i,j )+AvgPool2d(V i,j ) (1) Among them, V i,j P represents the local motion pattern obtained through local feature extraction. i,j This represents the local gait features obtained through lateral segmentation, where i and j represent the features of the i-th frame and the j-th region, respectively.
4. The gait recognition method based on a dynamic aggregation network as described in claim 1, characterized in that, In step 3, the features of each region are constrained by combining triplet loss and cross-entropy loss, and the final loss is calculated using a weighted summation method. The triplet loss L... tp for: L tp =max(D(a,p)-D(a,n)+m,0) (2) Where D(a,p) represents the Euclidean distance between anchor point a and positive sample p, D(a,n) represents the Euclidean distance between anchor point a and negative sample n, and m represents the interval of the triplet loss; the triplet loss calculates the Euclidean distance between all positive and negative samples, and stretches the Euclidean distance between the anchor point and the positive sample and the distance between the anchor point and the negative sample to a size of m; the cross-entropy loss L... ce for: L ce =∑-q i,j logP i,j (3) Where, q i,j The label of the sample, P i,j Labels representing network predictions.
5. A gait recognition system based on a dynamic aggregation network, employing any one of the methods described in claims 1-4, characterized in that, It includes an image acquisition module, an image preprocessing module, a feature extraction module, and a gait recognition module; The image acquisition module includes a human body detection submodule, a human body segmentation submodule, and a quality assessment submodule; the pedestrian detection submodule is used to detect human bodies from the original video and segment them from the video; the human body segmentation submodule is used to segment each human body image to obtain a silhouette of each image; the quality assessment submodule is used to assess the quality of the segmented data and filter out data with holes or missing limbs in the segmented images. The image preprocessing module includes a human body localization submodule and a human body alignment submodule. This module is used to scale data of inconsistent sizes, extract the center of gravity of the human body as the center position of the image, and scale it to a uniform size. The feature extraction module includes a local feature extraction submodule and a global feature extraction submodule. The local feature extraction module is used to dynamically extract the local feature relationships of the input silhouette sequence to obtain the local gait features of each pixel position. The global feature extraction module is used to dynamically select discriminative local features and further map them to obtain robust global features. The gait recognition module is used to match the gait sequences of unknown individuals with the gait feature database of known individuals in order to obtain the identity information of the human body. The connection relationships between the above modules are as follows: The output of the image acquisition module is connected to the input of the image preprocessing module; the output of the image preprocessing module is connected to the input of the feature extraction module; the output of the feature extraction module is connected to the input of the gait recognition module. Inside the image acquisition module, the output of the human body detection submodule is connected to the input of the human body segmentation submodule, and the output of the human body segmentation submodule is connected to the input of the quality assessment submodule. Inside the image preprocessing module, the output of the image delocalization submodule is connected to the input of the image alignment submodule M22; Inside the feature extraction module, the output of the local feature extraction submodule is connected to the input of the global feature extraction submodule.