A gait recognition method based on selective partial feature fusion
Patent Information
- Application Number
- CN202310798381.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-07-03
- Publication Date
- 2026-10-27
- Estimated Expiration
- 2043-07-03
AI Technical Summary
但由于穿着条件或移动幅度的变化可能会导致定位错误，从而降低识别准确率，此外，相邻帧的冗余性限制了时空差异模式的识别
[0035]本发明提供的基于选择性部分特征融合的步态识别方法，首先对数据集进行预处理得到步态轮廓的视频序列，通过选择性部分特征融合，多尺度局部时间聚合模块和空间注意力模块，得到的步态特征更具有判别性，从而大大提高了对于不同个体的识别效果。
Smart Images
Figure CN116798125B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of gait recognition technology, and more specifically to a gait recognition method based on selective partial feature fusion. Background Technology
[0002] Gait recognition is a biometric identification technology whose core idea is to identify individuals in a database by analyzing their walking posture. However, in practical applications, the performance of gait recognition is affected by many factors, such as changes in viewpoint, occlusion, and clothing conditions. To overcome these difficulties, current mainstream gait recognition methods utilize deep learning technology, employing convolutional neural networks to extract spatiotemporal information from gait data, and then process and classify it to achieve more accurate and stable recognition results. Based on whether the temporal order of frames is considered, gait recognition methods can be divided into two categories: set-based and sequence-based.
[0003] Set-based methods treat gait sequences as an unordered set, so the order of the inputs is irrelevant to gait evaluation. However, ignoring the temporal characteristics of the gait sequences can lead to the loss of discriminative local motion information, which may be the most useful feature in gait identification.
[0004] Sequence-based methods tend to explore an individual's gait patterns across multiple spatial and temporal scales. Since input sequences are typically aligned, using segmentation at the same level or intermediate layer features from the input images can improve recognition performance. However, variations in clothing conditions or movement amplitude can lead to localization errors, thus reducing recognition accuracy. Furthermore, redundancy between adjacent frames limits the recognition of spatiotemporally different patterns. Summary of the Invention
[0005] The purpose of this invention is to provide a gait recognition method based on selective partial feature fusion, which solves the following technical problems:
[0006] Most existing gait recognition methods treat all body parts equally, ignoring the importance of different body parts in recognition, thus reducing the ability to make judgments. Changes in clothing conditions or movement range may lead to positioning errors, thereby reducing recognition accuracy. In addition, the redundancy of adjacent frames also limits the effectiveness of gait recognition.
[0007] Therefore, this invention proposes a novel selective partial feature fusion network to improve the performance and accuracy of gait recognition.
[0008] The objective of this invention can be achieved through the following technical solutions:
[0009] A gait recognition method based on selective partial feature fusion includes the following steps:
[0010] A gait recognition model is established, which includes a 3D convolutional network, a multi-scale local temporal aggregation module, a spatial attention module, and a fully connected layer;
[0011] The gait recognition dataset samples are preprocessed to generate gait video contour sequences, and the gait recognition video contour sequences are input into a 3D convolutional network in batches for feature extraction.
[0012] The extracted features are input into a multi-scale local temporal aggregation module to capture discriminative motion features; the motion features are then input into a 3D convolutional network to extract fine-grained gait features; the gait features are input into a spatial attention module to extract discriminative spatial features; and the spatial features are input into a fully connected layer to obtain the predicted probability of each sample.
[0013] The model loss is calculated, and the model parameters are optimized using the Adam optimization algorithm. When the number of training iterations reaches the preset number, a gait recognition model that meets the requirements is output.
[0014] As a further aspect of the present invention: the preprocessing process is as follows:
[0015] Each frame in the gait video is extracted frame by frame. Based on the gait detection algorithm, each frame is identified sequentially. The region containing the human gait is selected. Within the gait region, the contour information of the human body is extracted through the edge detection algorithm and the image segmentation network. The extracted human contour is then subjected to noise reduction, contour smoothing and hole filling. Each gait contour is scaled, rotated and translated to make each gait contour have the same size, orientation and position.
[0016] As a further aspect of the present invention: the 3D convolutional network includes a global 3D convolutional network and a local 3D convolutional network, and the features extracted by the global 3D convolutional network and the features extracted by the local 3D convolutional network are fused together and then output.
[0017] As a further aspect of the present invention: the multi-scale local time aggregation module reduces the time dimension to one-third, outputs the features after the time dimension reduction, and concatenates the features obtained by the multi-scale local time aggregation module with the features obtained by feature fusion to output the concatenated features.
[0018] As a further aspect of the present invention: the process of feature extraction using a 3D convolutional network is as follows:
[0019] The input feature map is represented as Where C1 represents the number of channels, T represents the time dimension, and H and W represent the length and width of the feature map, the input feature map is divided into k parts along the horizontal dimension, where k is a positive integer, denoted as... i∈1,2,…,k, so the segmented feature map is represented as The selective feature fusion process can be represented by the following formula:
[0020]
[0021] in This represents a 3D convolution with a kernel size of 3×3×3, a stride of 1×1×1, and α. i These are parameters that can be learned during network training, representing the importance of each part in the discrimination. The output of a 3D convolutional network is represented as... Parameters are shared between the 3D convolutions in each part;
[0022] The feature maps of all parts are concatenated along the horizontal dimension, as shown in the following formula:
[0023]
[0024] in This indicates a splicing operation. This represents the feature map obtained after the SPFF operation.
[0025] As a further aspect of the present invention: the process of the multi-scale local time aggregation module extracting motion features is as follows:
[0026] The intermediate features are represented as The multi-scale local time aggregation module is expressed by the following formula:
[0027]
[0028] in This represents a 3D convolution with a kernel size of a×3×3 and a stride of 3×1×1; where This represents a 3D convolution with a kernel size of b×3×3 and a stride of 3×1×1; where To represent a 3D convolution with a kernel size of c×3×3 and a stride of 3×1×1, the output of MLTA is expressed as:
[0029] As a further aspect of the present invention: the spatial feature extraction process is as follows:
[0030] The features after time pooling are represented as follows: The spatial attention module can be expressed by the following formula:
[0031]
[0032] in This represents a 2D convolution with a kernel size of 1×1 and a stride of 1×1. softmax(·) represents the softmax activation function.
[0033] As a further aspect of the present invention, the model loss is calculated based on the cross-entropy loss function and the triplet loss function.
[0034] The beneficial effects of this invention are:
[0035] The gait recognition method based on selective partial feature fusion provided by this invention first preprocesses the dataset to obtain a video sequence of gait contours. Through selective partial feature fusion, multi-scale local temporal aggregation module and spatial attention module, the obtained gait features are more discriminative, thereby greatly improving the recognition effect for different individuals. Attached Figure Description
[0036] The invention will now be further described with reference to the accompanying drawings.
[0037] Figure 1 This is a flowchart illustrating a gait recognition method based on selective partial feature fusion according to the present invention.
[0038] Figure 2 This is a schematic diagram of the gait recognition model of the present invention;
[0039] Figure 3 This is a schematic diagram of the selective feature fusion structure of the present invention;
[0040] Figure 4 This is a schematic diagram of the spatial attention module of the present invention. Detailed Implementation
[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0042] Please see Figure 1-4 As shown, this invention is a gait recognition method based on selective partial feature fusion, comprising the following steps:
[0043] A gait recognition model is established, which includes a 3D convolutional network, a multi-scale local temporal aggregation module, a spatial attention module, and a fully connected layer;
[0044] The gait recognition dataset samples are preprocessed to generate gait video contour sequences, and the gait recognition video contour sequences are input into a 3D convolutional network in batches for feature extraction.
[0045] The extracted features are input into a multi-scale local temporal aggregation module to capture discriminative motion features; the motion features are then input into a 3D convolutional network to extract fine-grained gait features; the gait features are input into a spatial attention module to extract discriminative spatial features; and the spatial features are input into a fully connected layer to obtain the predicted probability of each sample.
[0046] The model loss is calculated, and the model parameters are optimized using the Adam optimization algorithm. When the number of training iterations reaches the preset number, a gait recognition model that meets the requirements is output.
[0047] This invention provides a deep learning network based on gait sequences for gait recognition and verification. This network can extract key features from the input gait sequence and map them to a discriminative space to achieve accurate identity recognition and verification. The specific implementation process is as follows:
[0048] Given a gait sequence S containing T frames as input to the network, the sequence is processed through a series of modules to extract robust and discriminative features.
[0049] First, the gait sequence undergoes a selective partial feature fusion module and a global 3D convolution operation, and the resulting features are summed. This step is performed twice and is called global-local feature fusion. Global-local feature fusion preserves both local and global motion information by partially fusing and globally fusing the input features.
[0050] Next, a multi-scale local temporal aggregation module reduces the temporal dimension to T / 3. This module can aggregate similar frames in the sequence, capturing discriminative motion information. The features obtained after multi-scale temporal aggregation are then concatenated with the features obtained after the global-local feature fusion module.
[0051] Subsequently, after a global-local feature fusion, spatial pooling is performed. Spatial pooling helps reduce the dimensionality of features while preserving important spatial information. The features obtained after spatial pooling are added to the features obtained from the multi-scale temporal aggregation module, and global-local feature fusion is performed again, this time using a concatenation method.
[0052] The obtained features are then subjected to temporal pooling to further reduce the temporal dimension and extract key motion information. Next, the features are processed by a spatial attention module to apply spatial attention weighting to highlight regions of interest.
[0053] Finally, generalized average pooling maps the features to the discriminant space. Generalized average pooling effectively extracts global information from the features and reduces their dimensionality. The mapped features are then used for gait recognition and verification tasks.
[0054] To train the model, this invention employs a joint training strategy of triplet loss and cross-entropy loss. Triplet loss constrains the model to learn better feature representations by minimizing the distance between features of the same identity and maximizing the distance between features of different identities. Cross-entropy loss is used to optimize gait classification tasks.
[0055] In a preferred embodiment of the present invention, the preprocessing process is as follows:
[0056] Extract each frame from the gait video: Use a video processing library or tool, such as OpenCV, to read the gait video file and extract the video frames one by one.
[0057] Gait detection and recognition: Apply a gait detection algorithm or gait recognition model to each frame to determine the regions containing human gait. This can be done using computer vision and deep learning techniques, such as feature point-based or human pose estimation methods.
[0058] Human contour extraction: Within the gait region, extract the contour information of the human body. Image segmentation algorithms or edge detection algorithms can be used to extract the contour, such as Canny edge detection, GrabCut, and deep learning-based segmentation networks.
[0059] Cleaning and preprocessing: The extracted human body contours are cleaned and preprocessed. This may include operations such as noise removal, contour smoothing, and hole filling to ensure the accuracy and consistency of the contours.
[0060] Standardization and normalization: Each gait profile is standardized and normalized to ensure they have the same size, orientation, and position. This can be achieved by scaling, rotating, and translating each profile.
[0061] In another preferred embodiment of the present invention, the 3D convolutional network includes a global 3D convolutional network and a local 3D convolutional network, and the features extracted by the global 3D convolutional network and the features extracted by the local 3D convolutional network are fused and then output.
[0062] It is worth noting that the multi-scale local time aggregation module reduces the time dimension to one-third and outputs the features after the time dimension reduction. The features obtained by the multi-scale local time aggregation module are concatenated with the features obtained by feature fusion and the concatenated features are output.
[0063] In another preferred embodiment of the present invention, the process of feature extraction by the 3D convolutional network is as follows:
[0064] The input feature map is represented as Where C1 represents the number of channels, T represents the time dimension, and H and W represent the length and width of the feature map, the input feature map is divided into k parts along the horizontal dimension, where k is a positive integer, denoted as... i∈1,2,…,k, so the segmented feature map is represented as The selective feature fusion process can be represented by the following formula:
[0065]
[0066] in This represents a 3D convolution with a kernel size of 3×3×3, a stride of 1×1×1, and α. i These are parameters that can be learned during network training, representing the importance of each part in the discrimination. The output of a 3D convolutional network is represented as... Parameters are shared between the 3D convolutions in each part;
[0067] The feature maps of all parts are concatenated along the horizontal dimension, as shown in the following formula:
[0068]
[0069] in This indicates a splicing operation. This represents the feature map obtained after the SPFF operation.
[0070] In another preferred embodiment of the present invention, the process of the multi-scale local time aggregation module extracting motion features is as follows:
[0071] The intermediate features are represented as The multi-scale local time aggregation module is expressed by the following formula:
[0072]
[0073] in This represents a 3D convolution with a kernel size of a×3×3 and a stride of 3×1×1; where This represents a 3D convolution with a kernel size of b×3×3 and a stride of 3×1×1; where To represent a 3D convolution with a kernel size of c×3×3 and a stride of 3×1×1, the output of MLTA is expressed as:
[0074] In another preferred embodiment of the present invention, the spatial feature extraction process is as follows:
[0075] The features after time pooling are represented as follows: The spatial attention module can be expressed by the following formula:
[0076]
[0077] in This represents a 2D convolution with a kernel size of 1×1 and a stride of 1×1. softmax(·) represents the softmax activation function.
[0078] In another preferred embodiment of the invention, the model loss is calculated based on the cross-entropy loss function and the triplet loss function.
[0079] The foregoing has provided a detailed description of one embodiment of the present invention, but this description is merely a preferred embodiment and should not be construed as limiting the scope of the invention. All equivalent variations and modifications made within the scope of the claims of this invention should still fall within the patent coverage of this invention.
Claims
1. A gait recognition method based on selective partial feature fusion, characterized in that, Includes the following steps: A gait recognition model is established, which includes a 3D convolutional network, a multi-scale local temporal aggregation module, a spatial attention module, and a fully connected layer; The gait recognition dataset samples are preprocessed to generate gait video contour sequences, and the gait recognition video contour sequences are input into a 3D convolutional network in batches for feature extraction. The extracted features are input into a multi-scale local temporal aggregation module to capture discriminative motion features; the motion features are then input into a 3D convolutional network to extract fine-grained gait features; The gait features are input into the spatial attention module to extract discriminative spatial features; the spatial features are then input into a fully connected layer to obtain the predicted probability of each sample. The model loss is calculated, and the model parameters are optimized using the Adam optimization algorithm. When the number of training iterations reaches the preset number, a gait recognition model that meets the requirements is output. The process of feature extraction using 3D convolutional networks is as follows: The input feature map is represented as Where C1 represents the number of channels, T represents the time dimension, and H and W represent the length and width of the feature map, the input feature map is divided into k parts along the horizontal dimension, where k is a positive integer, denoted as... For i∈1,2,…,k, the segmented feature map is represented as follows: The selective feature fusion process can be represented by the following formula: in This represents a 3D convolution with a kernel size of 3×3×3, a stride of 1×1×1, and α. i These are parameters that can be learned during network training, representing the importance of each part in the discrimination. The output of a 3D convolutional network is represented as... Parameters are shared between the 3D convolutions of each part; The feature maps of all parts are concatenated along the horizontal dimension, as shown in the following formula: Where © represents a splicing operation. This represents the feature map obtained after the SPFF operation; The process of extracting motion features by the multi-scale local time aggregation module is as follows: The intermediate features are represented as The multi-scale local time aggregation module is expressed by the following formula: in This represents a 3D convolution with a kernel size of a×3×3 and a stride of 3×1×1; where This represents a 3D convolution with a kernel size of b×3×3 and a stride of 3×1×1; where To represent a 3D convolution with a kernel size of c×3×3 and a stride of 3×1×1, the output of MLTA is expressed as: ; The process of extracting spatial features is as follows: The features after time pooling are represented as follows: The spatial attention module is expressed by the formula: in This represents a 2D convolution with a kernel size of 1×1 and a stride of 1×1. softmax(·) represents the softmax activation function.
2. The gait recognition method based on selective partial feature fusion according to claim 1, characterized in that, The preprocessing process is as follows: Each frame in the gait video is extracted frame by frame. Based on the gait detection algorithm, each frame is identified sequentially. The region containing the human gait is selected. Within the gait region, the contour information of the human body is extracted through the edge detection algorithm and the image segmentation network. The extracted human contour is then subjected to noise reduction, contour smoothing and hole filling. Each gait contour is scaled, rotated and translated to make each gait contour have the same size, orientation and position.
3. The gait recognition method based on selective partial feature fusion according to claim 1, characterized in that, The 3D convolutional network includes a global 3D convolutional network and a local 3D convolutional network. The features extracted by the global 3D convolutional network and the features extracted by the local 3D convolutional network are fused together and then output.
4. The gait recognition method based on selective partial feature fusion according to claim 3, characterized in that, The multi-scale local time aggregation module reduces the time dimension to one-third and outputs the features after the time dimension reduction. The features obtained by the multi-scale local time aggregation module are concatenated with the features obtained by feature fusion, and the concatenated features are output.
5. The gait recognition method based on selective partial feature fusion according to claim 1, characterized in that, The model loss is calculated based on the cross-entropy loss function and the triplet loss function.