Gait recognition feature recognition method based on multi-angle conversion

Abstract

The invention relates to the technical field of computer vision, and provides a gait recognition feature recognition method based on multi-angle conversion, and the method comprises the following steps: obtaining to-be-recognized video data, and extracting a segmentation map of a pedestrian gait sequence; judging a pedestrian angle in the pedestrian gait sequence based on the trained angle classification model; when the proportion of the pedestrian being the body is smaller than a set value, inputting the non-body sequence diagram into a VAE generation model for angle correction; the VAE generation model comprises an encoder, a decoder and a softmax classification layer which are connected in sequence; and combining the corrected body sequence image with a body sequence in the pedestrian gait sequence to obtain a gait sequence, and performing feature extraction to obtain a gait recognition result. According to the multi-angle conversion method provided by the invention, the gait images at different visual angles are converted to a unified visual angle, so that the influence of the visual angle difference on the recognition precision is reduced, and the robustness of gait feature recognition is improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field related to computer vision, and in particular to a gait recognition feature recognition method based on multi-angle conversion. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] Gait recognition is an important biometric recognition technology with the advantages of non-contact, remote, and no need for specific equipment. However, in practical applications, gait recognition still faces many challenges due to the complexity of human gait and the diversity of environmental factors.

[0004] The existing gait feature comparison method directly extracts gait features through the gait model. The feature similarity of the same person at different angles is low, and the impact of perspective changes on gait recognition accuracy is not considered.

[0005] However, the gait data collected in a specific laboratory scenario has a single angle, which will cause a perspective-dependent problem in the gait recognition process. In actual applications, due to the different positions, angles, and heights of the camera, the gait images of pedestrians are often captured at different perspectives. Perspective change is one of the important factors affecting the accuracy of gait recognition. The inventors found in their research that when traditional gait recognition methods perform feature matching after feature extraction, they are usually limited by the large angle difference between the two gait sequences, resulting in too low feature similarity at different angles for the same person, making it difficult to meet the gait feature retrieval requirements in actual application scenarios. Summary of the invention

[0006] In order to solve the above problems, the present disclosure proposes a gait recognition feature recognition method based on multi-angle conversion, and proposes a multi-angle conversion method to convert gait images under different perspectives into a unified perspective, thereby reducing the impact of perspective differences on recognition accuracy and improving the robustness of gait feature recognition.

[0007] In order to achieve the above objectives, the present disclosure adopts the following technical solutions:

[0008] One or more embodiments provide a gait recognition feature recognition method based on multi-angle conversion, comprising the following steps:

[0009] Obtain the video data to be identified and extract the segmentation map of the pedestrian gait sequence;

[0010] Based on the trained angle classification model, the pedestrian angle in the pedestrian gait sequence is judged;

[0011] When the proportion of pedestrians in the front view is less than the set value, the sequence graph of the non-front view is input into the VAE generation model for angle correction; the VAE generation model includes an encoder, a decoder and a softmax classification layer connected in sequence;

[0012] The corrected front-facing sequence graph is combined with the front-facing sequence in the pedestrian gait sequence to obtain the gait sequence, and the gait recognition result is obtained after feature extraction.

[0013] One or more embodiments provide a gait recognition system based on multi-angle conversion, including:

[0014] A segmentation module is configured to obtain video data to be identified and extract a segmentation map of a pedestrian gait sequence;

[0015] an angle classification module, configured to determine the angle of a pedestrian in a pedestrian gait sequence based on a trained angle classification model;

[0016] The angle correction module is configured to input the sequence graph of the non-rectangular pedestrian into the VAE generation model for angle correction when the proportion of the pedestrian in the upright position is less than a set value; the VAE generation model includes an encoder, a decoder and a softmax classification layer connected in sequence;

[0017] The recognition module is configured to merge the corrected front-facing sequence diagram with the front-facing sequence in the pedestrian gait sequence to obtain a gait sequence, and obtain a gait recognition result after feature extraction.

[0018] An electronic device includes a memory and a processor, and computer instructions stored in the memory and running on the processor. When the computer instructions are run by the processor, the steps in the above-mentioned gait recognition feature recognition method based on multi-angle conversion are completed.

[0019] A computer-readable storage medium is used to store computer instructions. When the computer instructions are executed by a processor, the steps in the above-mentioned gait recognition feature recognition method based on multi-angle conversion are completed.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] In order to reduce the redundancy of the model and avoid adding an additional angle classification model for joint training, the present invention adds a softmax classification layer at the end of the decoder of the VAE generation model, and adds a cross entropy loss to the loss function, so that the positive image generated by the decoder has the ability to distinguish. The present invention uses multi-angle conversion technology to improve the accuracy of gait recognition, obtains the pedestrian gait sequence through video data and performs angle correction to unify the perspective of the gait data, thereby improving the recognition effect.

[0022] The advantages of the present disclosure and the advantages of additional aspects will be described in detail in the following specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings constituting a part of the present disclosure are used to provide a further understanding of the present disclosure. The exemplary embodiments of the present disclosure and the description thereof are used to explain the present disclosure but do not constitute a limitation of the present disclosure.

[0024] Figure 1 is a method flow chart of a gait recognition method according to Embodiment 1 of the present disclosure;

[0025] Figure 2 is a block diagram of a gait recognition system according to Embodiment 1 of the present disclosure; DETAILED DESCRIPTION

[0026] The present disclosure is further described below in conjunction with the accompanying drawings and embodiments.

[0027] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present disclosure belongs.

[0028] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof. It should be noted that, in the absence of conflict, the various embodiments in the present disclosure and the features in the embodiments can be combined with each other. The embodiments will be described in detail below in conjunction with the accompanying drawings.

[0029] Example 1

[0030] In the technical solutions disclosed in one or more embodiments, Figure 1 As shown, a gait recognition feature recognition method based on multi-angle conversion includes the following steps:

[0031] Step 1: Obtain the video data to be identified and extract the segmentation map of the pedestrian gait sequence;

[0032] Step 2: Determine the pedestrian angle in the pedestrian gait sequence based on the trained angle classification model;

[0033] Step 3: When the proportion of pedestrians in the front view is less than the set value, the sequence diagram of the non-front view is input into the VAE generation model for angle correction; the VAE generation model includes an encoder, a decoder and a softmax classification layer connected in sequence;

[0034] Step 4: merge the corrected front-facing sequence graph with the front-facing sequence in the pedestrian gait sequence to obtain a gait sequence, and extract features to obtain a gait recognition result;

[0035] VAE (Variational Autoencoder) is a generative model and an unsupervised learning method in deep learning. VAE generates new data by learning the probability distribution of data, and is usually used for tasks such as image generation, data compression, and anomaly detection. The VAE generative model is usually used for image generation, but this embodiment needs to achieve high-precision conversion of pedestrians from sideways to frontal, which is different from the conventional application of VAE. Therefore, the structure of the model is improved in this embodiment, which is conducive to the VAE generative model to perform frontal conversion.

[0036] In step 1, the video data to be identified is obtained, and the segmentation map of the pedestrian gait sequence is extracted, which includes the following steps:

[0037] Step 11: preprocess the acquired video data;

[0038] Specifically, the video data to be recognized is preprocessed, including frame rate adjustment, resolution scaling, and noise removal;

[0039] The purpose of preprocessing is to improve the quality of video data and ensure the accuracy of gait segmentation maps extracted in subsequent steps;

[0040] Step 12: perform frame segmentation and extract key frames using motion detection algorithm;

[0041] Specifically, the preprocessed video data is segmented into single-frame images, and the key frames containing pedestrians are identified through a motion detection algorithm to reduce the processing of useless frames;

[0042] Among them, the motion detection algorithm can use background subtraction or optical flow method;

[0043] Step 13: Perform pedestrian detection and tracking on key frames, use pedestrian detection algorithm to detect pedestrians in video frames, and track pedestrians;

[0044] Pedestrian detection algorithms can use YOLO, Faster R-CNN, etc.

[0045] Specifically, the tracking algorithm may use methods such as Kalman filtering or SORT to ensure the continuity and integrity of the gait sequence.

[0046] Step 14: Gait segmentation: For the detected pedestrians, a segmentation network (such as Mask R-CNN or Deeplab, etc.) is used to segment the pedestrian area;

[0047] The purpose of segmentation is to extract the outline and posture of pedestrians for gait analysis.

[0048] Among them, the segmentation network can use Mask R-CNN or Deeplab, etc.

[0049] Step 15: Gait sequence generation: The segmentation maps of pedestrians detected in consecutive frames are combined into a gait sequence. Each gait sequence contains segmentation maps of pedestrians at multiple time points, which are used to describe the movement trajectory of pedestrians in the video.

[0050] In step 2, the pedestrian angle in the pedestrian gait sequence is determined based on the trained angle classification model. The specific implementation steps are as follows:

[0051] Step 21, feature extraction: Extract the features of each frame from the gait sequence obtained in step 1.

[0052] Specifically, feature extraction can use convolutional neural network (CNN), such as ResNet18, Inception, etc.; the extracted features are used for angle judgment;

[0053] Step 22: Based on the features extracted for each frame, map the features to output categories of angle classification through a fully connected layer;

[0054] Specifically, in this embodiment, a 5-classification model can be set, and the model can adopt ResNet18, and the angles include 0 degrees, 45 degrees, 90 degrees, 135 degrees and 180 degrees, and the front and back are 0 degrees and 180 degrees respectively;

[0055] In step 3, the ratio of the upright and non-upright frames in the gait sequence is counted, and a time window sliding statistical method can be used. The time window method is used to count the ratio of frames, which is particularly suitable for dynamic scenes or situations where gait changes greatly. The steps include:

[0056] Step 31, set a time window (eg, 5 frames), and count the ratio of the normal body to the non-normal body within this window.

[0057] Step 32: The sliding window moves forward frame by frame, one frame at a time, and the proportions in the window are recounted.

[0058] Step 33: Finally, according to the statistical results of all windows, the overall proportion is obtained by taking the average value or weighted average;

[0059] In step 3, in order to reduce the redundancy of the model and avoid adding an additional angle classification model for joint training, a softmax classification layer is added to the end of the decoder of the VAE generation model, and the cross entropy loss is added to the loss function, so that the positive image generated by the decoder can have discriminative ability;

[0060] Specifically, the VAE generation model includes an encoder and a decoder. The encoder converts the input image into a latent space representation, and the decoder generates a new frontal image corresponding to the side image from the latent space. In order to better generate the frontal image, a softmax classification layer is added to the decoder for frontal and side angle judgment;

[0061] Specifically, the encoder encodes the side image into the distribution parameters (mean and variance) of the latent space, and the decoder samples from the latent space to generate the corresponding frontal image;

[0062] Among them, the distribution parameters of the latent space may include mean and variance;

[0063] A further technical solution is that the cross entropy loss is added to the loss function of the VAE generation model training. During the training process, the softmax classification layer calculates a loss value for the positive image generated by the decoder; through back propagation, the parameters of the entire VAE generation model are adjusted based on the loss value, thereby optimizing the effect of the positive image generated by the VAE generation model.

[0064] Specifically, during the back-propagation process, the parameters of the encoder and decoder of the VAE generative model are adjusted and updated simultaneously.

[0065] Specifically, the loss function of VAE generative model training includes reconstruction loss L recon , KL divergence loss L kl and the cross entropy loss L class :

[0066] L total =α*L recon +β*L kl +γ*L class

[0067] Among them, α, β, and γ are the coefficients of each loss function respectively;

[0068] The reconstruction loss is the difference loss between the image and the reconstructed image, specifically:

[0069]

[0070] in, is the reconstructed image output by the decoder of the VAE generation model, and x is the image input to the VAE generation model

[0071] The KL divergence loss is used to make the latent space close to a standard normal distribution:

[0072]

[0073] Among them, μ represents the mean of the encoder output, which is used to represent the expected value of the latent variable; σ represents the standard deviation of the encoder output, which is used to represent the dispersion of the latent variable.

[0074] The frontal sequence images generated by the VAE generation model in the previous step and the sequence images determined by the angle model as the frontal body are merged to generate a gait sequence.

[0075] Furthermore, in step 3, if the angle classification model determines that the proportion of pedestrians in the gait sequence with the angle of the pedestrian being straight is not less than the set value, no angle correction is performed, and feature extraction is directly performed to obtain the gait recognition result;

[0076] Specifically, the set ratio may be set to be no less than 80%;

[0077] In this embodiment, when it is determined that more than 80% of the sequence segmentation images are correct, there is no need to use a variational autoencoder (VAE) to adjust the angle, thereby improving efficiency.

[0078] Furthermore, the method further includes the steps of training the angle classification model and the VAE generation model, including the following steps:

[0079] Step S1, obtaining a pedestrian video to be compared, and extracting a segmentation map of the pedestrian gait sequence, where a sequence contains multiple sequence segmentation maps;

[0080] Step S2, setting the classification number of the angle classification model, taking the sequence segmentation map as input, taking the corresponding classification angle as output, inputting it into the angle classification model for training, until the iteration termination condition is met, and obtaining the trained angle classification model;

[0081] Specifically, in this embodiment, the angle classification model can use ResNet18;

[0082] The angle classification model can be set to a 5-category model, including angles of 0, 45, 90, 135 and 180 degrees; the front and back are 0 and 180 degrees respectively, and the images classified as 45, 90 and 135 degrees are converted to the front;

[0083] VAE generative model training: pass the sequence segmentation map of step S1 into the variational autoencoder (VAE) to learn the potential representation of the image. 0 degrees and 180 degrees as well as 45 degrees, 90 degrees, and 135 degrees are trained as front and side image pairs respectively. The encoder-decoder structure is used to compress and generate data. The encoder encodes the side image into the distribution parameters (mean and variance) of the latent space, and the decoder samples from the latent space to generate the corresponding front image. After multiple iterations, the trained VAE generative model is obtained.

[0084] The above gait recognition method of this embodiment has improved recognition accuracy. Through multi-angle conversion, gait images under different viewing angles can be converted to a unified viewing angle. It adapts to the angle changes of cameras in different scenes and alleviates the problem of low similarity of feature comparison of the same person due to large angle differences. To a certain extent, it can eliminate the possibility that pedestrians may deliberately change their gait to evade recognition, or due to the influence of factors such as occlusion and lighting.

[0085] Example 2

[0086] Based on Example 1, this embodiment provides a gait recognition system based on multi-angle conversion, such as Figure 2 As shown, including:

[0087] A segmentation module is configured to obtain video data to be identified and extract a segmentation map of a pedestrian gait sequence;

[0088] an angle classification module, configured to determine the angle of a pedestrian in a pedestrian gait sequence based on a trained angle classification model;

[0089] The angle correction module is configured to input the sequence graph of the non-rectangular pedestrian into the VAE generation model for angle correction when the proportion of the pedestrian in the upright position is less than a set value; the VAE generation model includes an encoder, a decoder and a softmax classification layer connected in sequence;

[0090] The recognition module is configured to merge the corrected front-facing sequence diagram with the front-facing sequence in the pedestrian gait sequence to obtain a gait sequence, and obtain a gait recognition result after feature extraction.

[0091] It should be noted here that the various modules in this embodiment correspond one-to-one to the various steps in Example 1, and the specific implementation process is the same, which will not be repeated here.

[0092] Example 3

[0093] Based on Example 1, this embodiment provides an electronic device, including a memory and a processor, and computer instructions stored in the memory and running on the processor. When the computer instructions are executed by the processor, the steps in the gait recognition feature recognition method based on multi-angle conversion described in Example 1 are completed.

[0094] Example 4

[0095] Based on Example 1, this embodiment provides a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, the steps in the gait recognition feature recognition method based on multi-angle conversion described in Example 1 are completed.

[0096] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

[0097] Although the above describes the specific implementation methods of the present disclosure in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present disclosure. Technical personnel in the relevant field should understand that on the basis of the technical solution of the present disclosure, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present disclosure.

Claims

1. A gait recognition feature recognition method based on multi-angle conversion, characterized in that: The steps include: Obtain the video data to be identified and extract the segmentation map of the pedestrian gait sequence; Based on the trained angle classification model, the pedestrian angle in the pedestrian gait sequence is judged; When the proportion of pedestrians in the front view is less than the set value, the sequence graph of the non-front view is input into the VAE generation model for angle correction; the VAE generation model includes an encoder, a decoder and a softmax classification layer connected in sequence; The corrected front-facing sequence graph is combined with the front-facing sequence in the pedestrian gait sequence to obtain the gait sequence, and the gait recognition result is obtained after feature extraction.

2. The method for gait recognition feature recognition based on multi-angle conversion according to claim 1, characterized in that: In the VAE generation model, the encoder transforms the input image into a latent space representation, and the decoder generates a new frontal image corresponding to the side view image from the latent space. A softmax classification layer is added after the decoder. The softmax classification layer is used to judge the front and side angles.

3. The gait recognition feature recognition method based on multi-angle conversion as claimed in claim 2, characterized in that: The encoder encodes the side image into the distribution parameters mean and variance of the latent space, and the decoder samples from the latent space to generate the corresponding frontal image.

4. The method for gait recognition feature recognition based on multi-angle conversion according to claim 1, characterized in that: In the training of the VAE generation model, cross entropy loss is added to the loss function, and the softmax classification layer calculates a loss value for the positive image generated by the decoder; through back propagation, the parameters of the entire VAE generation model are adjusted based on the loss value.

5. The method for gait recognition feature recognition based on multi-angle conversion according to claim 1, characterized in that: The loss function for VAE generative model training includes the reconstruction loss L recon , KL divergence loss L kl and the cross entropy loss L class : L total =α*L recon +β*L kl +γ*L class Among them, α, β, and γ are the coefficients of each loss function respectively.

6. The method for gait recognition feature recognition based on multi-angle conversion according to claim 1, characterized in that: If the angle classification model determines that the proportion of pedestrians in the gait sequence with an angle of upright posture is not less than a set value, no angle correction is performed and feature extraction is performed directly to obtain the gait recognition result.

7. The method for gait recognition feature recognition based on multi-angle conversion according to claim 6, characterized in that: The set value is not less than 80%.

8. The method for gait recognition feature recognition based on multi-angle conversion according to claim 1, characterized in that: Obtaining the video data to be identified and extracting the segmentation map of the pedestrian gait sequence includes the following steps: Preprocessing the acquired video data; Perform frame segmentation and extract key frames using motion detection algorithm; Use pedestrian detection algorithms to detect pedestrians in video frames and track them; For the detected pedestrians, a segmentation network is used to segment the pedestrian area to obtain a pedestrian segmentation map; The segmentation maps of pedestrians detected in consecutive frames are combined into gait sequences.

9. The method for gait recognition feature recognition based on multi-angle conversion according to claim 1, characterized in that: The ratio of the front and non-front frames in the gait sequence is counted using a time window sliding statistics method.

10. The method for gait recognition feature recognition based on multi-angle conversion according to claim 1, characterized in that: Using the time window method to count the frame ratio includes the following steps: Set a time window and count the ratio of genuine and non-genuine figures within this window; The sliding window moves forward frame by frame, one frame at a time, and the proportion within the window is recounted; Finally, based on the statistical results of all windows, the overall proportion is obtained by taking the average or weighted average.

Images