A multi-gait recognition method based on occlusion repair

Abstract

The application discloses a multi-gait recognition method based on occlusion repair, and steps are as follows: preprocessing multi-person walking gait data set, detecting and tracking characters, generating single individual gait RGB image, carrying out occlusion repair on the gait RGB image, extracting black and white contour image to input into a gait recognition network, calculating Euclidean distance measurement and feature similarity by the gait recognition network, and outputting pedestrian identity information recognition result. The application realizes a multi-gait recognition method based on occlusion repair, carries out repair and supplement on gait representation, improves gait recognition accuracy, and solves the problem that pedestrian gait recognition cannot be carried out under complex conditions.

Description

Technical Field

[0001] This invention relates to the field of computer vision and pattern recognition technology, specifically to a multi-gait recognition method based on occlusion repair, mainly involving multi-gait recognition including biometric recognition, identity recognition, image processing, depth image and occlusion repair. Background Technology

[0002] Gait recognition is increasingly popular in academia and industry as it is one of the most critical long-range biometric identification technologies. Gait recognition methods based on black-and-white contours can be broadly categorized into three types: template-based methods, set-based methods, and video-based methods. Generally, template-based methods first aggregate all temporal information of the sequence to generate a template. Set-based methods extract gait features frame-by-frame from the gait sequence. Video-based methods directly use 3D CNNs to extract spatiotemporal features from the gait sequence for recognition purposes.

[0003] Pose-based gait recognition methods first extract the 2D or 3D pose of the human body, and then use a deep learning-based framework to generate feature representations. However, due to the very limited number of key human poses, current pose-based gait recognition methods cannot generate distinctive feature representations in real-world gait datasets.

[0004] Despite significant progress in single-person gait recognition, much evidence suggests that multi-person gait recognition is a more challenging task in the field because the presence of multiple people walking together creates significant human occlusion, which increases the difficulty of recognition. Summary of the Invention

[0005] The purpose of this invention is to solve the problem that the recognition accuracy of current multi-gait recognition methods is significantly reduced in practical applications due to human body occlusion, and to provide a multi-gait recognition method based on occlusion repair.

[0006] The objective of this invention can be achieved by adopting the following technical solutions:

[0007] A gait recognition method based on occlusion repair, the gait recognition method comprising the following steps:

[0008] S1. Collect and organize gait videos of multiple people walking together in an indoor scene. Collect videos of 1 person, 2 people and 3 people walking together from multiple perspectives to obtain gait video sequences. Use the gait video sequences as a multi-gait dataset and divide them into training set and test set.

[0009] S2. Extract RGB gait sequence images of a single person from the gait video sequence;

[0010] S3. Extract the 3D pose of the person from the RGB gait sequence image of the single person to obtain the occluded 3D pose gait sequence.

[0011] S4. Train the motion filler and use it to fill in the occluded 3D pose and gait sequence to obtain an unoccluded 3D pose and gait sequence. The motion filler adopts a Transformer architecture and includes a recognition network, a generator network, and a conditional network. The recognition network uses a Transformer encoder to encode discontinuous 3D poses into context sequences. The generator network uses a Transformer decoder and a multilayer perceptron (MLP) to generate unoccluded motion using latent codes and context sequences. The conditional network generates a Gaussian distribution for the latent code z.

[0012] S5. Dimensionally reduce the unobstructed 3D posture gait sequence to generate a two-dimensional black and white contour gait sequence image.

[0013] S6. Input the black and white contour image into the gait recognition network to extract features and obtain two-dimensional image gait features;

[0014] S7. Calculate the feature similarity between the two-dimensional image gait features obtained in step S6 and different gait video sequences in the multi-gait dataset. At the same time, use the loss function L to show the difference between the predicted value and the actual value, and train the gait recognition network by updating the parameters through backpropagation. When the value of the loss function L is less than the set threshold, the training is completed.

[0015] S8. Collect pedestrian gait videos. Execute steps S2-S6 to obtain gait features of two-dimensional images. Calculate the similarity between the gait features of two-dimensional images and the features of different gait video sequences in the multi-gait dataset. Use the sample ID corresponding to the gait video sequence with the highest similarity as the multi-gait recognition result.

[0016] Furthermore, in step S1, the collection and organization of gait videos of multiple people walking together is carried out as follows: Walking videos of 1 person, 2 people, and 3 people are simultaneously captured from four angles in an indoor scene. The four angles are four shooting perspectives spaced at 45-degree angles along a semicircle with the pedestrian's walking line as its diameter. Utilizing multi-angle gait data allows the gait recognition network to learn gait features from different angles, improving model accuracy and robustness.

[0017] Further, in step S2, the gait sequence RGB image of a single person is extracted, as follows:

[0018] This paper utilizes YOLOv3 to detect and track individuals in gait video sequences. Each individual in a multi-person walking video sequence is cropped to obtain a single-person RGB gait sequence image. Extracting both single-person and multi-person gait sequences from multi-person walking videos helps the gait recognition network learn invariant high-dimensional features between single-person and multi-person gaits. The YOLOv3 model is a current technology, derived from the paper "YOLOv3: An Incremental Improvement" by Joseph Redmon and Ali Farhadi.

[0019] Further, in step S3, the occluded 3D pose gait sequence is extracted, as follows:

[0020] Using RGB gait sequence images of a single person as input, the 3D pose estimation model HybrIK identifies the human pose from the gait sequence images, resulting in a human body mesh reconstructed based on 3D skeletal joints. The 3D skeletal joints include 29 human skeletal joints: top of head, nose, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hand, right hand, left hip, right hip, left knee, right knee, left ankle, right ankle, left heel, and right heel, used to reconstruct the human body mesh. The 3D pose estimation model HybrIK is existing technology, derived from the paper "Hybrik: A hybrid analytical-neural inverse kinematics solution for 3D human pose and shape estimation," authored by Li, Jiefeng and Xu, Chao and Chen, Zhicun and Bian, Siyuan and Yang, Lixin and Lu, and Cewu.

[0021] Furthermore, the training of the motion filler in step S4 is as follows:

[0022] The overall network design of the motion filler adopts a Transformer architecture, which consists of three parts: (1) a recognition network. A Transformer encoder is used to encode discontinuous 3D poses into context sequences; (2) a generation network. An unoccluded motion is generated using latent codes and context sequences through a Transformer decoder and a multilayer perceptron (MLP); (3) a conditional network. A Gaussian distribution is generated for the latent code z. The overall network flow is as follows: the discontinuous 3D pose is taken as input, and the recognition network generates a context sequence. Then, the context sequence and the continuous 3D pose are input into the conditional network to generate latent codes. Finally, the latent codes and context sequences are input into the generation network to generate continuous 3D poses.

[0023] The network structure is identified as a sequentially connected fully connected layer-1, fully connected layer-2, and Transformer encoder-1.

[0024] The generated network structure consists of a fully connected layer-1, a fully connected layer-2, a Transformer decoder-1, and a multilayer perceptron (MLP) connected in sequence.

[0025] The conditional network structure consists of a fully connected layer-1, a fully connected layer-2, and a Transformer decoder-1, connected in sequence.

[0026] A motion filler was trained using the AMASS motion capture dataset. For M consecutive frames of real motion states in the dataset, the real motion states in frames [N, MN] were randomly occluded, and the first N frames and the last N frames were retained as inputs to the motion filler recognition network. The M consecutive frames of real motion states were used as inputs to the motion filler conditional network, and the gradient values ​​were reduced using an ADMA optimizer with a learning rate of 0.001. The training epochs were 2000, and the batch size was 128. The AMASS dataset is existing technology and comes from the paper AMASS: Archive of Motion Capture as Surface Shapes, authored by Naureen Mahmood, Nima Ghorbani, Nikolaus F. Troje, Gerard Pons-Moll, and Michael J. Black.

[0027] Furthermore, in step S4, a motion filler is used to supplement the occluded 3D posture and gait sequence to obtain an unoccluded 3D posture and gait sequence. The process is as follows:

[0028] The occluded 3D posture and gait sequence obtained in step S3 is input into the motion filler above for inference to generate a continuous, non-missing complete 3D posture and gait sequence.

[0029] Further, in step S5, a two-dimensional black-and-white contour gait sequence image is generated, the process of which is as follows:

[0030] First, obtain the 3D pose vertex information and displacement information for each frame in the complete 3D pose gait sequence. Then, use the PyTorch3D library to generate a rendered image mask. Extract the effective pixels from the image mask, keeping them as white pixels and saving the background as black pixels, forming a black-and-white outline image with the human body in white and the background in black. The process of converting a 3D human pose into a 2D image in PyTorch3D is as follows:

[0031] R1. Projection: PyTorch3D uses camera projection to project 3D points onto a 2D image plane. Camera projection can be represented using a perspective projection matrix, which includes the camera's rotation, translation, and internal parameters (such as focal length and principal point). The expression is:

[0032] pose 2D =K × R × (verts - T)

[0033] Where verts are the vertex coordinates of the 3D pose, R is the camera rotation matrix, T is the camera translation matrix, and K is the camera intrinsic parameter matrix.

[0034] R2. Rasterization: The process of mapping projected 3D points onto an image plane, also known as rasterization. This includes determining which 3D point affects each pixel, and the color at that pixel, etc. This process can be represented as:

[0035] image = F(pose) 2D (faces)

[0036] Among them, pose 2D These are the projected pose coordinates, faces are the surface information of the 3D pose, and F() is the rasterization function.

[0037] R3. Shading: During the rendering process, the rasterized image can be shaded. The shading method depends on the type of shader used. This step can be represented as:

[0038] image 2D =S(image,lights)

[0039] `image` is the rasterized image, `lights` contains light source information, and `S()` is the shader. The output image... 2D The output is white pixels, and the background pixels are set to black, thus obtaining a black and white outline image.

[0040] Further, in step S6, the gait features of the two-dimensional image are extracted. The process is as follows: the gait recognition network structure consists of sequentially connected convolutional layers Conv-1, Conv-2, Conv-3, pooling layer Maxpool-1, convolutional layer Conv-4, convolutional layer Conv-5, pooling layer Maxpool-2, convolutional layer Conv-6, and convolutional layer Conv-7. Convolutional layer Conv-3 outputs the gait features of the shallow stage, convolutional layer Conv-5 outputs the gait features of the middle stage, and convolutional layer Conv-7 outputs the gait features of the deep stage. The output of convolutional layer Conv-7 is used as the gait features of the two-dimensional image.

[0041] Furthermore, the loss function L = L tri +L cla L tri L represents the ternary loss function. cla The expression for the cross-entropy loss function, the ternary loss function Ltri, is as follows:

[0042] L tri =max(d(x1,x2)-d(x1,x3)+margin,0)

[0043] Where d(.,.) represents the Euclidean distance between vectors, margin represents the adjustment factor, max() represents finding the maximum value of two numbers, and x1, x2, and x3 represent the three gait features input respectively;

[0044] Cross-entropy loss function L cla The expression is:

[0045]

[0046] Where D(Y) X |X) represents the predicted label Y corresponding to the complete gait feature X. X The probability, Y X X represents the true label corresponding to the complete gait feature X, and N represents the number of gait sequences.

[0047] Furthermore, the multi-gait recognition method also includes a testing step, the process of which is as follows:

[0048] T1. The test set is divided into two parts: galleries and probes. The gallery sequence consists of 6 individual walking gait data. The probe sequence consists of 4 multi-person walking gait data. Each label in the gallery sequence is unique.

[0049] T2. Input the gait data of the probe sequence and the gallery sequence into the gait recognition network respectively to obtain the probe gait vector and the gallery gait vector respectively;

[0050] T3. Calculate the Euclidean distance between the probe gait vector and the gallery gait vector respectively, and find their minimum value to obtain the gallery gait vector with the highest similarity to the probe gait vector, thus completing the multi-gait recognition.

[0051] The present invention has the following advantages and effects compared with the prior art:

[0052] 1. This invention defines gait recognition for multiple people walking together, extracting a sequence of gait images of a single individual from images of multiple people's gait, and extracting black-and-white contour images and human skeletal joint data from them. The testing process uses single-person walking gait data as a gallery and multiple-person walking gait data as a probe, comparing the two walking states to identify the identity of a single individual among the walking group.

[0053] 2. This invention fully considers the gait of multiple people walking together in a real environment, and captures a multi-gait dataset that conforms to the actual situation, providing data support for multi-gait recognition based on occlusion repair.

[0054] 3. This invention combines occlusion repair and gait recognition technologies to provide a novel solution and method for gait recognition in multi-person walking situations. Current multi-gait research utilizes occluded or incomplete gait features for gait recognition, resulting in low recognition rates for existing gait recognition models in multi-gait scenarios. The method proposed in this invention can effectively extract complete gait features of pedestrians and improve the accuracy of multi-gait recognition while reducing the impact of clothing. Attached Figure Description

[0055] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0056] Figure 1 This is a flowchart of a multi-gait recognition method based on occlusion repair disclosed in this invention;

[0057] Figure 2 This is a schematic diagram of data capture in an embodiment of the present invention;

[0058] Figure 3 This is a schematic diagram of the dataset in an embodiment of the present invention;

[0059] Figure 4 This is a schematic diagram of pedestrian segmentation in an embodiment of the present invention;

[0060] Figure 5 This embodiment of the invention extracts a person's 3D pose from a single person's RGB gait sequence image.

[0061] Figure 6 This is a schematic diagram of the overall structure of the motion filler in an embodiment of the present invention;

[0062] Figure 7 This is a schematic diagram of the generation network, recognition network, and conditional network structure of the motion filler in this embodiment of the invention;

[0063] Figure 8This is a schematic diagram of the gait recognition network structure in an embodiment of the present invention. Detailed Implementation

[0064] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0065] Example

[0066] This embodiment specifically discloses a multi-gait recognition method based on occlusion repair, the flowchart of which is as follows: Figure 1 As shown, the multi-gait recognition method includes the following steps:

[0067] S1. Collect and organize videos of multiple people walking together in indoor settings, such as... Figure 3 As shown, videos of one, two, and three people walking together from multiple perspectives were collected to obtain gait video sequences, forming a multi-gait dataset, which was divided into training and testing sets.

[0068] In step S1 of this embodiment, the process of collecting and organizing gait videos of multiple people walking together is as follows:

[0069] In an indoor setting, walking videos of 1 person, 2 people, and 3 people were simultaneously captured from four angles. The four angles were located on a semicircle with the walking line as its diameter, spaced at 45-degree intervals. The filming process was as follows: Figure 2 As shown.

[0070] S2. Extract RGB gait sequence images of a single person from the gait video sequence. The visualization results are as follows: Figure 4 As shown;

[0071] In step S2 of this embodiment, the operation of extracting the RGB gait sequence image of a single person from the gait video sequence is as follows: YOLOv3 is used to detect and track people in the gait video sequence; each person in the multi-person video sequence is cropped to obtain the RGB gait sequence image of a single person. The YOLOv3 model is existing technology, derived from the paper "YOLOv3: An Incremental Improvement" by Joseph Redmon and Ali Farhadi.

[0072] S3. Organize and classify the RGB gait sequence images of the above single person, extract the person's 3D pose from the RGB gait sequence images, and obtain the occluded 3D pose gait sequence. Visualization images are shown below. Figure 5 As shown;

[0073] In this embodiment, step S3 involves extracting the 3D pose of the character, and the process is as follows:

[0074] Using RGB gait sequence images of a single person as input, the 3D pose estimation model HybrIK identifies the human pose from the gait sequence images, resulting in a human body mesh reconstructed based on 3D skeletal joints. The 3D skeletal joints include 29 human skeletal joints: top of head, nose, neck, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hand, right hand, left hip, right hip, left knee, right knee, left ankle, right ankle, left heel, and right heel, used to reconstruct the human body mesh. The 3D pose estimation model HybrIK is existing technology, derived from the paper "Hybrik: A hybrid analytical-neural inverse kinematics solution for 3D human pose and shape estimation," authored by Li, Jiefeng and Xu, Chao and Chen, Zhicun and Bian, Siyuan and Yang, Lixin and Lu, and Cewu.

[0075] S4. Train the motion filler and use the motion filler to fill in the occluded 3D posture and gait sequence to obtain an unoccluded complete 3D posture and gait sequence.

[0076] In step S4 of this embodiment, the process of training the motion filler is as follows:

[0077] The overall network design of the motion filler adopts a Transformer architecture, which consists of three parts: (1) a recognition network. A Transformer encoder encodes discontinuous 3D poses into context sequences; (2) a generator network. A Transformer decoder and a multilayer perceptron (MLP) generate unoccluded motion using latent codes and context sequences; and (3) a conditional network. A Gaussian distribution is generated for the latent code z. The overall network flow is as follows: Discontinuous 3D poses are taken as input, and the recognition network generates context sequences. The context sequences and continuous 3D poses are then input into the conditional network to generate latent codes. Finally, the latent codes and context sequences are input into the generator network to generate continuous 3D poses. Visualization images are shown below. Figure 6 As shown.

[0078] The network structure is identified as a sequentially connected fully connected layer-1, fully connected layer-2, and Transformer encoder-1. For example... Figure 7 As shown.

[0079] The generative network structure consists of a sequentially connected fully connected layer-1, a fully connected layer-2, a Transformer decoder-1, and a multilayer perceptron (MLP). For example... Figure 7 As shown.

[0080] The conditional network structure consists of a sequentially connected fully connected layer-1, a fully connected layer-2, and a Transformer decoder-1. For example... Figure 7 As shown.

[0081] A motion filler was trained using the AMASS motion capture dataset. For M consecutive frames of real motion states in the dataset, the real motion states in frames [N, MN] were randomly occluded, and the first N frames and the last N frames were retained as inputs to the motion filler recognition network. The M consecutive frames of real motion states were used as inputs to the motion filler conditional network, and the gradient values ​​were reduced using an ADMA optimizer with a learning rate of 0.001. The training epochs were 2000, and the batch size was 128. The AMASS dataset is existing technology and comes from the paper AMASS: Archive of Motion Capture as Surface Shapes, authored by Naureen Mahmood, Nima Ghorbani, Nikolaus F. Troje, Gerard Pons-Moll, and Michael J. Black.

[0082] In step S4 of this embodiment, the operation of obtaining an unobstructed complete 3D pose and gait sequence using a motion filler is as follows:

[0083] The occluded 3D posture and gait sequence obtained in step S3 is input into the motion filler above for inference to generate a continuous, non-missing complete 3D posture and gait sequence.

[0084] S5. Reduce the dimensionality of the unobstructed 3D pose gait sequence obtained in S4 to generate a two-dimensional black and white contour gait sequence image. For example... Figure 6 As shown.

[0085] In this embodiment, step S5 involves generating a two-dimensional black-and-white contour gait sequence image, and the process is as follows:

[0086] First, obtain the 3D pose vertex information and displacement information for each frame in the complete 3D pose gait sequence. Then, use the PyTorch3D library to generate a rendered image mask. Extract the effective pixels from the image mask, keeping them as white pixels and saving the background as black pixels, forming a black-and-white outline image with the human body in white and the background in black. The process of converting a 3D human pose into a 2D image in PyTorch3D is as follows:

[0087] R1. Projection: PyTorch3D uses camera projection to project 3D points onto a 2D image plane. Camera projection can be represented using a perspective projection matrix, which includes the camera's rotation, translation, and internal parameters (such as focal length and principal point). The expression is:

[0088] pose 2D =K × R × (verts-t)

[0089] Where verts are the vertex coordinates of the 3D pose, R is the camera rotation matrix, T is the camera translation matrix, and K is the camera intrinsic parameter matrix.

[0090] R2. Rasterization: The process of mapping projected 3D points onto an image plane, also known as rasterization. This includes determining which 3D point affects each pixel, and the color at that pixel, etc. This process can be represented as:

[0091] image = F(pose) 2D (faces)

[0092] Among them, pose 2D These are the projected pose coordinates, faces are the surface information of the 3D pose, and F() is the rasterization function.

[0093] R3. Shading: During the rendering process, the rasterized image can be shaded. The shading method depends on the type of shader used. This step can be represented as:

[0094] image 2D =S(image,lights)

[0095] `image` is the rasterized image, `lights` contains light source information, and `S()` is the shader. The output image... 2D The output is white pixels, and the background pixels are set to black, thus obtaining a black and white outline image.

[0096] S6. Input the black and white contour image into the gait recognition network for feature extraction to obtain two-dimensional image gait features; the gait recognition network structure is as follows: Figure 8 As shown;

[0097] Step S6 extracts the gait features of the two-dimensional image. The process is as follows: The gait recognition network structure consists of sequentially connected convolutional layers Conv-1, Conv-2, Conv-3, pooling layer Maxpool-1, convolutional layer Conv-4, convolutional layer Conv-5, pooling layer Maxpool-2, convolutional layer Conv-6, and convolutional layer Conv-7. Convolutional layer Conv-3 outputs the gait features of the shallow stage, convolutional layer Conv-5 outputs the gait features of the middle stage, and convolutional layer Conv-7 outputs the gait features of the deep stage. The output of convolutional layer Conv-7 is used as the gait feature of the two-dimensional image.

[0098] S7. Calculate the feature similarity between the overall gait features obtained in step S6 and the different gait video sequences in the multi-gait dataset. At the same time, use the loss function L to display the difference between the predicted value and the actual value, and update the parameters through backpropagation for training. When the value of the loss function L is less than the set threshold, the training is completed.

[0099] In this embodiment, step S7 updates the model parameters using the loss function L, as follows:

[0100] The loss function L = L tri +L cla L tri L represents the ternary loss function. cla The expression for the cross-entropy loss function, the ternary loss function Ltri, is as follows:

[0101] L tri =max(d(x1,x2)-d(x1,x3)+margin,0)

[0102] Where d(.,.) represents the Euclidean distance between vectors, margin represents the adjustment factor, max() represents finding the maximum value of two numbers, and x1, x2, and x3 represent the three gait features input respectively;

[0103] Cross-entropy loss function L cla The expression is:

[0104]

[0105] Where D(Y) X |X) represents the predicted label Y corresponding to the complete gait feature X. X The probability, Y X X represents the true label corresponding to the complete gait feature X, and N represents the number of gait sequences.

[0106] S8. Collect pedestrian gait videos and execute steps S2-S6 to obtain overall gait features. Calculate the similarity between the overall gait features and the features of different gait video sequences in the multi-gait dataset. Use the sample ID corresponding to the gait video sequence with the highest similarity as the multi-gait recognition result.

[0107] The testing step S8 in this embodiment is as follows:

[0108] T1. The test set is divided into two parts: galleries and probes. The gallery sequence consists of 6 single-person walking gait data. The probe sequence consists of 4 multi-person walking gait data. Each label in the gallery sequence is different.

[0109] T2. Input the gait data of the probe sequence and the gallery sequence into the gait recognition network respectively to obtain the probe gait vector and the gallery gait vector respectively;

[0110] T3. Calculate the Euclidean distance between the probe gait vector and the gallery gait vector respectively, and perform a minimization operation to obtain the gallery gait vector with the highest similarity to the probe gait vector, thus completing multi-gait recognition. Table 1 shows the difference in recognition accuracy between occluded gait sequences and occluded repaired gait sequences. The average accuracy of gait recognition after occlusion repair is 4.56% higher than that without repair, indicating that occlusion repair technology can more comprehensively represent gait data.

[0111] Table 1. Recognition accuracy under different occlusion conditions

[0112] single Two people Three people Average accuracy Obstructing gait 72.3％ 60.12％ 52.34％ 61.59％ Gait restoration by occlusion 74.09％ 67.45％ 56.9％ 66.15％

[0113] In summary, this embodiment discloses a novel multi-gait recognition method based on occlusion restoration, which offers technical advantages such as higher recognition performance and stronger robustness in the face of gait changes and occlusion in multi-gait scenarios. Occlusion restoration technology can restore the gait representation of an occluded person to the greatest extent possible, thus providing data support for gait recognition networks. While reducing the impact of noise, it further simplifies the influence of clothing and environmental factors on gait recognition networks, further improving the performance and reliability of multi-gait recognition. The multi-gait recognition method disclosed in this embodiment solves the recognition challenges caused by partial occlusion and gait changes in multi-gait scenarios, playing a significant role in promoting the industrialization of multi-gait recognition.

[0114] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A multi-gait recognition method based on occlusion repair, characterized in that, The gait recognition method comprises the following steps: S1, collecting and sorting multiple people walking gait videos in an indoor scene, collecting 1 person, 2 people and 3 people walking videos from multiple perspectives, obtaining gait video sequences, taking the gait video sequences as a multi-gait dataset, and dividing the multi-gait dataset into a training set and a test set; S2, extracting single-person RGB gait sequence images from the gait video sequences; S3, extracting a person's 3D pose from the single-person RGB gait sequence images to obtain an occluded 3D pose gait sequence; S4, training a motion filler and using the motion filler to supplement the occluded 3D pose gait sequence to obtain a non-occluded 3D pose gait sequence, wherein the motion filler adopts a Transformer architecture, comprising an identification network, a generation network and a conditional network, the identification network uses a Transformer encoder to encode non-continuous 3D poses into context sequences; the generation network generates non-occluded motion using a Transformer decoder and a multi-layer perceptron (MLP) using latent codes and context sequences, and the conditional network generates a Gaussian distribution of latent codes z; The structure of the identification network is sequentially connected full connection layer-1, full connection layer-2, Transformer encoder-1; The structure of the generation network is sequentially connected full connection layer-1, full connection layer-2, Transformer decoder-1, and multi-layer perceptron (MLP); The structure of the conditional network is sequentially connected full connection layer-1, full connection layer-2, and Transformer decoder-1; S5, dimensionally reducing the non-occluded 3D pose gait sequence to generate a two-dimensional black-and-white contour gait sequence image; S6, inputting the black-and-white contour gait sequence image into a gait recognition network to extract two-dimensional image gait features; S7, calculating the feature similarity between the two-dimensional image gait features obtained in step S6 and different gait video sequences in the multi-gait dataset, simultaneously displaying the gap between the predicted value and the actual value using a loss function L, and updating the parameters through back propagation to train the gait recognition network, when the value of the loss function L is less than a set threshold, the training is completed; S8, collecting pedestrian gait videos to obtain two-dimensional image gait features by executing steps S2-S6, calculating the feature similarity between the two-dimensional image gait features and different gait video sequences in the multi-gait dataset, and taking the sample ID corresponding to the gait video sequence with the highest similarity as the multi-gait recognition result.

2. The multi-gait recognition method based on occlusion repair according to claim 1, characterized in that, The process of collecting and sorting multiple people walking gait videos in step S1 is as follows: 1 person, 2 people and 3 people walking videos are simultaneously collected from four perspectives in an indoor scene, wherein the four perspectives are four shooting perspectives with a 45-degree angle interval on a semicircle with the walking straight line of the pedestrians as the diameter.

3. The multi-gait recognition method based on occlusion repair according to claim 1, wherein, The process of step S2 is as follows: Person detection and tracking are performed on the gait video sequences, and each person in the gait video sequences is cropped to obtain single-person RGB gait sequence images.

4. The multi-gait recognition method based on occlusion repair according to claim 1, wherein, The process of step S3 is as follows: The single-person RGB gait sequence image is taken as an input to perform multi-person pose estimation to recognize human poses and obtain a 3D pose of a single person that is occluded.

5. The multi-gait recognition method based on occlusion repair according to claim 1, wherein, The step S4 is performed as follows: The 3D pose gait sequence that is occluded is taken as an input of a motion filler, the 3D pose gait sequence that is occluded is a non-continuous 3D pose, a context sequence is generated by using a recognition network, the context sequence and the continuous 3D pose are input into a conditional network to generate a latent code, and finally the latent code and the context sequence are input into a generation network to generate a continuous 3D pose.

6. The multi-gait recognition method based on occlusion repair according to claim 4, characterized in that, In the step S5, a mask of a human body grid in the single-person 3D pose is output as a black-and-white contour gait sequence image of the single person, wherein the background is black pixels and the human body is white pixels.

7. The multi-gait recognition method based on occlusion repair according to claim 1, wherein, The gait recognition network comprises sequentially connected convolution layers Conv-1, Conv-2, Conv-3, a pooling layer Maxpool-1, convolution layers Conv-4, Conv-5, a pooling layer Maxpool-2, convolution layers Conv-6 and Conv-7, and the output of the convolution layer Conv-7 is taken as a gait feature.

8. The multi-gait recognition method based on occlusion repair according to claim 1, wherein, the loss function wherein denotes a tri- loss function, denotes a cross-entropy loss function, a tri- loss function the expression of which is: wherein denotes the Euclidean distance between vectors, denotes an adjustment factor, ( ) represents the maximum of two numbers, respectively represent the three gait features input. cross-entropy loss function The expression is: wherein represents the probability that the predicted label corresponding to the complete gait feature X is represents the true label corresponding to the complete gait feature X, and N represents the number of gait sequences.​ 9. The multi-gait recognition method based on occlusion repair according to claim 1, wherein, The multi-gait recognition method further comprises a test step, and the process is as follows: T1, the test set is divided into a gallery and a probe, single-person walking gait data is taken as a gallery sequence, and multi-person walking gait data is taken as a probe sequence, and each label of the gallery sequence is different; T2, the gait data of the probe sequence and the gallery sequence are respectively input into the gait recognition network to respectively obtain a probe gait vector and a gallery gait vector; T3, the Euclidean distances of the probe gait vector and the gallery gait vector are respectively calculated, and a minimum operation is performed to obtain a gallery gait vector that has the highest similarity with the probe gait vector, and then identity recognition is completed.

Images