Gait recognition method and device based on body shape transformation and storage medium

Abstract

The application discloses a gait recognition method and device based on body shape transformation and a storage medium. The gait recognition method comprises the following steps: acquiring a gait image sequence, wherein the gait image sequence comprises a plurality of gait images, and acquiring a target gait contour and a target key point matrix corresponding to the same target based on each gait image; and performing width-height ratio transformation on a target contour in each target gait contour to obtain a target body shape transformation contour. Further, based on the target gait contour and the target body shape transformation contour, a gait space-time extraction network is used to acquire gait space-time features, based on the target key point matrix, a key point feature extraction network is used to acquire target key point features, then, the gait space-time features and the target key point features are fused to obtain fused measurement features. Through the above method, the application can effectively solve the problem of low target gait recognition accuracy caused by clothing obstruction, carrying objects and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of biometric recognition, in particular to a gait recognition method based on body shape transformation, a device and a storage medium. BACKGROUND

[0002] The physiological features of human body mainly include fingerprints, palm prints and facial features, and the behavioral features mainly include gait features, voice features, signatures and the like. In recent years, the biometric recognition technology based on gait features has gradually been applied. Gait recognition refers to feature extraction and gait recognition on the basis of extracting moving targets, and the gait recognition technology can complete collection under non-contact and long-distance conditions, so as to realize identity recognition under non-contact and long-distance conditions. Although the research on gait recognition technology is more and more in-depth and comprehensive, the current gait recognition still has the problem of low gait recognition accuracy due to the change of view angle (under different view angles, the posture of human body is different, and the gait features collected also have differences), the change of dress (the clothes are too long / thick, which causes the occlusion of the leg contour of human body, etc.) and the change of carrying objects (the carrying of backpacks / handbags and the like affect the integrity of gait contour map). SUMMARY

[0003] The present application provides a gait recognition method based on body shape transformation, a device and a storage medium, which can effectively solve the problem of low gait recognition accuracy caused by factors such as clothing occlusion in gait recognition by obtaining a target gait contour map, a target key point matrix and a target body shape transformation contour map.

[0004] In order to solve the above technical problems, one technical scheme adopted by the present application is to provide a gait recognition method based on body shape transformation, which comprises the following steps: obtaining a gait image sequence, wherein the gait image sequence comprises a plurality of gait images; obtaining a target gait contour map and a target key point matrix corresponding to the same target based on each gait image; performing width-height ratio transformation on the target contour in each target gait contour map to obtain a target body shape transformation contour map; obtaining gait space-time features by using a gait space-time extraction network based on the target gait contour map and the target body shape transformation contour map; obtaining target key point features by using a key point feature extraction network based on the target key point matrix; and performing feature fusion on the gait space-time features and the target key point features to obtain fusion measurement features.

[0005] In the method, the step of obtaining the gait image sequence comprises the following steps: obtaining an original image sequence formed by photographing the target in the walking process, wherein the original image sequence comprises a plurality of original images; performing target detection and target tracking on each original image; and selecting original images in which the target leg is not occluded from the plurality of original images to form the gait image sequence.

[0006] The process of transforming the aspect ratio of the target gait contour map to obtain the target body shape transformation contour map includes: inputting the original aspect ratio of the target contour of each target gait contour map into the body shape transformation function to obtain the corresponding target aspect ratio; or obtaining the corresponding target aspect ratio based on the expectation and standard deviation of the set of original aspect ratios of the target contours of the target gait contour; and transforming the target contour based on the target aspect ratio.

[0007] The process of inputting the original aspect ratio of the target contour of each target gait contour map into the body shape transformation function includes: using the body shape transformation function to calculate one of the maximum, minimum, mode, mean, or median of multiple original aspect ratios as the target aspect ratio; or, obtaining the expectation and standard deviation of the normally distributed set of original aspect ratios, and then selecting the target aspect ratio from the range of values ​​formed by the expectation and standard deviation.

[0008] Among them, the aspect ratios of the targets corresponding to the gait contour maps of each target may be the same or different.

[0009] Specifically, based on the target gait contour map and the target body shape transformation contour map, the gait spatiotemporal extraction network is used to obtain gait spatiotemporal features, including: stitching the corresponding target gait contour map and the target body shape transformation contour map according to the channel dimension to form a fused contour map; and using the gait spatiotemporal extraction network to extract gait spatiotemporal features from the fused contour map.

[0010] The process involves obtaining target keypoint features based on the target keypoint matrix using a keypoint feature extraction network, including: performing average pooling on the target keypoint features in the horizontal direction using at least two pooling kernels with the same width but different lengths; using fully connected layers to linearly map the output results of each pooling kernel, wherein the fully connected layers corresponding to different pooling layers are independent of each other; and arranging and concatenating the outputs of the fully connected layers in order according to the kernel size of the corresponding pooling kernel.

[0011] To solve the above-mentioned technical problems, another technical solution adopted in this application is to provide a gait recognition device based on body shape transformation. The device includes a memory and a processor. The memory is used to store program data, and the processor is used to execute the program data to realize the gait recognition method based on body shape transformation as described above.

[0012] To solve the above-mentioned technical problems, another technical solution adopted in this application is to provide a computer-readable storage medium that stores program data. When the program data is executed by a processor, it is used to implement the gait recognition method based on body shape transformation as described above.

[0013] The beneficial effects of the present application are: different from the prior art, the gait recognition method based on body shape transformation provided by the present application, by acquiring the target gait contour graph, the target key point matrix of each gait image in the gait image sequence, and the target body shape transformation contour graph obtained by performing aspect ratio transformation on the target contour in the target gait contour graph; further, based on the target gait contour graph and the target body shape transformation contour graph, the gait space-time extraction network is used to acquire the gait space-time feature, based on the target key point matrix, the key point feature extraction network is used to acquire the target key point feature; finally, the gait space-time feature and the target key point feature are fused to obtain the fusion measurement feature. In an embodiment, by acquiring the target gait contour graph, the target key point matrix and the target body shape transformation contour graph in each gait image, the combination of the three information can effectively solve the problem of low gait recognition accuracy caused by clothing shielding, carrying objects and the like, and can improve the accuracy of target gait recognition. BRIEF DESCRIPTION OF DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor. Among them:

[0015] Figure 1 is a flowchart of a first embodiment of the gait recognition method based on body shape transformation provided by the present application;

[0016] Figure 2 is a flowchart of a second embodiment of the gait recognition method based on body shape transformation provided by the present application;

[0017] Figure 3 is a flowchart of a third embodiment of the gait recognition method based on body shape transformation provided by the present application;

[0018] Figure 4 is a flowchart of a fourth embodiment of the gait recognition method based on body shape transformation provided by the present application;

[0019] Figure 5 is a flowchart of a fifth embodiment of the gait recognition method based on body shape transformation adopted by the present application;

[0020] Figure 6 is a flowchart of a sixth embodiment of the gait recognition method based on body shape transformation adopted by the present application;

[0021] Figure 7 is a structural schematic diagram of an embodiment of the gait recognition device based on body shape transformation provided by the present application;

[0022] Figure 8 FIG. 1 is a structural schematic diagram of an embodiment of the computer readable storage medium provided in the present application. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the protection scope of the present application.

[0024] Reference Figure 1 , Figure 1 FIG. 1 is a flow schematic diagram of a first embodiment of the gait recognition method based on body shape transformation provided in the present application, and the specific steps are as follows:

[0025] Step 11: acquiring a gait image sequence, wherein the gait image sequence comprises a plurality of gait images.

[0026] Specifically, the data acquisition in gait recognition is similar to the face data acquisition, and the input is a video sequence, but gait recognition is to recognize one frame of image after another, that is, the acquired video data is segmented by frame, and each frame containing the image of the target person is a gait image.

[0027] In an embodiment, the gait images in the gait image sequence are arranged in time sequence.

[0028] Step 11 of acquiring the gait image sequence specifically comprises the following steps (not shown in the figure):

[0029] S1: acquiring an original image sequence formed by shooting the target in the walking process, wherein the original image sequence comprises a plurality of original images.

[0030] Specifically, a video sequence is segmented by frame, and one frame obtains one image, that is, the image sequence containing the target can be acquired from the video containing the target.

[0031] S2: respectively performing target detection and target tracking on the original images.

[0032] Specifically, gait recognition mainly uses gait features to identify the identity of the human target, and the process of gait recognition can be divided into three parts of human target detection, feature extraction and gait recognition. Human target detection is to detect and extract the specified target from the video or image sequence, that is, it includes two parts of target detection and target tracking.

[0033] Target tracking can be categorized into single-target tracking (VOT / SOT), multiple-object tracking (MOT), person re-identification (Person Re-ID), multi-target multi-camera tracking (MTMCT), and pose tracking.

[0034] Target tracking can use tracking algorithms to track human targets. Optionally, tracking algorithms include DeepSORT, FairMOT, etc.

[0035] Specifically, the DeepSORT algorithm integrates motion and appearance information by combining two metrics: Mahalanobis distance and cosine similarity. Appearance information refers to the use of a simple CNN network to extract the appearance features of the detected object; motion information refers to the prediction results of Kalman filtering.

[0036] Specifically, FairMOT is an end-to-end algorithm that integrates detection and tracking. The detection part is based on CenterNet, and the tracking part is similar to DeepSORT.

[0037] S3: Select the original images from the plurality of original images in which the target leg is not obscured to form the gait image sequence.

[0038] Specifically, the statement that the target's legs are not obscured means that there are no other objects obscuring the human target in the original image, and does not mean that the human target is obscured by clothing worn by the target.

[0039] Understandably, a gait image sequence consists of several images of the target leg that are not occluded.

[0040] Step 12: Based on each of the gait images, obtain the target gait contour map and the target key point matrix corresponding to the same target.

[0041] Specifically, in one embodiment, each gait image is input into a trained semantic segmentation model. The gait images are segmented frame by frame, separating the foreground and background. Then, the separated foreground and background images are binarized, i.e., the pixel values ​​of the foreground image are set to a first value, and the pixel values ​​of the background image are set to a second value different from the first value. Foreground extraction is then performed using foreground extraction techniques. In one embodiment, the pixel value of the foreground image is set to 255 (eight bits), and the pixel value of the background image is set to 0. Of course, in other embodiments, the first and second pixel values ​​can be set to other values, such as sixteen bits, twenty-four bits, etc.

[0042] Specifically, in an embodiment, each of the gait images is input into a trained human key point detection model, and the gait images are detected frame by frame to obtain coordinate information and confidence of all key points of the human target, wherein each coordinate information corresponds to a confidence, and then the coordinate information of the key points and the confidence of the key points are sorted in a predetermined order (such as from left to right, from top to bottom, etc.). In an embodiment, the human key points include 17 key points, including left eye, right eye, nose tip, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle.

[0043] Step 13: performing aspect ratio transformation on the target contour in each of the target gait contour maps to obtain a target body shape transformation contour map.

[0044] Specifically, the target contour is obtained based on the target gait contour map, and the aspect ratio of the target contour is obtained, and then the aspect ratio of the target contour is changed to obtain a new target contour, and the image containing the new target contour is a target body shape transformation function.

[0045] Optionally, the aspect ratio of the new target contour can be the maximum value, the minimum value, the mode, the mean value, the median value, etc. of the aspect ratios of all target contours, or can be the expectation and standard deviation obtained after normal distribution of the aspect ratio set of all target contours, and then the value range is determined based on the expectation and the standard deviation, such as the value can be taken from the interval [expectation-standard deviation, expectation+standard deviation].

[0046] (Step 14 and step 15 are not distinguished in order)

[0047] Step 14: obtaining gait space-time features based on the target gait contour map and the target body shape transformation contour map by using a gait space-time extraction network.

[0048] In an embodiment, the target gait contour map and the target body shape transformation contour map are both binary human contour maps, and the image sizes are the same (the human contours are different).

[0049] In an embodiment, the gait space-time extraction network is an existing gait recognition network Gait Lateral Network (GLN). Of course, in other embodiments, it can be other deep gait feature extraction networks containing convolutional layers, pooling layers, activation layers, fully connected layers, and residual connections, etc., which are not limited here.

[0050] Step 15: obtaining target key point features based on the target key point matrix by using a key point feature extraction network.

[0051] Specifically, since the coordinate information of the key points in the key point matrix and the confidence corresponding to the coordinates correspond in the spatial position, an m x n matrix can be obtained, where m is the number of key points, and n is equal to the sum of the coordinate dimension and the confidence dimension.

[0052] Specifically, in an embodiment, the key point feature extraction network is a graph convolutional neural network, which can perform convolution operation on graph data and further extract the features of the graph. Since the graph data is a data composed of nodes and edges, the target key point matrix can be used to obtain the graph data, where the human key points can be regarded as nodes, and the connection relationship between the key points can be regarded as edges for connecting the nodes.

[0053] Specifically, inputting the target key point matrix into the key point feature extraction network can obtain the topological features and temporal features of the human key points.

[0054] It is worth noting that the graph convolutional neural network used in the present application is different from the existing graph convolutional neural network in that the graph convolutional neural network of the present application has a feature block reorganization module for extracting more detailed key point features.

[0055] Step 16: performing feature fusion on the gait space-time features and the target key point features to obtain fusion measurement features.

[0056] Specifically, the gait space-time features and the target key point features are mapped to the same dimension and spliced to obtain a plurality of fusion measurement features.

[0057] In addition, when the above-mentioned fusion measurement features need to be classified, each of the fusion measurement features can be input into a full connection layer and a softmax layer to obtain fusion classification features.

[0058] In addition, model training can be performed using the fusion measurement features and the fusion classification features to obtain a more optimal model (with less loss). Specifically, a horizontal pyramid method is used to horizontally block the fusion measurement features and the fusion classification features, and a triplet loss function is used to calculate the horizontal blocks of the fusion measurement features, and a cross-entropy loss function is used to calculate the horizontal blocks of the fusion classification features. Through the above-mentioned method, the model loss can be reduced during model training, and a better model can be obtained.

[0059] In addition, the fusion classification features can be used to test the model.

[0060] Different from the prior art, the gait recognition method based on body shape transformation provided in the application can effectively solve the problems of changes in the human body contour caused by clothing obstruction and carrying objects and the like by combining the target gait contour graph, the target key point matrix and the target body shape transformation contour graph in each gait image, thereby further improving the accuracy of target gait recognition.

[0061] Referring to Figure 2 , Figure 2 is a flowchart of the second embodiment of the gait recognition method based on body shape transformation provided in the application, and the specific steps are as follows:

[0062] Step 21: obtaining a gait image sequence, wherein the gait image sequence comprises a plurality of gait images.

[0063] Specifically, the gait image sequence is obtained from a gait video, wherein the gait video refers to a video containing a human target walking, the gait video is segmented by frames to obtain a plurality of gait images containing the target human body, and a plurality of gait images arranged in time sequence form a gait image sequence.

[0064] Step 22: obtaining a target gait contour graph and a target key point matrix corresponding to the same target based on each gait image.

[0065] Specifically, in an embodiment, each gait image is input into a trained semantic segmentation model to obtain a target gait contour graph, and the gait image is input into a trained human key point detection model to obtain a target key point matrix.

[0066] (Steps 23 and 24 do not exist at the same time)

[0067] Step 23: inputting the original width-height ratio of the target contour of each target gait contour graph into a body shape transformation function to obtain a corresponding target width-height ratio.

[0068] Specifically, the body shape transformation function is determined by the width-height ratios of the target contours in all gait contour graphs.

[0069] Specifically, the target width-height ratio can be the mode, maximum value, minimum value or mean value of the width-height ratios of all target contours, that is, the target width-height ratio can be the same as or different from the original width-height ratio.

[0070] Specifically, the definition of the width-height ratio is as follows: the leftmost point and the rightmost point of the target contour are obtained, the difference between the horizontal coordinates of the leftmost point and the rightmost point is defined as the width of the target contour, the highest point and the lowest point of the target contour are obtained, the difference between the vertical coordinates of the highest point and the lowest point is defined as the height of the target contour, and the difference between the width of the target contour and the height of the target contour is the width-height ratio of the target contour.

[0071] Step 24: based on the expectation and standard deviation of the original aspect ratio set of the target profile of the target gait profile, a corresponding target aspect ratio is obtained.

[0072] Specifically, in an embodiment, the set of original aspect ratios is normally distributed, that is, the target aspect ratio is obtained by obtaining the expectation and standard deviation of the set of original aspect ratios that are normally distributed.

[0073] Specifically, for how to obtain the target aspect ratio, please refer to Figure 3 .

[0074] Step 25: transform the target profile based on the target aspect ratio to obtain a target body shape transformation profile map.

[0075] Specifically, based on the target aspect ratio, by stretching, deforming, etc. Frame by frame operation on the target profile, the target body shape transformation profile map can be obtained.

[0076] Step 26: based on the target gait profile and the target body shape transformation profile, the gait space-time extraction network is used to obtain the gait space-time feature.

[0077] Specifically, in an embodiment, the gait space-time extraction network is Gait Lateral Network (GLN), and the target gait profile and the target body shape transformation profile can be used to obtain the time dimension and space-time dimension features.

[0078] Specifically, for how to obtain the gait space-time feature, please refer to Figure 4 , which will not be described in detail here.

[0079] Step 27: based on the target key point matrix, the key point feature extraction network is used to obtain the target key point feature.

[0080] Specifically, in an embodiment, the key point feature extraction network is a graph convolutional neural network, and the target key point matrix is input into the graph convolutional neural network to obtain the target key point.

[0081] Specifically, for how to obtain the target key point feature, please refer to Figure 4 , which will not be described in detail here.

[0082] Step 28: the gait space-time feature and the target key point feature are fused to obtain a fusion measurement feature.

[0083] Specifically, the gait space-time feature and the target key point feature are mapped to the same dimension and spliced to obtain the fusion measurement feature.

[0084] Different from the prior art, the gait recognition method based on body shape transformation provided in the application can effectively solve the problem of low gait recognition accuracy caused by clothing obstruction, carrying objects and the like, and further improve the accuracy of target gait recognition.

[0085] The way of obtaining the target aspect ratio is described below.

[0086] Referring to Figure 3 , Figure 3 is a flowchart of the third embodiment of the gait recognition method based on body shape transformation provided in the application, and the specific steps are as follows:

[0087] Step 31: using the body shape transformation function to calculate one of the maximum value, minimum value, mode value, mean value or median value of the plurality of original aspect ratios as the target aspect ratio.

[0088] It can be understood that the target aspect ratio may be the same as or different from the original aspect ratio.

[0089] (Steps 31 and steps 32-33 do not exist at the same time, that is, the way of obtaining the target aspect ratio can be obtained by the way described in step 31, or can be obtained by the way described in steps 32-33)

[0090] Step 32: obtaining the expectation and standard deviation of the original aspect ratio set in normal distribution.

[0091] It can be understood that all original aspect ratios are approximated as normal distribution, and the expectation and standard deviation in the normal distribution are obtained.

[0092] Step 33: selecting the target aspect ratio from the value range formed by the expectation and standard deviation.

[0093] Specifically, in an embodiment, the value range formed based on the expectation and standard deviation is [expectation-standard deviation, expectation+standard deviation].

[0094] It is worth noting that the target aspect ratios corresponding to the target gait profile graphs may be the same or different.

[0095] Different from the prior art, the above method can change the profile of the target in the target gait profile graph to obtain a target body shape transformation profile.

[0096] The way of obtaining the gait space-time feature is described in detail below.

[0097] Referring to Figure 4 , Figure 4 is a flowchart of the fourth embodiment of the gait recognition method based on body shape transformation provided in the application, and the specific steps are as follows:

[0098] Step 41: splice the corresponding target gait contour map and target body shape transformation contour map in the channel dimension to form a fusion contour map.

[0099] Specifically, the size and quantity of the target gait contour map and the target body shape transformation contour map are corresponding, so that the two can be spliced in the channel dimension.

[0100] Step 42: extract the gait spatiotemporal features from the fusion contour map using the gait spatiotemporal extraction network.

[0101] Specifically, in an embodiment, the gait spatiotemporal extraction network is a Gait Lateral Network (GLN), and using the gait spatiotemporal extraction network can obtain features in the gait temporal dimension and spatial dimension from the fusion contour map.

[0102] Unlike the prior art, the above-mentioned method can obtain gait spatiotemporal features, and the gait spatiotemporal features extracted by the above-mentioned method can effectively counter the problem of low gait recognition accuracy caused by clothes, carrying objects, etc.

[0103] The method for obtaining target key point features will be described in detail below.

[0104] Referring to Figure 5 , Figure 5 is a flowchart of the fifth embodiment of the body shape transformation-based gait recognition method adopted by the present application, and the specific steps are as follows:

[0105] Step 51: average-pool the target key point features in the horizontal direction using at least two pooling kernels with the same width but different lengths.

[0106] Specifically, the size of the pooling kernel used for pooling the target key point features is the same width but different length, for example, one pooling kernel size is 3*5, and the other pooling kernel size is 5*5.

[0107] Step 52: linearly map the output results of each of the pooling kernels using a fully connected layer, wherein the fully connected layers corresponding to different pooling layers are independent of each other.

[0108] Specifically, a fully connected layer is connected after each pooling layer for linearly mapping the output results of the pooling layer, and the fully connected layers are independent of each other.

[0109] Step 53: sequentially arrange and splice the outputs of the fully connected layers according to the kernel size of the corresponding pooling kernels.

[0110] Different from the prior art, the target key point feature can be obtained by the above manner, and the target key point feature is the recombined key point feature.

[0111] Referring to Figure 6 , Figure 6 is a flowchart of a sixth embodiment of the body shape transformation-based gait recognition method provided in the present application, and the specific steps are as follows:

[0112] Step 61: obtaining an original image sequence formed by photographing a target in a walking process, wherein the original image sequence comprises a plurality of original images.

[0113] It can be understood that the video of photographing a human target in a walking process is segmented by frames, and a plurality of original images can be obtained, and each original image contains a human target.

[0114] Step 62: respectively performing target detection and target tracking on the original images.

[0115] Specifically, target detection refers to detecting and extracting a specified target from a video or image sequence.

[0116] It can be understood that the traditional target detection method includes optical flow method, background subtraction method, etc. At present, the popular target detection method is a deep learning motion detection method, which mainly uses a deep learning target detector for detection. From the structure, the deep learning target detector can be divided into a dual-pole detector (a candidate region-based detection method) and a single-level detector (an end-to-end detection method).

[0117] Step 63: selecting the original images in which the target leg is not occluded from the plurality of original images to form the gait image sequence.

[0118] Specifically, the target leg not being occluded refers to the human target not being occluded by other articles not attached to the human target.

[0119] Step 64: respectively obtaining a target gait contour map and a target key point matrix corresponding to the same target based on each gait image.

[0120] Specifically, the target gait contour map is obtained by inputting the gait image into a trained semantic segmentation model; and the target key point matrix is obtained by inputting the gait image into a human key point detection model.

[0121] Step 65: performing aspect ratio transformation on the target contour in each target gait contour map to obtain a target body shape transformation contour map.

[0122] The aspect ratio transformation on the target contour involves obtaining the target aspect ratio, please refer to Figure 3 , which will not be repeated here.

[0123] Step 66: splicing the corresponding target gait profile and target body shape transformation profile according to the channel dimension to form a fusion profile.

[0124] Step 67: extracting the gait spatiotemporal feature from the fusion profile by using the gait spatiotemporal extraction network.

[0125] Specifically, the gait spatiotemporal feature is extracted from the fusion profile by using the gait spatiotemporal extraction network. Please refer to Figure 4 , which will not be repeated here.

[0126] (Steps 66-67 and steps 68-610 are not distinguished in sequence)

[0127] Step 68: performing average pooling on the target key point feature in the horizontal direction by using at least two pooling kernels with the same width but different lengths.

[0128] Step 69: performing linear mapping on the output results of each of the pooling kernels by using a fully connected layer, wherein the fully connected layers corresponding to different pooling layers are independent of each other.

[0129] Step 610: sequentially arranging and splicing the outputs of the fully connected layers according to the kernel size of the corresponding pooling kernels.

[0130] Step 611: performing feature fusion on the gait spatiotemporal feature and the target key point feature to obtain a fusion measurement feature.

[0131] Different from the prior art, the gait recognition method based on body shape transformation provided in the present application can solve the problem of low gait recognition accuracy caused by clothing obstruction, carrying objects, etc., thereby improving the accuracy of target gait recognition.

[0132] Referring to Figure 7 , Figure 7 is a structural schematic diagram of an embodiment of the gait recognition device based on body shape transformation provided in the present application. The gait recognition device 70 comprises a memory 701 and a processor 702. The memory 701 is used to store program data, and the processor 702 is used to execute the program data to realize the gait recognition method based on body shape transformation described above. Here, no further description is given.

[0133] Referring to Figure 8 , Figure 8 is a structural schematic diagram of an embodiment of the computer readable storage medium provided in the present application. The computer readable storage medium 80 stores program data 801. When the program data 801 is executed by a processor, it is used to realize the gait recognition method based on body shape transformation as described above. Here, no further description is given.

[0134] The processor in the embodiments of the present application can be referred to as a CPU (Central Processing Unit), an integrated circuit chip, a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application-Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0135] The computer readable storage medium in the embodiments of the present application can be a U disk, a mobile hard disk, a ROM (Read-Only Memory), a RAM (Random Access Memory), a magnetic disk or an optical disk, etc. which can store program instructions, or can also be a server which stores the program instructions and sends the stored program instructions to other devices for running, or can also run the stored program instructions by itself.

[0136] The above description is only an embodiment of the present application, and does not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation using the content of the specification and drawings of the present application, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A gait recognition method based on body shape transformation, characterized in that, The method includes: Obtain a gait image sequence, wherein the gait image sequence includes a plurality of gait images; Based on each of the gait images, a target gait contour map and a target key point matrix corresponding to the same target are obtained respectively; The original aspect ratio of the target contour of each target gait contour map is input into the body shape transformation function to obtain the corresponding target aspect ratio; or, the expected value and standard deviation of the set of original aspect ratios of the target contours of the target gait contours are used to obtain the corresponding target aspect ratio; the target aspect ratio is different from the original aspect ratio. The target contour is transformed based on the target aspect ratio to obtain a target body shape transformation contour map; wherein, the aspect ratio transformation of the target contour includes stretching and deforming the target contour; Based on the target gait contour map and the target body shape transformation contour map, gait spatiotemporal features are obtained using a gait spatiotemporal extraction network; Based on the target key point matrix, the target key point features are obtained using a key point feature extraction network. The gait spatiotemporal features and the target key point features are fused to obtain the fusion metric features.

2. The method according to claim 1, characterized in that, The acquisition of the gait image sequence includes: Acquire a sequence of raw images formed by photographing the target during its walking process, wherein the sequence of raw images includes several raw images; Target detection and target tracking are performed on the original image respectively; The gait image sequence is formed by selecting the original images from the plurality of original images in which the target leg is not obscured.

3. The method according to claim 1, characterized in that, The step of inputting the original aspect ratio of the target contour of each target gait contour map into the body shape transformation function includes: The target aspect ratio is calculated using the body shape transformation function, which is one of the maximum, minimum, mode, mean, or median of the original aspect ratios.

4. The method according to claim 1, characterized in that, The expectation and standard deviation of the original aspect ratio set of the target contour based on the target gait contour include: Obtain the expected value and standard deviation of the original set of aspect ratios that are normally distributed; The target aspect ratio is selected from the range of values ​​formed by the expected value and the standard deviation.

5. The method according to claim 1, characterized in that, The aspect ratios of the targets corresponding to the gait profiles of each target may be the same or different.

6. The method according to claim 1, characterized in that, The step of obtaining gait spatiotemporal features using a gait spatiotemporal extraction network based on the target gait contour map and the target body shape transformation contour map includes: The corresponding target gait contour map and target body shape transformation contour map are stitched together according to the channel dimension to form a fused contour map; The gait spatiotemporal features are extracted from the fused contour map using the gait spatiotemporal extraction network.

7. The method according to claim 1, characterized in that, The step of obtaining target key point features using a key point feature extraction network based on the target key point matrix includes: The target key point features are averaged in the horizontal direction using at least two pooling kernels with the same width but different lengths. A fully connected layer is used to linearly map the output of each pooling kernel, wherein the fully connected layers corresponding to different pooling layers are independent of each other, and each pooling layer includes the pooling kernel; The outputs of the fully connected layer are arranged and spliced ​​according to the corresponding core size of the pooling core.

8. A gait recognition device based on body shape transformation, characterized in that, The device includes a memory and a processor, the memory storing program data, and the processor executing the program data to implement the gait recognition method as described in any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program data, which, when executed by a processor, is used to perform the method as described in any one of claims 1-7.

Images