Training method of model for gait feature extraction across view angles

Abstract

The application provides a training method of a model for gait feature extraction across view angles, comprising: obtaining training samples; the training samples comprising video data and annotation information, the video data comprising video segments obtained after a plurality of cameras arranged in a semicircle shoot in a collection area, and filtering and video segment division based on preset conditions; the semicircular area formed by the plurality of cameras comprising two quarter circles, and the cameras in the two quarter circles being uniformly and interval distributed according to different first angles and second angles; the annotation information comprising position information, posture information and binary silhouette of the collected samples in frame images, and the position of the frame images in a gait cycle; training a target silhouette deep learning model and a gait feature extraction model based on the training samples to obtain trained models; the target silhouette deep learning model being used to determine binary silhouettes of input images and gait cycle flag values, and the gait feature extraction model being used to extract gait features.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of gait recognition, and more particularly to a training and application method of a model for cross-view gait feature extraction. BACKGROUND

[0002] On the one hand, in the case where a pedestrian wears a protective tool such as a mask, a biological recognition method such as face recognition is difficult to guarantee the recognition accuracy; on the other hand, in a specific scene with high public safety requirements, the tracking accuracy of pedestrians also has higher requirements. Therefore, the development of pedestrian recognition technology based on gait features (i.e., gait recognition technology) is urgent.

[0003] Gait recognition technology refers to a biometric technology for recognizing the identity of a person by the way the person walks, has the advantages of non-contact long distance and not easy to disguise, and is widely used in intelligent video monitoring. The uncontrolled intelligent video monitoring system built by relying on gait recognition technology can calibrate and recognize pedestrians in the current monitoring range, recognize abnormal behavior, and reduce the risk of public safety.

[0004] In a scene involving multiple cameras monitoring from different angles, even for the same pedestrian, the gait features extracted will be different due to different monitoring angles, thereby affecting the recognition accuracy and recognition effect. With the help of artificial intelligence, cross-view gait feature extraction can be achieved. However, the training data sets of the gait feature extraction related models, such as public data sets, CASIA gait data sets, OU-ISIR data sets, GREW data sets, etc., have defects such as low coverage of viewing angles, single background, and difficult to get good silhouette effect, and the quality of the training data sets is not high enough, which will directly affect the performance of the model. SUMMARY

[0005] In view of the defects in the related art, the embodiments of the present application provide a training method of a model for cross-view gait feature extraction, aiming to solve the problem of insufficient quality of the training data set of the gait feature extraction related model.

[0006] In a first aspect, the embodiments of the present application provide a training method of a model for cross-view gait feature extraction, comprising:

[0007] Obtaining training samples; wherein the training samples include video data and annotation information, the video data includes video segments of each collection sample in different dressings, different postures and different walking routes, which are obtained after a plurality of cameras arranged in a semicircle shoot in a collection area, are filtered and video segment divided based on a preset condition; the semicircular area formed by the plurality of cameras includes two quarter circles, the cameras in one quarter circle are uniformly distributed at a first angle, the cameras in the other quarter circle are uniformly distributed at a second angle, and the first angle and the second angle are different; the annotation information includes position information, posture information and binary silhouette of the collection sample in the frame image, and a position of the frame image in a gait cycle;

[0008] Training the target silhouette deep learning model and the gait feature extraction model based on the training samples to obtain the trained target silhouette deep learning model and the gait feature extraction model; wherein the target silhouette deep learning model is used to determine the binary silhouette of the collection sample in the input image and the gait cycle flag value of the silhouette image, the gait cycle flag value is used to represent the position of the silhouette image in the gait cycle, and the gait feature extraction model is used to extract the gait feature in the silhouette image.

[0009] In view of the defects of the related art, the embodiments of the present application also provide a cross-view gait feature extraction method, aiming to solve the problem of insufficient gait feature extraction accuracy.

[0010] In a second aspect, the embodiments of the present application also provide a cross-view gait feature extraction method, comprising:

[0011] Preprocessing the video shot by the cross-view camera to obtain an initial image sequence, and the images in the initial image sequence are minimum rectangular images containing the target;

[0012] Inputting the initial image sequence into the target silhouette deep learning model to generate an initial silhouette sequence of the target and a gait cycle flag value of the silhouette image in the initial silhouette sequence, the gait cycle flag value is used to represent the position of the silhouette image in the gait cycle;

[0013] Based on the gait cycle flag value of the silhouette image in the initial silhouette sequence, obtaining a complete gait cycle silhouette sequence of the target;

[0014] Inputting the gait cycle silhouette sequence into the gait feature extraction model to generate an n-dimensional feature vector, the n-dimensional feature vector is used to represent the gait feature corresponding to the initial image sequence;

[0015] The target silhouette deep learning model and the gait feature extraction model are obtained by the training method described in the first aspect or any one of the possible ways of the first aspect.

[0016] In a third aspect, the embodiments of the present application further provide a training device for a model for gait feature extraction across view angles, comprising:

[0017] an acquisition module configured to acquire training samples, wherein the training samples comprise video data and annotation information, the video data comprises video segments of each collection sample in different dressings, different postures and different walking routes, which are acquired based on preset conditions after being filtered and video segment divided after being shot in a collection area by a plurality of cameras arranged in a semicircle; the semicircle area formed by the plurality of cameras comprises two quarter circles, the cameras in one quarter circle are uniformly distributed at a first angle, the cameras in the other quarter circle are uniformly distributed at a second angle, and the first angle and the second angle are different; and the annotation information comprises position information, posture information and binary silhouette of the collection sample in a frame image, and a position of the frame image in a gait cycle;

[0018] a training module configured to train a target silhouette deep learning model and a gait feature extraction model based on the training samples, and acquire the trained target silhouette deep learning model and the gait feature extraction model; wherein the target silhouette deep learning model is configured to determine a binary silhouette of a collection sample in an input image and a gait cycle flag value of a silhouette image, the gait cycle flag value is configured to represent a position of the silhouette image in a gait cycle, and the gait feature extraction model is configured to extract gait features in the silhouette image.

[0019] In a fourth aspect, the embodiments of the present application further provide a device for gait feature extraction across view angles, comprising:

[0020] a first acquisition module configured to pre-process a video shot by a camera across view angles, and acquire an initial image sequence, an image in the initial image sequence being a minimum rectangular image containing a target;

[0021] a first generation module configured to input the initial image sequence to a target silhouette deep learning model, and generate an initial silhouette sequence of the target and a gait cycle flag value of a silhouette image in the initial silhouette sequence, the gait cycle flag value being configured to represent a position of the silhouette image in a gait cycle;

[0022] a second acquisition module configured to acquire a complete gait cycle silhouette sequence of the target based on the gait cycle flag value of the silhouette image in the initial silhouette sequence;

[0023] a second generation module configured to input the gait cycle silhouette sequence to a gait feature extraction model, and generate an n-dimensional feature vector, the n-dimensional feature vector being configured to represent a gait feature corresponding to the initial image sequence;

[0024] The target silhouette deep learning model and the gait feature extraction model are obtained by the training method described in the first aspect or any one of the possible manners of the first aspect.

[0025] In a fifth aspect, an electronic device is provided, and the electronic device includes at least one memory configured to store programming, and at least one processor configured to execute the programming stored in the memory, wherein the processor is configured to perform the method described in the first aspect or any possible implementation of the first aspect, or the processor is configured to perform the method described in the second aspect or any possible implementation of the second aspect.

[0026] In a sixth aspect, a computer-readable storage medium is provided, and the computer-readable storage medium stores a computer program, and when the computer program is run on a processor, the processor is caused to perform the method described in the first aspect or any possible implementation of the first aspect, or the processor is caused to perform the method described in the second aspect or any possible implementation of the second aspect.

[0027] In a seventh aspect, a computer program product is provided, and when the computer program product is run on a processor, the processor is caused to perform the method described in the first aspect or any possible implementation of the first aspect, or the processor is caused to perform the method described in the second aspect or any possible implementation of the second aspect.

[0028] The training method of the model for cross-view gait feature extraction provided in the embodiments of the present application changes the arrangement of the angles of the cameras in the sample collection process from the actual application, covers various scenes of normal walking in daily life, and increases the labeling of the gait cycle flag in the data labeling process, so that the dimensions are more comprehensive and more in line with the demand of gait feature extraction in the application scene, so that the subsequent gait feature extraction result is more accurate. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the present application or related art, the following will briefly introduce the drawings needed to be used in the embodiments or related art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0030] Figure 1 is a flowchart of the training method of the model for cross-view gait feature extraction provided in the embodiments of the present application;

[0031] Figure 2 is a schematic diagram of the erection of multiple cameras across views provided in the embodiments of the present application;

[0032] Figure 3 is a structural schematic diagram of the target silhouette deep learning model provided in the embodiments of the present application;

[0033] Figure 4 is one of the flowcharts of the gait feature extraction method across views provided by the embodiments of the present application;

[0034] Figure 5 is another flowchart of the gait feature extraction method across views provided by the embodiments of the present application;

[0035] Figure 6 is a structural schematic diagram of the training device for the model for gait feature extraction across views provided by the embodiments of the present application;

[0036] Figure 7 is a structural schematic diagram of the gait feature extraction device provided by the embodiments of the present application;

[0037] Figure 8 is a structural schematic diagram of the electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0038] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0039] Figure 1 is a flowchart of the training method for the model for gait feature extraction across views provided by the embodiments of the present application, as shown in Figure 1 The training method at least includes the following steps (Step):

[0040] S101, obtaining training samples.

[0041] The training samples include video data and annotation information. The video data includes video segments of each collection sample in different dressings, different postures and different walking routes, which are obtained after the video segments are screened and cut based on preset conditions after the collection sample is shot in a collection area by a plurality of cameras arranged in a semicircle. The semicircle area formed by the plurality of cameras includes two quarter circles. The cameras in one quarter circle are uniformly distributed at a first angle, and the cameras in the other quarter circle are uniformly distributed at a second angle. The first angle and the second angle are different. The annotation information includes position information, posture information and binary silhouette of the collection sample in the frame image, and the position of the frame image in the gait cycle.

[0042] S102, training the target silhouette deep learning model and the gait feature extraction model based on the training samples, to obtain the trained target silhouette deep learning model and the gait feature extraction model.

[0043] The target silhouette deep learning model is used to determine a binary silhouette of the collected sample in the input image and a gait cycle flag value of the silhouette image, and the gait cycle flag value is used to represent the position of the silhouette image in the gait cycle. The gait feature extraction model is used to extract the gait feature in the silhouette image.

[0044] Specifically, the target silhouette deep learning model can be used to determine a binary silhouette of the collected sample in the input image and a gait cycle flag value of the silhouette image, and the gait cycle flag value is used to represent the position of the silhouette image in the gait cycle. The gait feature extraction model can be used to extract the gait feature in the silhouette image. The performance of the model depends largely on the quality of the training data set, so the quality of the training sample will directly affect the effect of the final gait feature extraction. The following takes a pedestrian as an example to illustrate the process of obtaining the training sample.

[0045] The plurality of cameras are arranged at different angles and in a semicircular form. Generally, the plurality of cameras are uniformly arranged in the semicircular region, but in order to obtain as much video data as possible under different visual conditions and facilitate the labeling of the video data, the semicircular region is divided into two quarter circles, the cameras in one quarter circle are uniformly distributed at a first angle, and the cameras in the other quarter circle are uniformly distributed at a second angle, the first angle and the second angle being different.

[0046] In order to facilitate the collection and labeling of data during the training process, the collected video data is filtered and video segments are divided based on preset conditions. The preset conditions include: a preset walking route, clothing, posture, etc.

[0047] Figure 2 is a schematic diagram of the arrangement of the plurality of cameras across the visual angle provided by the embodiments of the present application, as shown in Figure 2 There are a total of 12 cameras arranged in a semicircular form, of which 6 cameras are used in the left quarter circle, and the included angle between each camera and the center is 18°; 7 cameras are used in the right quarter circle (the cameras at the 90° position are shared), and the included angle between each camera and the center is 15°.

[0048] In the collected region, the collected video data is filtered based on preset conditions, including: Figure 2 Four walking routes are shown in the figure, and the video data collected on the four walking routes is filtered, and the personnel, clothing, and posture of the collected sample are distinguished. Optionally, the clothing can be distinguished by coat, trousers, shoes, and hat, etc.; optionally, the posture includes normal walking, fast walking, umbrella, backpack, crossbody bag, handbag, mobile phone, and phone call, etc. The posture, clothing, etc. can be set according to actual conditions.

[0049] Further, the collected video data is split. The collected video data is split into data segments to obtain video segments of each collection sample in different dressings, different postures and different walking routes. Specifically, the video data is split according to the collection sample, dressing, posture, walking direction and the like to obtain complete video segments of different collection samples walking in different dressings, different postures and different directions. For example, a total of 12 video angles can be collected, and according to the included angle of the collection angle and the walking direction, the angles collected within a quadrant, i.e. 90°, are 0°, 15°, 18°, 30°, 36°, 45°, 54°, 60°, 72°, 75° and 90°. In combination with a preset personnel walking route, 40 angles within a 360° range of the same collection sample can be collected. In combination with 8 directions for each round trip, n dressings (assuming n is 2) and 8 postures, 12 (number of cameras) x 8 (directions) x 2 (dressings) x 8 (postures) = 1536 video segments of each collection sample can be obtained. Figure 2

[0050] Further, after the splitting of the video segments is completed, the frame images in each video segment are labeled, and the labeling information at least includes the position information, posture information and binary silhouette of the collection sample, and the position of the frame image in the gait cycle. Optionally, all the frame images are adjusted to a uniform size before labeling.

[0051] Further, the labeled video data is used as a training sample to train the target silhouette deep learning model and the gait feature extraction model.

[0052] The training method of the model for gait feature extraction across different angles provided by the embodiments of the present application changes the angle arrangement of the camera in the sample collection process from the actual application, covers various scenes of normal walking in daily life, and increases the labeling of the gait cycle flag in the data labeling process, considers more comprehensive dimensions, and is more in line with the demand for gait feature extraction in the application scene, so that the subsequent gait feature extraction result is more accurate.

[0053] In some embodiments, the position of the frame image in the gait cycle is labeled by the following method:

[0054] Based on the time sequence order of the frame image in the video segment and the position of the head and limbs of the collection sample in the frame image, the position of the frame image in the gait cycle is determined.

[0055] ​Specifically, during walking, running and the like, a person will periodically use four limbs to complete regular actions, for example, when the left foot moves forward, the right hand moves forward, the left hand slightly moves backward, and then the right foot moves forward, the left hand moves forward, the right hand slightly moves backward, and then the left foot moves forward again to repeat the previous action. Generally, the above process is habitually regarded as a complete gait cycle. Therefore, the time sequence order of the frame images in the video segment and the positions of the head and limbs of the sample collected in the frame images can be used to determine the position of the frame image in the gait cycle.

[0056] In some embodiments, determining the position of the frame image in the gait cycle specifically includes:

[0057] In the time-sequentially continuous frame images, the frame image between the right heel touch-down and the left toe lift-off of the sample collected is marked as the first position of the complete gait cycle;

[0058] The frame image between the left toe lift-off and the left heel touch-down of the sample collected is marked as the second position;

[0059] The frame image between the left heel touch-down and the right toe lift-off of the sample collected is marked as the third position;

[0060] The frame image between the right toe lift-off and the right heel touch-down of the sample collected is marked as the fourth position, completing the marking of a complete gait cycle.

[0061] Specifically, in a certain video segment, a plurality of complete gait cycles of the sample collected under specific dressing, specific posture and specific visual angle are included, in the time-sequentially continuous frame images, the frame image between the right heel touch-down and the left toe lift-off of the sample collected is marked as the first position (for example, marked as 0) of the complete gait cycle; the frame image between the left toe lift-off and the left heel touch-down of the sample collected is marked as the second position (for example, marked as 1); the frame image between the left heel touch-down and the right toe lift-off of the sample collected is marked as the third position (for example, marked as 0); and the frame image between the right toe lift-off and the right heel touch-down of the sample collected is marked as the fourth position (for example, marked as 1), then the frame images continuously changing from 0 to 1, from 1 to 0 and from 0 to 1 constitute a complete gait cycle.

[0062] It is conceivable that the marking of the first position can be adjusted, for example, the frame image between the left toe lift-off and the left heel touch-down is marked as the first position, the frame image between the left heel touch-down and the right toe lift-off is marked as the second position, and so on.

[0063] The training method of the model for gait feature extraction across visual angles provided by the embodiments of the present application marks the position of the frame image in the gait cycle during the training process of the model, which is more in line with the requirements of gait feature extraction in actual application scenarios.

[0064] In some embodiments, the position information of the sample is collected by the following method:

[0065] The frame image is input into a position detection deep learning model to obtain the position information of the sample, the position information including a minimum circumscribed rectangle frame of the sample profile and vertex coordinate information of the minimum circumscribed rectangle frame, the vertex coordinate information being used to determine the position of the minimum circumscribed rectangle frame.

[0066] The position detection deep learning model is obtained based on a sample image and pre-determined position labeling information after training, and the position detection deep learning model uses a yolov5 model.

[0067] Specifically, for labeling of the position information, the label can be designed as a detection frame independent of each sample, the detection frame being a minimum circumscribed rectangle frame containing the sample; the position of the sample is labeled by vertex coordinate information of the rectangle frame. Specifically, the position of the sample in the frame image is labeled by coordinates of any two opposite vertices, or coordinates of any three vertices, or coordinates of four vertices.

[0068] The position information of the sample is obtained by a predetermined algorithm, for example, by a position detection deep learning model, the position detection deep learning model being trained using an original image and labeled pedestrian position information, the model input being the original image and the model output being the pedestrian position information in the image. Optionally, the model uses a general yolov5 model.

[0069] In some embodiments, the binarized silhouette is labeled by the following method:

[0070] The edge profile of the sample in the frame image is labeled by a polygon;

[0071] Based on the edge profile of the sample, it is determined whether a pixel point in the frame image is a foreground pixel point or a background pixel point;

[0072] The foreground pixel point is set to 255 and the background pixel point is set to 0 to obtain a binarized silhouette image with black background and white sample silhouette.

[0073] Specifically, for labeling of the binarized silhouette, the edge profile of each sample is labeled by a polygon, and the labeling form is a closed area of a polygon.

[0074] By using the edge contour of the collected sample, it can be distinguished whether the pixel point in the frame image is a foreground pixel point or a background pixel point. For example, by using the python code, the saved json file can be loaded and parsed to obtain the contour coordinates, and by using the pointPolygonTest method of opencv, it can be distinguished whether the pixel point outside the contour (background pixel point) or the pixel point inside the contour (foreground pixel point) in the frame image.

[0075] Different color values are attached to the background pixel points and foreground pixel points in the image, for example, the background pixel points are set to 0 and the foreground pixel points are set to 255, and the saved picture is obtained, that is, a binary silhouette image with black background and white collected sample silhouette.

[0076] In some embodiments, for the annotation of the posture information: the label can be designed as a serial number corresponding to different postures, and different postures can be distinguished by the serial number.

[0077] The target silhouette deep learning model and the gait feature extraction model in the embodiments of the present application are introduced as follows.

[0078] In some embodiments, the target silhouette deep learning model adopts a segmentation convolutional network, which includes an encoder network, a decoder network, a coding-decoding connection module and a pixel-level classification layer. The encoder network includes a convolutional layer and a maximum pooling layer, and the decoder network includes a convolutional layer and an up-sampling layer. The network structures of the encoder network and the decoder are symmetrical. The coding-decoding connection module adopts a skip connection, and each convolutional layer before a maximum pooling layer in the encoder network is connected to the last convolutional layer of the corresponding size in the decoder network. The encoder network records boundary information through a pooling index, and the decoder network realizes nonlinear up-sampling through the pooling index derived from the corresponding maximum pooling layer.

[0079] Specifically, the target silhouette deep learning model adopts a segmentation convolutional neural network to improve the performance of image segmentation, and uses a set index in the segmentation convolutional network to further improve the accuracy of the segmentation boundary.

[0080] The segmentation convolutional network includes an encoder module, a decoder network, a coding-decoding connection module and a pixel-level classification layer. The encoder module includes a convolutional layer and a maximum pooling layer, and the decoder module includes a convolutional layer and an up-sampling layer. The encoder module and the decoder module adopt a symmetrical network structure, that is, each layer in the encoder module has a corresponding layer in the decoder.

[0081] The encoder module records boundary information through a pooling index. During the encoding process, the size of the feature map changes after the max-pooling operation of the pooling layer, and the low-resolution features generated by the down-sampling operation lose some information; in order to retain more details, the segmentation convolutional network uses the pooling index to record the boundary information, that is, the pixel position of the maximum value selected by the pooling layer in the max-pooling operation. During the decoding process, the up-sampling operation is the inverse process of the pooling operation, the feature map is first enlarged, and then the data of the feature map is put into the corresponding position according to the pooling index.

[0082] The decoder module realizes the non-linear up-sampling through the pooling index derived from the corresponding max-pooling layer, so that the up-sampling layer does not need to be learned, and the training parameters can be reduced. Due to the introduction of the pooling index, more boundary information can be obtained, and the segmentation performance of the model can be improved. The role of the convolution operation is to enrich the information of the sparse feature map generated by the up-sampling layer, and to generate a more continuous feature map.

[0083] In some embodiments, the encoding-decoding connection module adopts a jump connection, and the convolution layer before each max-pooling layer in the encoder module is connected to the last convolution layer of the corresponding size in the decoder module.

[0084] Specifically, the segmentation convolutional network also implements a jump layer fusion mechanism, that is, different scale feature maps generated by different layers are fused into a fusion map. After each down-sampling or up-sampling operation, the scale of the feature map changes, so the feature map in the segmentation convolutional network has multiple scales.

[0085] In order to utilize the sparse feature maps and continuous feature maps of different scales, a cross-layer fusion mechanism is established to connect the encoder network and the decoder network. The encoding-decoding connection module adopts a jump connection, and the convolution layer before each max-pooling layer in the encoder module is connected to the last convolution layer of the corresponding size in the decoder network.

[0086] Figure 3 is a structural schematic diagram of a target silhouette deep learning model provided by an embodiment of the present application, as Figure 3 The network structure of the encoder module is consistent with VGG16, and contains 13 convolution layers and 5 pooling layers; the network structure of the decoder module is symmetrical to the network structure of the encoder module, and contains 13 convolution layers and 5 up-sampling layers, and each layer of the encoder module has a corresponding layer in the decoder module. The decoder can realize the non-linear up-sampling operation through the pooling index derived from the corresponding max-pooling layer, so that the up-sampling layer does not need to be learned, and the training parameters can be reduced.

[0087] Meanwhile, more boundary information can be obtained due to the introduction of the pooling index, and the performance of the segmentation is improved. In the encoding process, the size of the feature map will change after the max-pooling operation, and some information will be lost in the low-resolution feature generated by the down-sampling operation; in order to preserve more details, the segmentation convolutional network uses the pooling index to record the boundary information, that is, the pixel position of the maximum value selected by the pooling layer in the max-pooling operation. In the decoding process, the up-sampling operation is the inverse process of the pooling operation, first enlarging the feature map, and then putting the data of the feature map into the corresponding position according to the pooling index. The role of the convolution operation is to enrich the information of the sparse feature map generated by the up-sampling layer, and to generate a more continuous feature map.

[0088] Meanwhile, a skip layer fusion mechanism is implemented on the segmentation convolutional network, that is, different scale feature maps generated by different levels are fused into a fusion map. Each time the down-sampling or up-sampling operation is performed, the scale of the feature map will change, so there are five scales of the feature map in the segmentation convolutional network. In order to utilize the sparse feature maps and continuous feature maps of different scales, a cross-layer fusion mechanism is established to connect the encoder network and the decoder network, and the convolutional layer before each max-pooling layer in the encoder network is connected to the last convolutional layer of the corresponding scale in the decoder network.

[0089] In some embodiments, the gait feature extraction module adopts a GAN network, the GAN network includes k sets of generative adversarial networks, k is the number of perspectives of the input image, and the generator in the k sets of generative adversarial networks generates gait maps in k perspectives and outputs after fusion.

[0090] Specifically, the GAN network is used as the gait feature extractor, and the GAN network includes a generator and a discriminator. The generator G is composed of an encoder and a decoder, each of which has seven convolutional layers, and a feature vector is extracted from the last layer of the encoder in the generator. The output of the generator G is a vector with a dimension of 512x1x1. Since this vector can be decoded into an output image, the vector itself can effectively express the features of the input. The core idea of the GAN network for extracting gait features is to use the generator features of predicting the next frame of the generated image as the gait feature encoding, so as to better model the time-related gait features.

[0091] In order to fuse the gait features under multiple perspectives to improve the recognition rate, k sets of generative adversarial networks are used, and k is the number of perspectives of the input image. The encoder in the k sets of generative adversarial networks receives the gait image as input at the same time to obtain feature information and obtain a hidden feature vector. Then, the generator generates a gait image by taking the hidden feature vector as input. In order to ensure that the gait image retains correct feature information, a discriminator is set to perform supervised learning. In the test stage, the k sets of generative adversarial networks can obtain generated gait images under k angles, fuse the feature images under the k angles, and obtain the final recognition result based on the fused gait feature image.

[0092] The training method of the model for gait feature extraction across perspectives provided by the embodiments of the present application, the gait feature extraction model adopts a GAN network, the GAN network includes k sets of generative adversarial networks, and k is the number of perspectives of the input image, so that the model supports the fusion of gait sequence features across perspectives, improves the dimension uniformity of gait feature extraction in the cross-perspective scene, and improves the accuracy of subsequent pedestrian identity recognition and the like in the cross-perspective scene.

[0093] Figure 4 is one of the flowcharts of the gait feature extraction method across perspectives provided by the embodiments of the present application, as shown in Figure 4 The method at least includes the following steps:

[0094] S401, pre-processing the video captured by the cameras across perspectives to obtain an initial image sequence, and the images in the initial image sequence are minimum rectangular images containing the target.

[0095] Specifically, a plurality of cameras are usually fixedly installed in a certain space range, the plurality of cameras capture from different perspectives, and real-time videos captured by the cameras across perspectives are obtained and pre-processed to obtain an initial image sequence of the target.

[0096] The installation position of the camera is usually fixed, and the collectable range is also determined, a camera can obtain a complete video in a specific area within a continuous time period, and multiple cameras can obtain multiple videos captured at multiple angles.

[0097] Optionally, the pre-processing of the video captured by the cameras across perspectives includes: real-time frame extraction of the video (stream), and target position detection and target tracking are only performed on the extracted frame images. Considering that the gait feature of the target does not change significantly in a very short duration, the frame extraction processing can reduce the overall operation amount and operation complexity of the system while not affecting the final recognition effect.

[0098] Optionally, the video captured by the cameras across the view angles is preprocessed, including: after determining the frame image containing the target, the frame image is cropped to obtain a minimum rectangular image containing the target contour, and the initial image sequence corresponding to the target is obtained by arranging the minimum rectangular images in time sequence.

[0099] Optionally, the video captured by the cameras across the view angles is preprocessed, including: the positions of different targets in the frame images of the video are detected by a target position detection algorithm (for example, the aforementioned position detection deep learning model), and each detected target is tracked by a target tracking algorithm (for example, the general DeepSort algorithm), that is, the positions of the same target in different frame images are located. The target position detection algorithm and the target tracking algorithm in the embodiments of the present application are not limited.

[0100] S402, inputting the initial image sequence into the target silhouette deep learning model to generate an initial silhouette sequence of the target and a gait cycle flag value of a silhouette image in the initial silhouette sequence, the gait cycle flag value being used to represent the position of the silhouette image in the gait cycle.

[0101] Specifically, when it is detected that the target leaves the field of view of the camera or the number of images (the number of images of real-time frame extraction) accumulated in the initial image sequence reaches a certain threshold, the sequence images are sequentially input into the pre-trained target silhouette deep learning model to generate an initial silhouette sequence of the target and a gait cycle flag value of a silhouette image in the initial silhouette sequence, the gait cycle flag value being used to represent the position of the silhouette image in the gait cycle.

[0102] S403, obtaining a complete gait cycle silhouette sequence of the target based on the gait cycle flag value of the silhouette image in the initial silhouette sequence and the positions of the head and limbs of the target in the silhouette image.

[0103] Specifically, the complete gait cycle silhouette sequence of the target is obtained by using the positions of the head and limbs of the target in the initial silhouette sequence. During walking, running and the like, the target (for example, a pedestrian) will periodically use the four limbs to complete regular actions, for example, the left foot moves forward while the right hand moves forward, the left hand slightly moves backward, then the right foot moves forward while the left hand moves forward and the right hand slightly moves backward, and the next left foot moves forward to repeat the previous action; generally, the above process is habitually regarded as a complete gait cycle. Therefore, the positions of the frame images in the gait cycle can be determined by using the time sequence order of the frame images in the video segment and the positions of the head and limbs of the target in the frame images, and the complete gait cycle silhouette sequence of the target is obtained.

[0104] S404, inputting the complete gait cycle silhouette sequence into the gait feature extraction model to generate an n-dimensional feature vector, the n-dimensional feature vector being used to represent the gait feature corresponding to the initial image sequence.

[0105] Specifically, the gait cycle silhouette sequence is input into the gait feature extraction model to obtain an n-dimensional feature vector for representing the gait feature corresponding to the initial image sequence. Optionally, n is 1024, that is, a 1024-dimensional feature vector is obtained. Optionally, the 1024-dimensional feature vector is normalized to obtain a 1024-dimensional unit vector for representing the gait feature corresponding to the initial image sequence.

[0106] The cross-view gait feature extraction method provided by the embodiments of the present application can automatically extract gait features of pedestrians under multi-view camera monitoring, so as to facilitate subsequent pedestrian identity recognition and the like, and support accurate extraction of cross-view gait features based on massive historical videos and real-time video streams.

[0107] In some embodiments, the target complete gait cycle silhouette sequence is obtained, including:

[0108] Based on the gait cycle flag value, an initial gait cycle silhouette sequence containing the complete gait cycle of the target is selected from the initial silhouette sequence;

[0109] Based on the positions of the head and limbs of the target in the image, a mutation image with a position mutation in each target sequence is determined, and the mutation image is corrected; the target sequence refers to a silhouette sequence with the same gait cycle flag value in the initial gait cycle silhouette sequence;

[0110] The corresponding relationship between the silhouette images in each target sequence and the silhouette images in the target sequence with the largest number of images is determined;

[0111] The proportion of the number of silhouette images in each target sequence is taken as the weight of the silhouette image, the silhouette images in the target sequence with the largest number of images and the corresponding silhouette images in other target sequences are weighted and averaged to determine a new image sequence with the same length as the target sequence with the largest number of images;

[0112] The new image sequence is binarized to obtain a gait cycle silhouette sequence.

[0113] Specifically, the initial silhouette sequence is directly taken as the input of the gait feature extraction model, although accurate semantic information can be obtained, multiple gait cycles are transmitted in the time sequence information, and the starting position is uncertain, which may obtain redundant time sequence information, and the length of the input silhouette sequence also affects the completeness of the obtained information. Therefore, sequence cutting is considered from the initial silhouette sequence to obtain a target complete gait cycle silhouette sequence, so as to ensure the consistency of the sequence input into the gait feature extraction model.

[0114] In the initial silhouette sequence, the first bit to the fourth bit of the gait cycle flag value continuously corresponds to a complete gait cycle of the target. Based on the gait cycle flag value, an initial gait cycle silhouette sequence containing one or more complete gait cycles can be cut from the initial silhouette sequence.

[0115] Optionally, an initial gait cycle silhouette sequence containing a complete gait cycle of the target can be directly cut from the initial silhouette sequence for subsequent gait feature extraction.

[0116] Optionally, the initial gait cycle silhouette sequence is processed and then used for subsequent gait feature extraction. The initial gait cycle silhouette sequence is processed as follows:

[0117] The initial gait cycle silhouette sequence can be split into multiple time-sequentially continuous target sequences according to the gait cycle flag value. The gait cycle flag value of the silhouette images in each target sequence is the same, that is, one complete gait cycle silhouette sequence can be split into four target sequences.

[0118] Optionally, based on the positions of the head and limbs of the target in the image, a mutation image with position mutation in each target sequence is determined and the mutation image is corrected. It is conceivable that the mutation correction can be performed on the silhouette sequence first, and then the gait cycle silhouette sequence is cut; or the gait cycle silhouette sequence can be cut first, and then the mutation correction is performed.

[0119] Since the original images are all minimum rectangular images containing complete human bodies, the contour position of the person in the image will occupy the entire image. For example, the highest point of the head of the person is the white position closest to the upper edge in the binary image, the lowest point of the foot is the white position closest to the lower edge in the binary image, and similarly, the white positions closest to the left and right edges are the limbs. According to the structure of the human body, a threshold (for example, 0.3) is set, considering that below 0.3 of the image height is the lower limbs and above 0.3 is the upper limbs, so the positions of the head and limbs of the person in the image can be obtained. In a complete gait cycle silhouette sequence, in the sequence images with the same gait cycle flag value, according to the time sequence relationship, the positions of the head and limbs should not change. For three time-sequentially continuous silhouette images, the position of a point in the first image is A point, the position of the point in the third image is B point, and the position of the point in the second image is not between A point and B point, so the second image is considered to be a mutation image and needs to be corrected. Optionally, the correction method can be to smooth the second image to the average value of A point and B point.

[0120] Optionally, the silhouette images in the target sequence with the largest number of images and the corresponding silhouette images in other target sequences are subjected to weighted average processing to determine a new sequence image with the same length as the target sequence with the largest number of images. In the weighted processing, the proportion of the number of silhouette images in the respective target sequence is taken as the weight of the silhouette image. For example, a target sequence with a certain gait cycle marker value of 0 contains 4 silhouette images, and the weight of each of the 4 silhouette images is 1 / 4.

[0121] Since there are multiple complete target sequences, the number of silhouette images in each target sequence can be the same or different, and the fitting processing is performed based on the length of the target sequence with the largest number of images.

[0122] The correspondence between the silhouette images in each target sequence and the silhouette images in the target sequence with the largest number of images is determined. Assuming that the number of silhouette images in the target sequence with the largest number of images is M, the other target sequences (with N number of silhouette images) are placed in the length of M in proportion, such as sequence one M = 5 and sequence two N = 3. The first image in sequence two is placed at the first position of sequence one, the second image in sequence two is placed at the third position of sequence one, and the third image in sequence two is placed at the fifth position of sequence one, thereby determining the correspondence between the silhouette images in different target sequences.

[0123] The binary images in different target sequences at the same position or with the correspondence are subjected to weighted average processing, a new image sequence containing M silhouette images is calculated, and then binary processing with a threshold of 0.5 is performed, thereby obtaining the fitted complete gait cycle silhouette sequence.

[0124] Optionally, after the mutation correction of the silhouette sequence, the fitting processing of multiple target sequences is performed. Figure 5 is a flowchart of a model application method for cross-view gait feature extraction provided by an embodiment of the present application, as shown in Figure 5 After the initial silhouette sequence is obtained by the target silhouette deep learning model, the initial silhouette sequence is subjected to mutation correction, the gait cycle is segmented, and then the fitting processing of multiple target sequences is performed.

[0125] The cross-view gait feature extraction method provided by the embodiment of the present application extracts the complete gait cycle sequence from the initial silhouette sequence by using the gait cycle marker value, ensures the time sequence consistency of the input sequence of the gait feature extraction model, corrects the mutation images in the same stage of the gait cycle, reduces the error of the gait feature extraction result, fits multiple target sequences, integrates the semantic and time sequence information of the multiple sequences, and enhances the robustness.

[0126] In some embodiments, the gait feature extraction method further comprises:

[0127] determining the contour centroid of the target in each silhouette image in the initial silhouette sequence;

[0128] aligning the contour centroids of different silhouette images in the initial silhouette sequence.

[0129] Specifically, in order to maintain the original size of each silhouette to avoid any loss or reduce the size of the distorted information, the silhouette sequence is centroid-aligned: the contour centroids of the target in different silhouette images are aligned, and then the silhouette images can be stretched to the same size.

[0130] In order to determine the contour centroid of the target in the silhouette image, the weighted average value of the center of each row is calculated to obtain the X coordinate, and the weighted average value of the center of each column is calculated to obtain the Y coordinate, which is as follows:

[0131]

[0132] where C x is the X coordinate of the contour centroid, L n is the weight of the nth row, i.e. its length, X n is the x coordinate of the center of the nth row, S r is the sum of the weights of all rows. Since the weights of the rows are their lengths, S r can be expressed as the sum of the lengths of all rows.

[0133] The contour centroid is used for alignment instead of the center of the upper part of the contour, because the upper center of the pedestrian contour is easily affected by the change of clothing (including coat and backpack, etc.). The centroid of each silhouette contour is aligned to the centroid of the black image with the same size as the video frame. For example, in OpenCV, the images are combined by providing the coordinates of the upper left corner of the smaller image (silhouette image) in the larger image frame (black image), as well as the height and width of the smaller image (silhouette image).

[0134] Figure 6 is a structural schematic diagram of a training device for a model for extracting gait features across view angles provided by an embodiment of the present application, as shown in Figure 6 The device at least includes:

[0135] The acquisition module 601 is configured to acquire training samples; wherein the training samples include video data and annotation information, the video data includes video segments of each collection sample in different dressing, different postures and different walking routes, which are obtained based on preset conditions after filtering and video segment division after the collection sample is shot in a collection area by a plurality of cameras arranged in a semicircle; the semicircle area formed by the plurality of cameras includes two quarter circles, the cameras in one quarter circle are uniformly distributed at a first angle, the cameras in the other quarter circle are uniformly distributed at a second angle, and the first angle and the second angle are different; the annotation information includes position information, posture information and binary silhouette of the collection sample in a frame image, and a position of the frame image in a gait cycle;

[0136] The training module 602 is configured to train a target silhouette deep learning model and a gait feature extraction model based on the training samples, and obtain the trained target silhouette deep learning model and the gait feature extraction model; wherein the target silhouette deep learning model is used to determine a binary silhouette of the collection sample in an input image and a gait cycle flag value of a silhouette image, the gait cycle flag value is used to represent a position of the silhouette image in the gait cycle, and the gait feature extraction model is used to extract gait features in the silhouette image.

[0137] In some embodiments, the position of the frame image in the gait cycle is annotated by the following way:

[0138] The position of the frame image in the gait cycle is determined based on a time sequence order of the frame image in the video segment and positions of a head and limbs of the collection sample in the frame image.

[0139] In some embodiments, the position of the frame image in the gait cycle is determined by the following way:

[0140] In the frame images in time sequence, the frame images from the right heel touch-down of the collection sample to the left toe lift-off are annotated as the first position of a complete gait cycle;

[0141] The frame images from the left toe lift-off to the left heel touch-down of the collection sample are annotated as the second position;

[0142] The frame images from the left heel touch-down to the right toe lift-off of the collection sample are annotated as the third position;

[0143] The frame images from the right toe lift-off to the right heel touch-down of the collection sample are annotated as the fourth position, and the annotation of a complete gait cycle is completed.

[0144] In some embodiments, the position information of the collection sample is obtained by the following way:

[0145] Input the frame image into the position detection deep learning model to obtain position information of the collected sample, the position information including a minimum circumscribed rectangle of the collected sample contour and vertex coordinate information of the minimum circumscribed rectangle, the vertex coordinate information being used to determine the position of the minimum circumscribed rectangle.

[0146] The position detection deep learning model is obtained based on sample images and pre-determined position labeling information after training, and the position detection deep learning model adopts a yolov5 model.

[0147] In some embodiments, the binary silhouette is labeled by the following way:

[0148] The edge contour of the collected sample in the frame image is labeled by a polygon;

[0149] Based on the edge contour of the collected sample, it is distinguished whether a pixel point in the frame image is a foreground pixel point or a background pixel point;

[0150] The foreground pixel point is set to 255, and the background pixel point is set to 0, to obtain a binary silhouette image with black background and white collected sample silhouette.

[0151] In some embodiments, the target silhouette deep learning model adopts a segmentation convolutional network, the segmentation convolutional network including an encoder module, a decoder module, an encoding-decoding connection module and a pixel-level classification layer;

[0152] The encoder module includes a convolutional layer and a maximum pooling layer, the decoder module includes a convolutional layer and an up-sampling layer, and the network structures of the encoder module and the decoder module are symmetrical;

[0153] The encoder module records boundary information through a pooling index, and the decoder module realizes nonlinear up-sampling through a pooling index derived from a corresponding maximum pooling layer;

[0154] The encoding-decoding connection module adopts a skip connection, and each convolutional layer before a maximum pooling layer in the encoder module is connected to a last convolutional layer of a corresponding size in the decoder module.

[0155] In some embodiments, the gait feature extraction model adopts a GAN network, the GAN network including k sets of generative adversarial networks, k being the number of perspectives of the input image, and a generator in the k sets of generative adversarial networks generating gait maps in k perspectives and outputting after fusion.

[0156] It can be understood that the detailed function implementation of each unit / module described above can refer to the description in the foregoing model training method embodiments, which will not be repeated here. It should be understood that the above device is used to execute the model training method in the foregoing embodiments, the corresponding program modules in the device, the implementation principles and technical effects of which are similar to the description in the foregoing model training method, and the working process of the device can refer to the corresponding process in the foregoing model training method, which will not be repeated here.

[0157] Figure 7 is a structural schematic diagram of a cross-view gait feature extraction device provided by the embodiment of the present application, as shown in the figure, the device at least includes: Figure 7

[0158] The first acquisition module 701 is configured to pre-process a video captured by a cross-view camera to obtain an initial image sequence, and the images in the initial image sequence are minimum rectangular images containing a target;

[0159] The first generation module 702 is configured to input the initial image sequence into a target silhouette deep learning model to generate an initial silhouette sequence of the target and a gait cycle flag value of a silhouette image in the initial silhouette sequence, the gait cycle flag value being used to represent a position of the silhouette image in a gait cycle;

[0160] The second acquisition module 703 is configured to obtain a complete gait cycle silhouette sequence of the target based on the gait cycle flag value of the silhouette image in the initial silhouette sequence;

[0161] The second generation module 704 is configured to input the gait cycle silhouette sequence into a gait feature extraction model to generate an n-dimensional feature vector, the n-dimensional feature vector being used to represent a gait feature corresponding to the initial image sequence.

[0162] In some embodiments, the second acquisition module 703 is specifically configured to:

[0163] select an initial gait cycle silhouette sequence containing a complete gait cycle of the target from the initial silhouette sequence based on the gait cycle flag value;

[0164] determine a mutation image with a position mutation in each target sequence based on the positions of the head and limbs of the target in the image, and correct the mutation image; the target sequence refers to a silhouette sequence with the same gait cycle flag value in the initial gait cycle silhouette sequence;

[0165] determine a corresponding relationship between the silhouette images in each target sequence and the silhouette images in a target sequence with the most images;

[0166] ​The weight of the silhouette image is the proportion of the number of the silhouette image in the respective target sequence, the silhouette image in the target sequence with the largest number of images and the corresponding silhouette image in other target sequences are processed by weighted average to determine a new image sequence with the same length as the target sequence with the largest number of images;

[0167] The new image sequence is binarized to obtain a gait cycle silhouette sequence.

[0168] In some embodiments, the apparatus further comprises an alignment module configured to:

[0169] Determine the contour centroid of the target in each silhouette image in the initial silhouette sequence;

[0170] Align the contour centroids of different silhouette images in the initial silhouette sequence.

[0171] It can be understood that the detailed function implementation of each unit / module described above can refer to the description in the foregoing model application method embodiments, which will not be repeated here. It should be understood that the above apparatus is used to execute the model application method in the foregoing embodiments, the corresponding program modules in the apparatus, the implementation principles and technical effects are similar to the description in the foregoing model application method, and the working process of the apparatus can refer to the corresponding process in the foregoing model application method, which will not be repeated here.

[0172] Based on the method in the foregoing embodiments, the embodiments of the present application provide an electronic device. The device can include at least one memory for storing programs and at least one processor for executing the programs stored in the memory. Wherein, when the programs stored in the memory are executed, the processor is configured to execute the model training method or the pedestrian gait feature extraction method described in the foregoing embodiments.

[0173] Figure 8 is a structural schematic diagram of an electronic device provided by the embodiments of the present application, as shown in Figure 8 The electronic device can include a processor 801, a communications interface 820, a memory 803 and a communications bus 804, wherein the processor 801, the communications interface 802 and the memory 803 complete mutual communication through the communications bus 804. The processor 801 can invoke the software instructions in the memory 803 to execute the model training method or the pedestrian gait feature extraction method described in the foregoing embodiments.

[0174] In addition, the logic instructions in the memory 803 described above can be implemented in the form of software function units and sold or used as independent products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that make contributions to the related art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the method of the present application.

[0175] Based on the method in the above embodiments, the embodiments of the present application provide a computer readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the training method of the model or the pedestrian gait feature extraction method in the above embodiments.

[0176] Based on the method in the above embodiments, the embodiments of the present application provide a computer program product, which runs on a processor. When the computer program product runs on the processor, the processor executes the training method of the model or the pedestrian gait feature extraction method in the above embodiments.

[0177] It can be understood that the processor in the embodiments of the present application can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor can be a microprocessor or any conventional processor.

[0178] The method steps in the embodiments of the present application can be implemented by means of hardware, or by means of a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in a Random Access Memory (RAM), a flash memory, a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), a register, a hard disk, a mobile hard disk, a CD-ROM, or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor, so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0179] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to the embodiments of the present application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted by a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through a wired (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. containing one or more available media sets. The available media can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a Solid State Disk (SSD)), etc.

[0180] It can be understood that the various numerical numbers involved in the embodiments of the present application are only for the convenience of differentiation, and do not limit the scope of the embodiments of the present application.

[0181] Those skilled in the art can easily understand that the above are only the preferred embodiments of the present application, and are not used to limit the present application, any modification, equivalent replacement and improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for training a model for gait feature extraction across view angles, characterized in that, The method comprises the following steps: acquiring training samples; wherein the training samples comprise video data and annotation information, the video data comprises video segments of each collection sample in different dressings, different postures and different walking routes, which are obtained after a plurality of cameras arranged in a semicircle shoot in a collection area, are screened based on a preset condition and are cut into video segments; the semicircle area formed by the plurality of cameras comprises two quarter circles, the cameras in one quarter circle are uniformly distributed at a first angle, the cameras in the other quarter circle are uniformly distributed at a second angle, and the first angle and the second angle are different; the annotation information comprises position information, posture information and binary silhouette of the collection sample in a frame image, and a position of the frame image in a gait cycle; training a target silhouette deep learning model and a gait feature extraction model based on the training samples to obtain the trained target silhouette deep learning model and the gait feature extraction model; wherein the target silhouette deep learning model is used to determine a binary silhouette of a collection sample in an input image and a gait cycle flag value of a silhouette image, the gait cycle flag value is used to represent a position of the silhouette image in the gait cycle, and the gait feature extraction model is used to extract gait features in the silhouette image; the target silhouette deep learning model adopts a segmentation convolutional network, the segmentation convolutional network comprises an encoder module, a decoder module, an encoding-decoding connection module and a pixel-level classification layer; the encoder module comprises a convolutional layer and a max-pooling layer, the decoder module comprises a convolutional layer and an up-sampling layer, and the network structures of the encoder module and the decoder module are symmetrical; the encoder module records boundary information through a pooling index, and the decoder module realizes nonlinear up-sampling through a pooling index derived from a corresponding max-pooling layer; the encoding-decoding connection module adopts a skip connection, and a convolutional layer before each max-pooling layer in the encoder module is connected to a last convolutional layer of a corresponding size in the decoder module to obtain a feature map of a corresponding scale, and different scale feature maps are fused into a fusion map. 2.The method of Claim 1, wherein, The position of the frame image in the gait cycle is annotated in the following way: based on the time sequence order of the frame image in the video segment and the positions of the head and limbs of the collection sample in the frame image, the position of the frame image in the gait cycle is determined. 3.The method of Claim 2, wherein, The determination of the position of the frame image in the gait cycle comprises: in the time-sequentially continuous frame images, the frame images from the right heel touch-down of the collection sample to the left toe lift-off are annotated as the first position of a complete gait cycle; the frame images from the left toe lift-off to the left heel touch-down of the collection sample are annotated as the second position; the frame images from the left heel touch-down to the right toe lift-off of the collection sample are annotated as the third position; the frame images from the right toe lift-off to the right heel touch-down of the collection sample are annotated as the fourth position, and the annotation of a complete gait cycle is completed. 4.The method of Claim 1, wherein The position information of the collection sample is obtained in the following way: inputting the frame image into a position detection deep learning model to obtain position information of the collected sample, the position information including a minimum circumscribed rectangle of a profile of the collected sample and vertex coordinate information of the minimum circumscribed rectangle, the vertex coordinate information being used to determine the position of the minimum circumscribed rectangle; wherein the position detection deep learning model is obtained based on sample images and pre-determined position labeling information after training, and the position detection deep learning model adopts a yolov5 model. 5.The method of Claim 1, wherein The binary silhouette is labeled by the following method: an edge profile of the collected sample in the frame image is labeled by a polygon; based on the edge profile of the collected sample, it is determined whether a pixel point in the frame image is a foreground pixel point or a background pixel point; the foreground pixel point is set to 255, and the background pixel point is set to 0, to obtain a binary silhouette image in which the background is black and the collected sample silhouette is white. 6.The method of Claim 1, wherein The gait feature extraction model adopts a GAN network, the GAN network includes k sets of generative adversarial networks, k is the number of perspectives of the input image, and a generator in the k sets of generative adversarial networks generates gait graphs in k perspectives and outputs after fusion.

7. A gait feature extraction method across view angles, characterized in that, including: preprocessing a video captured by a camera across perspectives to obtain an initial image sequence, and images in the initial image sequence being minimum rectangular images containing a target; inputting the initial image sequence into a target silhouette deep learning model to generate an initial silhouette sequence of the target and a gait cycle flag value of a silhouette image in the initial silhouette sequence, the gait cycle flag value being used to represent a position of the silhouette image in a gait cycle; based on the gait cycle flag value of the silhouette image in the initial silhouette sequence, obtaining a complete gait cycle silhouette sequence of the target; inputting the gait cycle silhouette sequence into a gait feature extraction model to generate an n-dimensional feature vector, the n-dimensional feature vector being used to represent a gait feature corresponding to the initial image sequence; wherein the target silhouette deep learning model and the gait feature extraction model are obtained by the training method in any one of claims 1 to 6.

8. The cross-view gait feature extraction method of claim 7, wherein, The method further includes: based on the gait cycle flag value, selecting an initial gait cycle silhouette sequence containing a complete gait cycle of the target from the initial silhouette sequence; based on positions of a head and limbs of the target in an image, determining a mutation image in which a position mutates in each target sequence, and correcting the mutation image; the target sequence refers to a silhouette sequence in which the gait cycle flag values are the same in the initial gait cycle silhouette sequence; determining a corresponding relationship between a silhouette image in each target sequence and a silhouette image in a target sequence with the most images; taking a proportion of the number of silhouette images in each target sequence as a weight value of the silhouette image, and performing weighted average processing on the silhouette image in the target sequence with the most images and the corresponding silhouette images in other target sequences to determine a new image sequence with a length same as that of the target sequence with the most images; performing binary processing on the new image sequence to obtain the gait cycle silhouette sequence.

9. The cross-view gait feature extraction method of claim 7, wherein, The method further includes: determining a contour centroid of the target in each silhouette image in the initial silhouette sequence; aligning the contour centroids of different silhouette images in the initial silhouette sequence.

Images