Methods, apparatus, devices and readable media for gait identification

Abstract

Embodiments of this disclosure provide a method, apparatus, device, and medium for gait recognition. The method includes: generating a corresponding part analysis sequence of the target object by parsing a target object in a target image sequence, wherein the part analysis sequence indicates the corresponding positions of multiple parts constituting the target object in the target image sequence; extracting global gait features of the target object based on the part analysis sequence; extracting multiple local gait features of the target object based on the part analysis sequence, wherein each local gait feature characterizes a specific part among the multiple parts; and determining a gait recognition result of the target object based on a combination of the global and local gait features. The solution of this disclosure can analyze the gait of a target object using global and local features based on the part analysis result, thereby improving the accuracy of gait recognition.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein generally relate to the field of computer vision technology, and more specifically, to methods, apparatus, devices, and computer-readable storage media for gait recognition. Background Technology

[0002] With technological advancements and societal development, particularly in areas like smart cities and intelligent transportation, people's lives are becoming increasingly convenient. Alongside these advancements, biometric technologies are also evolving to balance user convenience with information security. For example, user identification can be based on biometric features such as face, iris, and fingerprint. However, these technologies are typically limited by recognition distance and require the active cooperation of the user. Furthermore, because these features are relatively fixed, they are easily stolen and forged.

[0003] In this context, gait recognition technology has emerged. Gait recognition can identify individuals based on their walking posture, offering advantages such as non-contact, long-range accuracy, and resistance to spoofing, enabling seamless identification during walking. Therefore, improving the precision and accuracy of gait recognition is a crucial and pressing need. Summary of the Invention

[0004] In a first aspect of this disclosure, a method for gait recognition is provided. The method includes: generating a part analysis sequence corresponding to a target object by parsing a target object in a target image sequence, wherein the part analysis sequence indicates the corresponding positions of multiple parts constituting the target object in the target image sequence; extracting global gait features of the target object based on the part analysis sequence; extracting multiple local gait features of the target object based on the part analysis sequence, wherein each local gait feature characterizes a specific part among the multiple parts; and determining a gait recognition result of the target object based on a combination of the global gait features and the local gait features.

[0005] In a second aspect of this disclosure, an apparatus for gait recognition is provided. The apparatus includes: a parsing module configured to generate a part analysis sequence corresponding to a target object by parsing a target object in a target image sequence, wherein the part analysis sequence indicates the corresponding positions of multiple parts constituting the target object in the target image sequence; a first extraction module configured to extract global gait features of the target object based on the part analysis sequence; a second extraction module configured to extract multiple local gait features of the target object based on the part analysis sequence, wherein each local gait feature characterizes a specific part among the multiple parts; and a determination module configured to determine the gait recognition result of the target object based on a combination of the global gait features and the local gait features.

[0006] In a third aspect of this disclosure, an electronic device is provided. The electronic device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method of the first aspect of this disclosure when executed by the at least one processing unit.

[0007] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program that can be executed by a processor to perform the method according to a first aspect of this disclosure.

[0008] It should be understood that the description in the Summary of the Invention section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0009] The above and other features, advantages, and aspects of various implementations of this disclosure will become more apparent in the following detailed description, taken in conjunction with the accompanying drawings. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:

[0010] Figure 1 A schematic diagram of an example environment in which embodiments of the present disclosure can be implemented is shown;

[0011] Figure 2A A schematic diagram of an architecture for gait recognition according to some embodiments of the present disclosure is shown;

[0012] Figure 2B A schematic diagram of the architecture of a global sub-model according to some embodiments of the present disclosure is shown;

[0013] Figure 2C A schematic diagram of the architecture of a partial sub-model according to some embodiments of the present disclosure is shown;

[0014] Figure 2D A schematic diagram of the architecture of a training gait recognition model according to some embodiments of the present disclosure is shown;

[0015] Figure 3 A schematic diagram illustrating a gait recognition process according to some embodiments of the present disclosure is shown;

[0016] Figure 4 A block diagram of a device for gait recognition according to some embodiments of the present disclosure is shown; and

[0017] Figure 5 A block diagram of an electronic device in which one or more embodiments of the present disclosure may be implemented is shown. Detailed Implementation

[0018] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0019] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0020] As used in this paper, the term "model" refers to a system that learns the relationship between inputs and outputs from training data, enabling it to generate corresponding outputs for a given input after training. Model generation can be based on machine learning techniques. Deep learning is a machine learning algorithm that uses multiple layers of processing units to process inputs and provide corresponding outputs. In this paper, "model" may also be referred to as a "machine learning model," "machine learning network," "neural network," or simply "network," and these terms are used interchangeably.

[0021] A neural network is a machine learning network based on deep learning. A neural network processes input and provides a corresponding output, typically consisting of an input layer, an output layer, and one or more hidden layers between the input and output layers. Neural networks used in deep learning applications often include many hidden layers, thus increasing the network's depth. The layers of a neural network are connected sequentially, so that the output of the previous layer is provided as the input to the next layer. The input layer receives the input to the neural network, while the output layer's output serves as the final output. Each layer of a neural network includes one or more nodes (also called processing nodes or neurons), each node processing the input from the layer above.

[0022] Machine learning typically comprises three phases: training, testing, and application (also known as inference). In the training phase, a given model is trained using a large amount of training data, iteratively updating its parameter values ​​until the model can consistently generate inferences that meet the expected goals from the training data. Through training, the model can be considered to have learned the relationship between inputs and outputs (also known as the input-output mapping) from the training data. The parameter values ​​of the trained model are determined. In the testing phase, test inputs are applied to the trained model to test whether it can provide the correct output, thus determining the model's performance. In the application phase, the model can be used to process actual inputs based on the trained parameter values ​​to determine the corresponding output.

[0023] It should be noted that the acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0024] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure through appropriate means in accordance with relevant laws and regulations, and user authorization should be obtained.

[0025] For example, in response to receiving a user's active request, a prompt message is sent to the user to clearly inform the user that the requested operation will require the acquisition and use of the user's personal information, thereby enabling the user to choose whether to provide personal information to the software or hardware such as electronic devices, applications, servers or storage media that perform the operation of the technical solution disclosed herein, based on the prompt message.

[0026] As an optional but non-restrictive implementation, in response to a user's active request, a prompt message can be sent to the user, for example, via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose whether to "agree" or "disagree" to provide personal information to the electronic device.

[0027] For gait recognition technology, the accuracy and precision of gait recognition are directly related to the amount and quality of information available during recognition. For example, in schemes that extract object features from image sequences and use these features to achieve gait recognition, the amount and quality of information (e.g., information entropy) included in the object features directly affect the accuracy and precision of gait recognition. Therefore, embodiments of this disclosure provide a method for gait recognition. According to embodiments of this disclosure, by parsing the target object in a target image sequence, a corresponding part analysis sequence of the target object is generated, wherein the part analysis sequence indicates the corresponding positions of multiple parts constituting the target object in the target image sequence; global gait features of the target object are extracted based on the part analysis sequence; multiple local gait features of the target object are extracted based on the part analysis sequence, wherein each local gait feature characterizes a specific part among the multiple parts; and the gait recognition result of the target object is determined based on the combination of global and local gait features. Thus, based on the part analysis result of the object, global and local features can be used to analyze the gait of the target object, thereby improving the accuracy of gait recognition.

[0028] The following description will focus on exemplary embodiments of the present disclosure with reference to the accompanying drawings.

[0029] Figure 1 A schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented is shown. Figure 1 In environment 100, electronic device 110 can acquire a target image sequence 120 including a target object, typically a human body, thereby enabling gait recognition of the human body. The target image sequence 120 can be obtained based on continuously acquired images of the human body, for example, an image sequence acquired by an optical image acquisition device for a scene including a human body. The target image sequence 120 may include a series of target images.

[0030] After acquiring the target image sequence 120, the electronic device 110 can process it to identify, for example, the gait recognition result 130 of the human object included in the target image sequence 120.

[0031] exist Figure 1In this context, electronic device 110 can include any computing system with computing capabilities, such as various computing devices / systems, terminal devices, servers, etc. Terminal devices can involve any type of mobile terminal, fixed terminal, or portable terminal, including mobile phones, desktop computers, laptop computers, notebook computers, netbook computers, tablet computers, media computers, multimedia tablets, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. Servers include, but are not limited to, mainframes, edge computing nodes, computing devices in cloud environments, etc.

[0032] It should be understood that Figure 1 The components and arrangements shown in environment 100 are merely examples, and a computing system suitable for implementing the exemplary implementations described in this disclosure may include one or more different components, other components, and / or different arrangements. Implementations of this disclosure are not limited in this respect.

[0033] Gait recognition is a special case of pedestrian re-identification. Gait is a highly promising biometric characteristic that reflects a pedestrian's walking pattern. Due to differences in movement and body shape, each person's gait is unique, and this uniqueness can be used to identify target pedestrians in videos. As explained above, improving the accuracy and precision of gait recognition is a noteworthy and urgent need.

[0034] In some approaches, gait recognition methods can be broadly categorized into two types: model-based methods and model-free methods. Model-based methods primarily model the structured model of the human body, such as sequences of human key points. Early gait recognition methods mostly belonged to the model-based category. For example, they modeled 3D gait temporal information from human-estimated skeletal sequences; they used a complex multi-camera system to build a 3D voxel-based dataset and modeled the lower leg based on a cylindrical structure of human joints, giving each joint 3D degrees of freedom; and they modeled key points, motion information, and optical flow information from 3D human key point sequences, integrating these three types of information and using convolutional neural networks to model gait features for gait recognition.

[0035] Model-free methods model human contour sequences. For example, gait contour maps are treated as a set of images without temporal relationships, allowing deep neural networks to extract useful gait features from the gait set. Based on the different gait information contained in various parts of the gait sequence in the horizontal dimension, the network learns fine features of each part and uses short sequences to replace informationally redundant complete gait sequences, thus achieving the highest gait recognition accuracy among existing publicly available methods at the time. Modeling gait temporal information simultaneously considers short-term, long-term, and appearance information, enabling the model to learn a better spatiotemporal representation of gait. 3D convolution is used to simultaneously extract the temporal and spatial features of the gait sequence, and global and local gait features are integrated based on the GLConv module. A ResNet9-like network is used for modeling. Thus, gait recognition is achieved.

[0036] While these methods can achieve high recognition accuracy in laboratory settings, they are typically based on gait contour sequences, a type of gait representation with low information entropy. This makes them unsuitable for the challenges posed by arbitrary walking trajectories, variable walking speeds, and three-dimensional perspective changes in real-world scenarios. Consequently, when applied to real-world situations, recognition efficiency and accuracy significantly decrease.

[0037] This disclosure provides an improved scheme for gait recognition. In this scheme, multiple parts of a target object included in a target image sequence can be analyzed. For ease of understanding, taking a human body as an example, the multiple parts of the target object can include the face, hands, torso, etc. In embodiments of this disclosure, to better recognize gait, the multiple parts can be, for example, the head, torso, left arm, right arm, left hand, right hand, left leg, right leg, left foot, right foot, etc. By analyzing these parts, a part analysis sequence can be obtained to indicate the corresponding positions of the multiple parts constituting the target object in the target image sequence. Therefore, by introducing object part analysis technology, the various parts of the target object can be analyzed in a more granular manner to more accurately identify the object's gait.

[0038] Fine-grained object segmentation is a special case of image semantic segmentation. Image semantic segmentation refers to the process of locating and classifying the category (e.g., road, vehicle, pedestrian, tree, etc.) of each pixel in an image, given an image. Fine-grained object segmentation aims to locate and classify the individual parts (e.g., human hair, face, left / right hand, torso, etc.) that make up the target object in the image. Image semantic segmentation typically employs a deep convolutional neural network structure. The network model is trained on large-scale labeled data, and during the testing phase, the trained model is used to perform forward propagation on the input image to obtain the segmentation results for each type of part in the image.

[0039] Furthermore, after obtaining the part resolution sequence, the electronic device can process the partial resolution sequence to obtain the global gait features and multiple local gait features of the target object. The global gait features can be used to represent the overall information of the target object (e.g., the overall features of the human body when walking). Local gait features are used to represent the local information of specific parts among multiple parts; for example, the local gait features for the right leg can represent the condition of the right leg when the human body is walking. In this way, the condition of each part of the human body can be represented in a more granular way through multiple local features.

[0040] Furthermore, this allows for the combination of global and local gait features to enhance the amount of information obtained about the target object (for example, the combined result can have high information entropy). Therefore, by utilizing the combined result, the gait of human objects can be identified more accurately, improving the precision of gait recognition.

[0041] For ease of understanding, the following explanation will use an example with the human body as the target object.

[0042] The following description will continue with reference to the accompanying drawings, which will provide some exemplary embodiments of this disclosure.

[0043] Figure 2A A schematic diagram of an architecture 200A for gait recognition according to some embodiments of the present disclosure is shown. For ease of discussion, reference will be made to... Figure 1 The environment 100 is used to describe the architecture 200A.

[0044] Example reference Figure 2A .exist Figure 2AIn this embodiment, the electronic device 110 generates a corresponding part analysis sequence of the target object by parsing the target object in the target image sequence. In this embodiment, after acquiring the target image sequence 120, the electronic device 110 can parse it to generate a part analysis sequence 210 of the human body object. The part analysis sequence 210 can be used to indicate the relative positions of various parts of the human body object. For example, for a target image in the target image sequence, the corresponding analysis result can be the positions (e.g., corresponding pixel positions) of the head, torso, left arm, right arm, left hand, right hand, left leg, right leg, left foot, right foot, and bodysuit of the human body object included in the target image. It should be understood that the content in the part analysis sequence 210 can be directly the pixel content of the human body object. For example, the part analysis sequence 210 can be composed directly using the results extracted from the pixel content of the human body object in the target image, or it can be obtained by adding markers to the target image corresponding to the human body object and its parts, for example, to achieve a similar purpose. Thus, the part analysis sequence 210 can be used to indicate the corresponding positions of multiple parts constituting the human body object in the target image sequence 120.

[0045] In some embodiments, the electronic device 110 may utilize the semantic segmentation network 212 as a human body parsing network. The electronic device 110 inputs each target image in the target image sequence 120 into the semantic segmentation network 212 to perform semantic segmentation on each target image, determining the pixel positions of multiple parts of the target object included in the target image, for example... Figure 2A The P() operation is shown in the diagram. This generates a part resolution sequence 210. Specifically, the electronic device 110 can utilize a fully convolutional neural network as a semantic segmentation network 212. This fully convolutional neural network consists of N convolutional layers and M deconvolutional layers, for a total of l layers. The l-th layer takes the input target image I or the output Y of the previous convolutional layer as input. l-1 The input is the feature tensor Y, and the output is the feature tensor Y. l Or, given the segmentation result M, the entire fully convolutional neural network can be represented by the following formula:

[0046] M = P(I) = f l (f l-1 (…f 1 (I))), Y l =f l (Y l-1 ), Y 0 =I (1)

[0047] Among them, f n () represents the convolution operation in the nth convolutional layer, where n = [1, l] ∈ N, and P() represents the operation performed by the semantic segmentation network 212.

[0048] The kernel size in the N convolutional layers of the fully convolutional neural network used to implement the semantic segmentation network 212 can be k. l ×k l ×c l Operation step size s l =k l / 2, therefore the length H of the feature tensor l and width W l Decrease layer by layer; the kernel size of the M deconvolution layers is k. l ×k l ×c l The operation step size sl = 1, therefore the length H of the output feature tensor is... l and width W l The size gradually increases. Each convolutional layer undergoes a batch regularization operation and an activation function, such as a linear rectified function. The kernel size of the last convolutional layer in a fully convolutional neural network is k. l ×k l ×C, where C is the number of predefined human body part categories, outputting a tensor with the same size as the original image.

[0049] A fully convolutional neural network can output features of size W×H×C, where C is the predefined number of segmentation component categories. For each pixel location, a normalization exponential function is used to normalize the channel dimension c, as shown in the formula:

[0050]

[0051] In this way, we can obtain a W×H matrix M, where each pixel position in matrix M corresponds to a category of human body part in the image (e.g., one of the following, as described above: torso, left arm, right arm, left hand, right hand, left leg, right leg, left foot, right foot, and bodysuit).

[0052] Therefore, based on a fully convolutional neural network and predefined human body parts, a novel gait representation can be introduced. For ease of understanding, the above-described part parsing sequence can also be called the Gait Parsing Sequence (GPS). Correspondingly, the fully convolutional neural network described above can also be called the Human Body Part Parsing Network.

[0053] For example, a sequence of target images of a human walking is Where I i ∈R 3×H×W It is an RGB image frame, where H and W are the height and width of the image frame, and N is the sequence length. The human body part parsing network can be used to extract the human body part parsing result x for each frame in the GPS. i =P(I i), where P() can be the processing performed by the semantic segmentation network 212. Therefore, the part resolution sequence X of the target image sequence 120 can be expressed as the following formula:

[0054]

[0055] Where, x i ∈R H×W This represents the human body part analysis results for each frame of the target image. Simultaneously, x... i Each pixel p in j ∈{0,1,…,K-1}, where 0 can be used to refer to the background, and 1 to K-1 can refer to the various human body parts defined above (e.g., head, torso, left arm, right arm, left hand, right hand, left leg, right leg, left foot, right foot, and bodysuit).

[0056] To facilitate understanding, the differences between GPS and traditional gait profile sequences can be discussed from an information theory perspective. According to information theory, GPS gait representation can be expressed by the following formula:

[0057]

[0058] Where, p k K represents the probability that a pixel in the target image belongs to human body part category k. Usually, K (i.e., the total number of categories) is greater than 10. For ease of understanding, we can use GPS with K=16 as an example. Substituting K=16 into the above formula (4), GPS can contain 4 bits of information. Traditional gait contour sequence representation only includes two categories: foreground human body contour and background, so the corresponding K=2. Substituting into the above formula (4), we can find that it contains only 1 bit of information. Thus, the information content of GPS representation is 4 times that of traditional gait contour sequence representation. In other words, GPS is a high information entropy gait representation. Therefore, this high information entropy gait representation can be used to perform gait recognition more accurately.

[0059] In some embodiments, to facilitate GPS analysis and processing, the electronic device 110 can also generate intermediate features from the part parsing sequence. For example, in Figure 2A In this process, electronic device 110 can generate intermediate features 220 based on the part parsing sequence 210. For example, electronic device 110 can utilize... Figure 2A The feature extraction model 214 (denoted as F()) shown is used to extract intermediate features 220. Specifically, after obtaining the GPS gait representation, intermediate features, or intermediate gait features, are extracted through a multi-layer convolutional neural network or Transformer. For example, intermediate features F can be extracted based on the following formula: i :

[0060] F i =F(x) i (5)

[0061] Where, x i Characterizing gait features under GPS, F() represents a multi-layer convolutional neural network or a transformer model. For example, a ResNet9-like model from GaitBase can be used as the feature extraction model 214 to achieve this purpose. i ∈R c×h×w The extracted intermediate feature 220 can be, for example, a frame-level feature, where c represents the number of channels in the feature, and h and w represent the height and width of the feature. Therefore, the intermediate feature 220 can be used to extract global and local gait features of a human object.

[0062] First, the extraction of global gait features of a target object will be explained. Electronic device 110 extracts global gait features of the target object based on a part parsing sequence. In embodiments of this disclosure, electronic device 110 can generate global gait features of the target object based on a part parsing sequence. For example, global features of the human body in the part parsing sequence can be extracted using a pooling operation. For example, the part parsing sequence can be processed using a horizontal pyramid pooling operation to extract global features of the human body in the part parsing sequence. In some embodiments, if intermediate features are generated in the above manner, electronic device 110 can utilize these intermediate features to extract global gait features of the target object. Thus, the extraction quality of global gait features is improved by utilizing optimized intermediate features.

[0063] In some embodiments, the electronic device 110 may also reduce the dimensionality of the features to be processed (e.g., intermediate features 220) in the time dimension to improve the processing quality of horizontal pyramid pooling.

[0064] In some embodiments, the electronic device 110 can also utilize a gait recognition model to achieve gait recognition. The gait recognition model may be configured with at least a global sub-model. Thus, the electronic device 110 can utilize the global sub-model to extract global gait features. Referring, as exemplarily to FIG2, architecture 200A may further include a global sub-model 230. The electronic device 110 can utilize the global sub-model 230 to extract global gait features 250.

[0065] Taking the processing of intermediate feature 220 as an example, in some embodiments, the global sub-model 230 may include a temporal pooling layer and a horizontal pyramid pooling layer. Thus, the temporal pooling layer can be used to reduce the dimensionality of the intermediate feature 220 in the temporal dimension, and then the dimensionality reduction result can be processed using a horizontal pyramid pooling operation to extract global gait features. Specifically, the global sub-model 230 uses temporal pooling to pool feature F... i Dimensionality reduction is performed in the time dimension, followed by horizontal pyramid pooling to further extract more refined global gait features. You can refer to the following formula:

[0066]

[0067] Here, T() represents temporal pooling, which in practice uses max pooling along the time dimension, and H() represents horizontal pyramid pooling.

[0068] For easier understanding, please refer to Figure 2B . Figure 2B A schematic diagram of the architecture of a global sub-model 230 according to some embodiments of the present disclosure is shown. Figure 2B As shown, the global sub-model 230 may include a temporal pooling layer 231 for reducing the dimensionality of the intermediate features 220 in the temporal dimension. After dimensionality reduction, the intermediate features 220 can be reduced to dimensionality-reduced intermediate features 232, which can then be processed by a horizontal pyramid pooling layer 233 to obtain the global gait features 250.

[0069] Therefore, while facilitating extraction using the global sub-model in the gait recognition model, the extraction quality is improved.

[0070] Next, the extraction of local gait features of a target object will be described. Electronic device 110 extracts multiple local gait features of the target object based on a part parsing sequence. In embodiments of this disclosure, each local gait feature, as described above, is used to characterize a specific part among multiple parts. Electronic device 110 can perform processing based on the corresponding position of the part indicated in the part parsing sequence (e.g., extracting pixel content at the corresponding position to identify and process the pixel content to obtain local gait features).

[0071] Similarly, if intermediate features are generated in the manner described above, the electronic device 110 can also utilize these intermediate features to extract local gait features of the target object. Thus, the quality of local gait feature extraction is improved by utilizing optimized intermediate features.

[0072] Alternatively, a gait recognition model can be configured to extract local gait features. In some embodiments, the gait recognition model can be configured with two branches to simultaneously extract global and local gait features. For example, the gait recognition model may include two branches: a global sub-model and a local sub-model.

[0073] For example, Figure 2A The global sub-model 230 and the local sub-model 240 can be provided by the same gait recognition model, for example, by the gait recognition model 290. This allows the electronic device 110 to simultaneously extract global gait features 250 using the global sub-model 230 and extract local gait features 260 using the local sub-model 240.

[0074] For local sub-model 240, further reference can be made. Figure 2C . Figure 2C A schematic diagram of the architecture of a local sub-model 240 according to some embodiments of the present disclosure is shown. Figure 2C In this process, the electronic device 110 can use the separator 241 of the local sub-model 240 to separate multiple masking sequences (e.g., masking sequences 242-1, 242-2) corresponding to multiple parts from the part parsing sequence 210. Specifically, the part parsing sequence can be represented in matrix form, such as matrix Furthermore, for ease of understanding, all subsequent examples will be presented in matrix form. Taking one of multiple parts as an example, the electronic device 110 can obtain the masking matrix corresponding to the part based on the following formula. (For example, masking sequences 242-1 and 242-2):

[0075]

[0076] in, The function is an indicator; it returns 1 when the input satisfies the condition x = k, and 0 otherwise. Therefore, mask sequences 242-1 and 242-2 can be used to indicate the corresponding positions of multiple parts within the target image sequence. Taking one of the mask sequences as an example, such as mask sequence 242-1, when represented in matrix form, we can obtain the mask matrix. Masking matrix The element corresponding to the human body part k in the image has a value of 0, and the others have a value of 1.

[0077] In some embodiments, when processing the part resolution sequence 210 using the separator 241 to separate the mask sequences 242-1 and 242-2, the size of the part resolution sequence 210 can also be adjusted, and the separator 241 can be used to process the size-adjusted part resolution sequence 210 to improve processing quality. For example, this method can avoid data loss, identification errors, etc., caused by size mismatch. Specifically, this is also exemplified in matrix form. The electronic device 110 can use the nearest neighbor interpolation method to convert the W×H part resolution matrix M into the w×h matrix of the example described above.

[0078] Furthermore, the electronic device 110 can at least utilize masking sequences (e.g., masking sequences 242-1, 242-2) to determine multiple part-level intermediate features corresponding to multiple parts from the intermediate features 220. For example, in Figure 2C In this embodiment, the electronic device 110 can use operation 243 (e.g., multiplication operation) to use masking sequences 242-1, 242-2 to determine multiple part-level intermediate features corresponding to multiple parts from the intermediate features 220. It should be understood that, in embodiments of this disclosure, one or both of the masking sequences 242-1, 242-2 can be used to determine the part-level intermediate features based on the actual situation.

[0079] In some embodiments, when determining multiple site-level intermediate features corresponding to multiple sites, the electronic device 110 can also use control factors and mask sequences 242-1 and 242-2 to determine multiple site-level intermediate features corresponding to multiple sites from intermediate features 220. Alternatively, control factors can be introduced during the process of determining multiple site-level intermediate features corresponding to multiple sites from intermediate features 220 using mask sequences 242-1 and 242-2. Typically, control factors correspond to multiple sites, meaning there are multiple control factors for multiple sites. In some embodiments, multiple control factors are determined during the training of the gait recognition model 290. Specifically, a control factor is a learnable factor whose value can be determined during the training of the local sub-model 240, or in other words, the value of the control factor is determined following the training of the local sub-model 240. Thus, the electronic device 110 can adjust the foreground and background weights of human body parts based on the values ​​of the control factors. Therefore, by using control factors to automatically control the contribution weights of each human body part to the gait recognition task, attention weights for key human body parts can be effectively achieved.

[0080] For example, electronic device 110 can use masking sequences 242-1 and 242-2 to determine the location-level intermediate features for location k from intermediate features 220 based on the following formula.

[0081]

[0082] Among them, F i For example, intermediate feature 220, For masking sequences (e.g., matrix-form masking sequences 242-1, 242-2), γ k This is a control factor.

[0083] Furthermore, the electronic device 110 can utilize the location-level intermediate features obtained from the above steps. This allows us to obtain multiple local gait features of the target object. Similarly, we can obtain matrix-form location-level intermediate features. For example, electronic device 110 can control the G of w×h×l. c (I) The matrix is ​​averaged and summed along its third dimension to obtain the feature vector r of body part k. c The feature vector r c The dimension can be 1×l.

[0084] Furthermore, the electronic device 110 can convert each feature vector r c Combine them to obtain a feature matrix composed of feature vectors of different parts. For example, the feature matrix It can be in the following forms:

[0085]

[0086] In embodiments of this disclosure, an adjacency graph of human body parts can be constructed, enabling the electronic device 110 to further process part-level intermediate features, for example, through convolution. This allows the local gait features 260 obtained based on part-level intermediate features (e.g., a feature matrix composed of part-level intermediate features) to include various information such as part adjacency relationships. In some embodiments, the electronic device 110 can utilize a graph convolutional neural network 244 to process the feature matrix. The data is processed to obtain the local gait features 260 of the target object. Specifically, the electronic device 110 can process the feature matrix... The input graph is a convolutional neural network G(·) (e.g., a convolutional neural network 244). G(·) can have multiple convolutional layers (e.g., L layers). Each convolutional layer can be calculated using the following formula:

[0087]

[0088] Where A is the adjacency matrix of K types of human body parts, D is the degree matrix of matrix A, and X (L) W is the output of the Lth layer. (L)Let σ be the parameter of the Lth layer, and σ(·) be the activation function. The input of the first layer of the convolutional neural network G(·) is the feature matrix. The output is the feature matrix after L layers of operations. Therefore, graph convolutional neural networks are used to model the relationships between different parts of the human body, thereby improving the ability to represent local features.

[0089] In some embodiments, in order to achieve dimensional matching between the obtained global gait features 250 and local gait features 260, the electronic device 110 may also perform time pooling (e.g., the graph convolutional neural network 244 may also include a time pooling layer) to process the output of the graph convolutional neural network G(). After time pooling, local gait features are obtained. (For example, local gait feature 260).

[0090] It should be understood that the graph convolutional neural network 244 can be configured in the local sub-model 240 to enable the local sub-model 240 to have the aforementioned processing capabilities. In some embodiments, the graph convolutional neural network 244 can also be configured independently of the local sub-model 240, and correspondingly, the local sub-model 240 can independently utilize the graph convolutional neural network 244 to obtain the aforementioned processing capabilities.

[0091] It should be understood that, depending on actual needs, one can choose to prioritize the extraction of global gait features, prioritize the extraction of local gait features, or extract global and local gait features simultaneously. This disclosure is not intended to limit the extraction order of global and local gait features.

[0092] Furthermore, the electronic device 110 determines the gait recognition result of the target object based on the combination of global gait features 250 and local gait features 260. In embodiments of this disclosure, the electronic device 110 can utilize the combination of global gait features 250 and local gait features 260, for example, to perform gait analysis on the part parsing sequence 210 based on feature pairs and feature groups obtained from global gait features 250 and local gait features 260. For example, feature pairs and feature groups can be analyzed based on a pre-trained feature processing model, a feature processing sub-model included in the gait recognition model, etc., to obtain gait recognition results. For example, a feature pair of type A (e.g., where global gait feature 250 can be classified as type A1 and local gait feature 260 can be classified as type A2) can have a corresponding gait B.

[0093] In some embodiments, the electronic device 110 can concatenate global gait features 250 and local gait features 260 to combine the global gait features 250 and local gait features 260 (e.g., combining global gait features 250 and local gait features 260 based on combination operation 270) to obtain combined gait features (e.g., combined gait features 280). Thus, gait can be identified based on the fused features (or combined features) of global gait features 250 and local features 260. For example, the electronic device 110 can extract the... and Features are concatenated to achieve combination. In some embodiments, the combination operation 270 may further include a feature enrichment and remapping operation to further optimize the resulting combined gait features 280.

[0094] In embodiments of this disclosure, after obtaining the combined gait features 280, the electronic device 110 can obtain the gait recognition result 130 of the corresponding target object based on the analysis of the combined gait features 280. For example, the electronic device 110 can match a reference gait feature to the combined gait features 280 based on the comparison results (e.g., cosine similarity, Euclidean distance, etc.) between the combined gait features 280 and known reference gait features, so as to utilize the existing reference gait features to complete the analysis of the combined gait features 280. Alternatively, the electronic device 110 can process the combined gait features 280 based on the direct processing results (e.g., it can also utilize a pre-trained feature processing model, a feature processing sub-model included in the gait recognition model, etc.) to generate the gait recognition result 130. In some embodiments, the reference features can be associated with reference gait sequences. For example, the electronic device 110 can pre-configure and maintain multiple reference features, and maintain a corresponding reference gait sequence for each reference feature (e.g., different types of pedestrian sequences obtained in advance based on analysis). In this scenario, the electronic device 110 can determine a target reference gait feature based on the similarity between the combined gait feature 280 and each reference gait feature in a set of reference gait features. Furthermore, the electronic device 110 can determine the reference gait sequence associated with the target reference gait feature as the recognition result corresponding to the combined gait feature 280, thereby completing the recognition. Thus, gait recognition can be achieved by pre-configuring reference gait features and reference gait sequences, and then using feature matching.

[0095] For example, multiple reference gait features M(G) can be pre-configured. The cosine distance between each reference gait feature M(G) and the combined gait feature M(Q) is calculated, and they are sorted by similarity from smallest to largest. Based on the sorting results, the target reference gait feature with the highest similarity is selected, and the reference gait sequence associated with the target reference gait feature is determined as the least likely recognition result 130. In some embodiments, the electronic device 110 may also pre-maintain a human gait matching set, which maintains the association between reference gait features and reference gait sequences.

[0096] Therefore, based on global gait features 250 and local gait features 260, the discriminative power and robustness of the features can be improved, and accurate gait recognition results can be achieved.

[0097] In the embodiments of this disclosure, the extraction of global gait features 250, the extraction of multiple local gait features 260, and the determination of gait recognition result 130 are achieved through gait recognition model 290. Correspondingly, electronic device 110 can also train the gait recognition model 290. Electronic device 110 can use a training sample set to train the gait recognition model 290 based on a preset loss function. Thus, a dual-branch neural network model can be trained to extract effective features from high-information-entropy gait representation data for human gait similarity matching. Such a gait recognition model can include a global feature extraction network (e.g., global sub-model 230) to extract features from the overall appearance information of the human body based on the parsing sequence of body parts, ensuring that the gait recognition model possesses basic gait recognition capabilities. Furthermore, the gait recognition model can utilize its local feature extraction network (e.g., local sub-model 240) to extract visual features from each body part based on precise segmentation of the body parts, and then use a graph convolutional neural network to learn the adjacency relationships between body parts, thereby extracting more effective local features. The final gait recognition model can combine the two network branches to improve feature quality and increase the accuracy of gait recognition technology.

[0098] In some embodiments, at least one of the triplet loss function and the cross-entropy loss function can be used as a preset loss function to train the gait recognition model. In some embodiments, the triplet loss function and the cross-entropy loss function can also be used simultaneously, for example, in combination with the Adaptive Moment Estimation (Adam) algorithm, to improve the gait recognition model's capabilities.

[0099] For example, you can refer to Figure 2D . Figure 2D A schematic diagram of the architecture 200D of a training gait recognition model according to some embodiments of the present disclosure is shown.

[0100] In architecture 200D, the gait recognition feature model 290 can be trained using a loss function 291 based on a training sample set 292. The training sample set 292 includes multiple training samples, each of which is input into the gait recognition model 290 to be trained, so that the gait recognition model 290 can extract corresponding combined gait features 294. Based on the combined gait features 294 of the training samples, a corresponding loss function 291 (e.g., including one or more of the triplet loss function and cross-entropy loss function) can be designed to update the model parameters of the gait recognition model 290.

[0101] In some embodiments, the triplet loss function can be determined based on: the difference between a first combined gait feature extracted by the gait recognition model 290 from the anchor sample part parsing sequence in the training sample set 292 and a second combined gait feature of the positive sample gait sequence of the anchor sample part parsing sequence; and the difference between the first combined gait feature and a third combined gait feature of the negative sample gait sequence of the anchor sample part parsing sequence. Specifically, the anchor sample part parsing sequence can be based on a pre-determined gait database. In some embodiments, the gait database may store sample part parsing sequences corresponding to various real-world scenarios (e.g., image sequences in real-world scenarios). In some embodiments, the electronic device 110 may also obtain the sample part parsing sequence by acquiring an image sequence in a real-world scenario and then performing parsing as described above. Further, the training sample set 292 can be composed of multiple anchor sample part parsing sequences for training. For the anchor sample part parsing sequences in the training sample set 292, the triplet loss function L can be set, for example, based on the following formula. tri :

[0102]

[0103] Where N is the number of gait sequences in the training sample set (e.g., training sample set 292), and M(·) represents the process of the deep learning model extracting gait features. It is an anchor sample. and These are the positive and negative samples relative to the anchor sample, respectively. It is the first combination of gait features. and These are the second and third combined gait features, respectively, relative to the first combined gait features, where m is the threshold parameter. It is the difference between the first combination of gait features and the second combination of gait features of the negative sample gait sequence of the anchor sample part analytical sequence. This is the difference between the first combination of gait features and the third combination of gait features from the negative sample gait sequence of the anchor sample parsing sequence. Therefore, a triplet loss function can be used as supervision to enable the gait recognition model 290 to learn the similarity or difference between samples, ensuring that the features extracted by the gait recognition model 290 for anchor samples are as close as possible to the features extracted for positive samples, while being as far away as possible from the features extracted for negative samples.

[0104] In some embodiments, the cross-entropy loss function is determined based on the difference between the predicted gait recognition result of the parsed sequence of sample parts in the training sample set 292 using the gait recognition model 290 and the labeled gait recognition result. Specifically, the cross-entropy loss function L can be set based on the following formula. ce :

[0105]

[0106] Where N is the number of gait sequences in the training sample set (e.g., training sample set 292), and C is the total number of people in the training set, which can also be called the number of categories; The label is a symbolic function; if the true class of sample i is c, then... Set the value to 1, otherwise set it to 0; This is the predicted probability that sample i belongs to category c. This allows us to utilize the predicted gait identification results (e.g., based on...). The results of determining whether sample i belongs to category c) and the labeled gait recognition results (e.g., The training process of gait recognition model 290 is supervised by the differences in the indicated content. Thus, the cross-entropy loss function can be used as supervision so that gait recognition model 290 can learn intra-class and inter-class differences in the sample space of the entire dataset.

[0107] In some embodiments, the electronic device 110 may also simultaneously utilize the triplet loss function L tri and cross-entropy loss function L ce To supervise the training of the gait recognition model 290. For example, by setting weights and simultaneously utilizing the triplet loss function L. tri and cross-entropy loss function L ce The training of the gait recognition model 290 is supervised. This allows the gait recognition model 290 to learn similarities or differences between samples, and to learn intra-class and inter-class differences across the entire dataset's sample space.

[0108] For example, the loss function L used to train the gait recognition model 290 can be set based on the following formula. total :

[0109] L total =αL tri +βL ce (13)

[0110] Where α is the weight coefficient corresponding to the triplet loss function, and β is the weight coefficient corresponding to the cross-entropy loss function.

[0111] Subsequently, according to embodiments of this disclosure, by parsing the target object in the target image sequence, a corresponding part analysis sequence of the target object is generated, wherein the part analysis sequence is used to indicate the corresponding positions of multiple parts constituting the target object in the target image sequence; global gait features of the target object are extracted based on the part analysis sequence; multiple local gait features of the target object are extracted based on the part analysis sequence, wherein each local gait feature is used to characterize a specific part among the multiple parts; and the gait recognition result of the target object is determined based on the combination of global gait features and local gait features. The solution of this disclosure can analyze the gait of the target object using global and local features based on the part analysis result of the object, thereby improving the accuracy of gait recognition.

[0112] Figure 3 A flowchart of a gait recognition process 300 according to some embodiments of the present disclosure is shown. Process 300 can be implemented at an electronic device 110.

[0113] In block 310, the electronic device 110 generates a part resolution sequence corresponding to the target object by parsing the target object in the target image sequence. In embodiments of this disclosure, the part resolution sequence is used to indicate the corresponding positions of multiple parts that make up the target object in the target image sequence.

[0114] In box 320, electronic device 110 extracts global gait features of the target object based on the part parsing sequence.

[0115] In block 330, electronic device 110 extracts multiple local gait features of a target object based on a part parsing sequence. In embodiments of this disclosure, each local gait feature is used to characterize a specific part among multiple parts.

[0116] In box 340, electronic device 110 determines the gait recognition result of the target object based on a combination of global gait features and local gait features.

[0117] In some embodiments, process 300 further includes: generating intermediate features from the part parsing sequence; and extracting global gait features of the target object based on the part parsing sequence, including: extracting global gait features of the target object from the intermediate features; and extracting multiple local gait features of the target object based on the part parsing sequence, including: extracting multiple local gait features of the target object from the intermediate features based on the part parsing sequence.

[0118] In some embodiments, global gait features are extracted using a global sub-model in a gait recognition model, and the extraction of global gait features of a target object from intermediate features includes: reducing the dimensionality of intermediate features in the time dimension using a temporal pooling layer of the global sub-model; and processing the dimensionality-reduced intermediate features using a horizontal pyramid pooling layer of the global sub-model to obtain global gait features of the target object.

[0119] In some embodiments, local gait features are extracted using local sub-models in a gait recognition model, and the extraction of local gait features of a target object from intermediate features based on part parsing sequences includes: using a separator of the local sub-model to separate multiple mask sequences corresponding to multiple parts from the part parsing sequence, each mask sequence indicating the corresponding position of a part among the multiple parts in the target image sequence; determining multiple part-level intermediate features corresponding to multiple parts from the intermediate features using at least the multiple mask sequences; and extracting multiple local gait features of the target object from the multiple part-level intermediate features.

[0120] In some embodiments, determining multiple site-level intermediate features corresponding to multiple sites includes: using multiple masking sequences and multiple control factors for multiple sites to determine multiple site-level intermediate features corresponding to multiple sites from the intermediate features.

[0121] In some embodiments, multiple control factors are determined during the training of the gait recognition model.

[0122] In some embodiments, extracting multiple local gait features of a target object from multiple part-level intermediate features includes: processing multiple part-level intermediate features using a graph convolutional neural network in a local sub-model to obtain multiple local gait features of the target object.

[0123] In some embodiments, process 300 further includes: adjusting the size of the part parsing sequence to a target size; and using a separator of the local submodel to separate multiple masking sequences corresponding to multiple parts from the part parsing sequence, including: using a separator of the local submodel to separate multiple masking sequences corresponding to multiple parts from the size-adjusted part parsing sequence.

[0124] In some embodiments, determining the gait recognition result includes: extracting combined gait features of the target object from global gait features and local gait features; and determining the gait recognition result based on the combined gait features.

[0125] In some embodiments, determining the gait recognition result based on the combined gait features includes: determining multiple similarities between the combined gait features and multiple reference gait features, wherein each of the multiple reference features is associated with a reference gait sequence; selecting a reference gait feature from the multiple reference gait features based on the determined multiple similarities; and determining the reference gait sequence associated with the selected reference gait feature as the gait recognition result.

[0126] In some embodiments, the extraction of global gait features, the extraction of multiple local gait features, and the determination of gait recognition results are achieved through a gait recognition model. The gait recognition model is trained based on a preset loss function, which includes at least one of a triplet loss function and a cross-entropy loss function.

[0127] In some embodiments, the triplet loss function is determined based on the following: the difference between a first combined gait feature extracted by the gait recognition model from the anchor sample part parsing sequence in the training sample set and a second combined gait feature of the positive sample gait sequence of the anchor sample part parsing sequence; and the difference between the first combined gait feature and a third combined gait feature of the negative sample gait sequence of the anchor sample part parsing sequence.

[0128] In some embodiments, the cross-entropy loss function is determined based on the difference between the predicted gait recognition result of the parsed sequence of sample parts in the training sample set using the gait recognition model and the labeled gait recognition result.

[0129] In some embodiments, parsing the target object includes: segmenting the target image in the target image sequence using a semantic segmentation network.

[0130] Figure 4 A block diagram of a device 400 for gait recognition according to some embodiments of the present disclosure is shown. The device 400 may be implemented as or include an electronic device 110.

[0131] The apparatus 400 includes a parsing module 410 configured to generate a corresponding part parsing sequence of the target object by parsing a target object in a target image sequence, wherein the part parsing sequence is used to indicate the corresponding positions of multiple parts constituting the target object in the target image sequence. The apparatus 400 further includes a first extraction module 420 configured to extract global gait features of the target object based on the part parsing sequence. The apparatus 400 further includes a second extraction module 430 configured to extract multiple local gait features of the target object based on the part parsing sequence, wherein each local gait feature is used to characterize a specific part among the multiple parts. The apparatus 400 further includes a determining module 440 configured to determine the gait recognition result of the target object based on a combination of global and local gait features.

[0132] In some embodiments, the apparatus 400 further includes: a generation module configured to generate intermediate features from the part parsing sequence; and to extract global gait features of a target object based on the part parsing sequence, including: extracting global gait features of the target object from the intermediate features; and to extract multiple local gait features of the target object based on the part parsing sequence, including: extracting multiple local gait features of the target object from the intermediate features based on the part parsing sequence.

[0133] In some embodiments, global gait features are extracted using a global sub-model in a gait recognition model, and the extraction of global gait features of a target object from intermediate features includes: reducing the dimensionality of intermediate features in the time dimension using a temporal pooling layer of the global sub-model; and processing the dimensionality-reduced intermediate features using a horizontal pyramid pooling layer of the global sub-model to obtain global gait features of the target object.

[0134] In some embodiments, local gait features are extracted using local sub-models in a gait recognition model, and the extraction of local gait features of a target object from intermediate features based on part parsing sequences includes: using a separator of the local sub-model to separate multiple mask sequences corresponding to multiple parts from the part parsing sequence, each mask sequence indicating the corresponding position of a part among the multiple parts in the target image sequence; determining multiple part-level intermediate features corresponding to multiple parts from the intermediate features using at least the multiple mask sequences; and extracting multiple local gait features of the target object from the multiple part-level intermediate features.

[0135] In some embodiments, determining multiple site-level intermediate features corresponding to multiple sites includes: using multiple masking sequences and multiple control factors for multiple sites to determine multiple site-level intermediate features corresponding to multiple sites from the intermediate features.

[0136] In some embodiments, multiple control factors are determined during the training of the gait recognition model.

[0137] In some embodiments, extracting multiple local gait features of a target object from multiple part-level intermediate features includes: processing multiple part-level intermediate features using a graph convolutional neural network in a local sub-model to obtain multiple local gait features of the target object.

[0138] In some embodiments, the apparatus 400 further includes: an adjustment module configured to adjust the size of the part parsing sequence to a target size; and to separate multiple masking sequences corresponding to multiple parts from the part parsing sequence using a local submodel separator, including: separating multiple masking sequences corresponding to multiple parts from the size-adjusted part parsing sequence using a local submodel separator.

[0139] In some embodiments, determining the gait recognition result includes: extracting combined gait features of the target object from global gait features and local gait features; and determining the gait recognition result based on the combined gait features.

[0140] In some embodiments, determining the gait recognition result based on the combined gait features includes: determining the similarity between the combined gait features and a plurality of reference gait features, wherein each reference feature is associated with a reference gait sequence; selecting a reference gait feature from the plurality of reference gait features based on the determined similarity; and determining the reference gait sequence associated with the selected reference gait feature as the gait recognition result.

[0141] In some embodiments, the extraction of global gait features, the extraction of multiple local gait features, and the determination of gait recognition results are achieved through a gait recognition model. The gait recognition model is trained based on a preset loss function, which includes at least one of a triplet loss function and a cross-entropy loss function.

[0142] In some embodiments, the triplet loss function is determined based on the following: the difference between a first combined gait feature extracted by the gait recognition model from the anchor sample part parsing sequence in the training sample set and a second combined gait feature of the positive sample gait sequence of the anchor sample part parsing sequence; and the difference between the first combined gait feature and a third combined gait feature of the negative sample gait sequence of the anchor sample part parsing sequence.

[0143] In some embodiments, the cross-entropy loss function is determined based on the difference between the predicted gait recognition result of the parsed sequence of sample parts in the training sample set using the gait recognition model and the labeled gait recognition result.

[0144] In some embodiments, parsing the target object includes: performing semantic segmentation on each target image in the target image sequence using a semantic segmentation network.

[0145] The modules included in device 400 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more units can be implemented using software and / or firmware, such as machine-executable instructions stored on a storage medium. In addition to or as an alternative to machine-executable instructions, some or all of the units in device 400 can be implemented at least partially by one or more hardware logic components. By way of example, and not limitation, exemplary types of hardware logic components that can be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0146] Figure 5 A block diagram of an electronic device 500 in which one or more embodiments of the present disclosure may be implemented is shown. It should be understood that... Figure 5 The electronic device 500 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein.

[0147] like Figure 5 As shown, electronic device 500 is in the form of a general-purpose electronic device. Components of electronic device 500 may include, but are not limited to, one or more processors or processing units 510, memory 520, storage device 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. Processing unit 510 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 520. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 500.

[0148] Electronic device 500 typically includes multiple computer storage media. Such media can be any available media accessible to electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 520 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 530 can be a removable or non-removable medium and can include machine-readable media, such as flash drives, disks, or any other media that can be used to store information and / or data (e.g., training data for training) and can be accessed within electronic device 500.

[0149] Electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 5 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 520 may include computer program product 525 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0150] Communication unit 540 enables communication with other electronic devices via a communication medium. Additionally, the functionality of components of electronic device 500 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, electronic device 500 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0151] Input device 550 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 560 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 500 can also communicate with one or more external devices (not shown) via communication unit 540 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 500, or with any device that enables electronic device 500 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0152] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores one or more computer instructions, wherein the one or more computer instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transient computer-readable medium and includes computer-executable instructions that are executed by a processor to implement the methods described above.

[0153] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0154] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0155] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0156] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0157] Various implementations of this disclosure have been described above. The foregoing description is exemplary and not exhaustive, nor is it limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the implementations disclosed herein.

Claims

1. A method for gait recognition, comprising: By parsing the target object in the target image sequence, a part parsing sequence corresponding to the target object is generated, wherein the part parsing sequence is used to indicate the corresponding positions of multiple parts that make up the target object in the target image sequence; Intermediate features are generated from the sequence of the described region using a feature extraction model; The global gait features of the target object are extracted from the intermediate features using a gait recognition model. Based on the parsed sequence of the body parts, the gait recognition model is used to determine multiple site-level intermediate features corresponding to the multiple body parts from the intermediate features, including: The gait recognition model utilizes a separator in its local sub-models to extract multiple mask sequences corresponding to the various body parts from the parsed sequence. Each mask sequence indicates the corresponding position of the respective body part within the target image sequence. Using the multiple masking sequences and multiple control factors for the multiple parts, multiple part-level intermediate features corresponding to the multiple parts are determined from the intermediate features, wherein the multiple control factors are determined during the training process of the gait recognition model; Multiple local gait features of the target object are extracted from the multiple site-level intermediate features, wherein each local gait feature is used to characterize a specific site among the multiple sites; and The gait recognition result of the target object is determined based on the combination of the global gait features and the local gait features.

2. The method of claim 1, wherein the global gait features are extracted using a global sub-model in a gait recognition model, and wherein extracting the global gait features of the target object from the intermediate features comprises: The intermediate features are reduced in dimensionality over time by utilizing the temporal pooling layer of the global sub-model. as well as The intermediate features after dimensionality reduction are processed using the horizontal pyramid pooling layer of the global sub-model to obtain the global gait features of the target object.

3. The method according to claim 1, wherein extracting multiple local gait features of the target object from the multiple site-level intermediate features comprises: The graph convolutional neural network in the local sub-model is used to process the multiple part-level intermediate features to obtain multiple local gait features of the target object.

4. The method according to claim 1, further comprising: Adjust the size of the parsed sequence of the aforementioned part to the target size; as well as The separator of the local sub-model is used to separate multiple masking sequences corresponding to the multiple parts from the parsed sequence of the parts, including: The separator of the local sub-model is used to separate multiple masking sequences corresponding to the multiple parts from the size-adjusted part parsing sequence.

5. The method according to claim 1, wherein determining the gait recognition result comprises: Extract combined gait features of the target object from the global gait features and the local gait features; as well as The gait recognition result is determined based on the combined gait features.

6. The method according to claim 5, wherein determining the gait recognition result based on the combined gait features comprises: Determine multiple similarities between the combined gait features and multiple reference gait features, wherein each of the multiple reference gait features is associated with a reference gait sequence; Based on the determined multiple similarities, reference gait features are selected from the multiple reference gait features; and The reference gait sequence associated with the selected reference gait features is determined as the gait recognition result.

7. The method according to claim 1, wherein the extraction of global gait features, the extraction of multiple local gait features, and the determination of gait recognition results are achieved by a gait recognition model, wherein the gait recognition model is trained based on a preset loss function, and the preset loss function includes at least one of a triplet loss function and a cross-entropy loss function.

8. The method of claim 7, wherein the triplet loss function is determined based on: The difference between the first combined gait features extracted by the gait recognition model from the parsed sequences of anchor sample locations in the training sample set and the second combined gait features from the positive sample gait sequences of the parsed sequences of anchor sample locations; and The difference between the first combined gait features and the third combined gait features of the negative sample gait sequence of the anchor sample part parsing sequence.

9. The method of claim 7, wherein the cross-entropy loss function is determined based on the difference between the predicted gait recognition result of the parsed sequence of sample parts in the training sample set using the gait recognition model and the labeled gait recognition result.

10. The method of claim 1, wherein parsing the target object comprises: A semantic segmentation network is used to perform semantic segmentation on each target image in the target image sequence.

11. A device for gait recognition, comprising: The parsing module is configured to generate a part parsing sequence corresponding to the target object by parsing the target object in the target image sequence, wherein the part parsing sequence is used to indicate the corresponding positions of multiple parts that make up the target object in the target image sequence. The generation module is configured to generate intermediate features from the sequence of the said region using a feature extraction model; The first extraction module is configured to extract global gait features of the target object from the intermediate features using a gait recognition model; The second extraction module is configured to determine, based on the parsed sequence of the body parts, multiple location-level intermediate features corresponding to the multiple body parts from the intermediate features using the gait recognition model; Multiple local gait features of the target object are extracted from the multiple site-level intermediate features, wherein each local gait feature is used to characterize a specific site among the multiple sites, and wherein the second extraction module is further configured to: The gait recognition model utilizes a separator in its local sub-models to extract multiple mask sequences corresponding to the various body parts from the parsed sequence. Each mask sequence indicates the corresponding position of the respective body part within the target image sequence. Using the multiple masking sequences and multiple control factors for the multiple parts, multiple part-level intermediate features corresponding to the multiple parts are determined from the intermediate features, wherein the multiple control factors are determined during the training process of the gait recognition model; as well as The determination module is configured to determine the gait recognition result of the target object based on a combination of the global gait features and the local gait features.

12. An electronic device, comprising: At least one processing unit; as well as At least one memory, coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 10 when executed by the at least one processing unit.

13. A computer-readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the method according to any one of claims 1 to 10.

Images