Gait recognition method and device based on quality evaluation, electronic equipment and medium

Abstract

The application provides a gait recognition method and device based on quality evaluation, an electronic device and a medium. The method is used for a plurality of gait picture sequences. According to the features of a plurality of quality attributes in each gait picture, the quality score of the gait picture is determined. The frame-level gait feature sequence of the gait picture sequence is extracted, and the frame-level gait feature sequence is fused based on the quality score of each gait picture to obtain the fused to-be-recognized gait feature. Based on the to-be-recognized gait feature, the gait feature library is searched to obtain at least one target gait feature, so as to determine the gait recognition result of the plurality of gait pictures based on the at least one target gait feature. In this way, quality perception is integrated into gait sequence processing and gait feature extraction, and gait features with higher discrimination are obtained.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and more specifically, to a gait recognition method, apparatus, electronic device, and medium based on quality assessment. Background Technology

[0002] In recent years, with the continuous development of artificial intelligence technology, more and more biometric features can be extracted and used for identity verification. Currently, there are face verification, fingerprint verification, voiceprint verification, and gait verification. In the process of gait verification using gait recognition models, due to the lack of a quality assessment mechanism, the gait image sequence input to the model contains frames with very poor segmentation quality. Furthermore, during gait feature extraction, frames within the same silhouette sequence are assigned the same weight, which causes the poor-quality image frames to affect the accuracy of the extracted gait features, thus impacting the effectiveness of gait verification. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide a gait recognition method, apparatus, electronic device and medium based on quality assessment, which can reduce the impact of poorly segmented gait image frames on the accuracy of gait features, thereby improving the accuracy of gait verification.

[0004] This application provides a gait recognition method based on quality assessment, comprising:

[0005] For a multi-frame gait image sequence, the quality score of the gait image is determined based on the features of multiple quality attributes in each frame.

[0006] Extract frame-level gait feature sequences from the gait image sequence, and fuse the frame-level gait feature sequences based on the quality score of each frame of gait image to obtain the fused gait features to be identified;

[0007] Based on the gait features to be identified, a gait feature database is searched to obtain at least one target gait feature, and the gait recognition results of multiple frames of gait images are determined based on the at least one target gait feature.

[0008] In some embodiments, the quality attributes in the gait recognition method based on quality assessment include: the sharpness of the gait image, the degree of occlusion of the silhouette and / or key points in the gait image.

[0009] In some embodiments, after determining the quality score of the gait image frame, the gait recognition method based on quality assessment further includes:

[0010] Select gait images whose quality scores meet the preset quality conditions and update the gait image sequence.

[0011] In some embodiments, in the gait recognition method based on quality assessment, when the gait image sequence is a silhouette image sequence, the step of extracting frame-level gait feature sequences from the gait image sequence and fusing the frame-level gait feature sequences based on the quality score of each frame of the gait image to obtain the fused gait features to be recognized includes:

[0012] Extract the target gait feature sequence representing the detailed information in each gait image sequence, and fuse the target gait feature sequence based on the quality score of each gait image to obtain the first fused gait feature;

[0013] The low-level gait feature sequence of the gait image sequence is extracted, and the low-level gait feature sequence is fused based on the quality score of each frame of gait image to obtain the second fused gait feature that represents the context information of the gait image sequence.

[0014] The first and second gait features are fused together to obtain the fused gait features to be identified.

[0015] In some embodiments, the gait recognition method based on quality assessment extracts a low-level gait feature sequence from the gait image sequence, and fuses the low-level gait feature sequence based on the quality score of each gait image frame to obtain a second fused gait feature representing the contextual information of the gait image sequence, including:

[0016] The gait image sequence is input into the trained silhouette feature extraction model, and the low-level gait feature sequence of the gait image sequence is extracted through the first part of the convolutional layer of the silhouette feature extraction model;

[0017] Based on the quality score of each gait image, the underlying gait feature sequence is fused to obtain the underlying fused gait features that include gait image sequence context information;

[0018] The second part of the silhouette feature extraction model extracts the contextual information of the underlying fused gait features through the convolutional layer, thus obtaining the second fused gait features that represent the contextual information of the gait image sequence.

[0019] In some embodiments, in the gait recognition method based on quality assessment, when the gait image sequence is a keypoint image sequence, frame-level gait feature sequences are extracted from the gait image sequence, and the frame-level gait feature sequences are fused based on the quality score of each frame of gait image to obtain the fused gait features to be recognized, including:

[0020] The gait image sequence is input into the key point feature extraction model, and the frame-level key point feature sequence of the gait image sequence is extracted by the key point feature extraction model.

[0021] Based on the quality score of each gait image, the features of each frame-level keypoint in the frame-level keypoint feature sequence are weighted, and the weighted frame-level keypoint feature sequence is output.

[0022] The weighted frame-level key point feature sequences are fused together, and the fused gait features to be identified are output.

[0023] In some embodiments, the gait recognition method based on quality assessment involves retrieving at least one target gait feature from a pre-built gait feature database based on the gait feature to be identified, including:

[0024] Calculate the similarity between the gait feature to be identified and at least some gait features in a pre-built gait feature library;

[0025] Target gait features whose similarity meets preset similarity conditions are selected from the at least some gait features.

[0026] In some embodiments, a gait recognition device based on quality assessment is also provided, the device comprising:

[0027] The determination module is used to determine the quality score of a gait image for a multi-frame gait image sequence based on the features of multiple categories of quality attributes in each frame of gait image;

[0028] The fusion module is used to extract the frame-level gait feature sequence of the gait image sequence, and fuse the frame-level gait feature sequence based on the quality score of each frame of gait image to obtain the fused gait feature to be identified.

[0029] The retrieval module is used to search a pre-built gait feature library based on the gait features to be identified, to obtain at least one target gait feature, and to determine the gait recognition result of multiple frames of gait images based on the at least one target gait feature.

[0030] In some embodiments, an electronic device is also provided, including: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory via the bus, and when the machine-readable instructions are executed by the processor, the steps of the gait recognition method based on quality assessment are performed.

[0031] In some embodiments, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, performs the steps of the gait recognition method based on quality assessment.

[0032] This application provides a gait recognition method, apparatus, electronic device, and medium based on quality assessment. The method, for a multi-frame gait image sequence, determines the quality score of each frame based on the characteristics of multiple quality attributes in each frame; extracts frame-level gait feature sequences from the gait image sequence, and fuses these sequences based on the quality scores of each frame to obtain fused gait features to be recognized; searches a pre-built gait feature library based on these features to obtain at least one target gait feature, and determines the gait recognition result of the multi-frame gait images based on the at least one target gait feature. In this way, quality perception is integrated into gait sequence processing and gait feature extraction, reducing the impact of low-quality gait image frames on gait features, thereby obtaining more discriminative gait features and improving the accuracy of gait verification. Attached Figure Description

[0033] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0034] Figure 1 A schematic diagram of the silhouette sequence described in an embodiment of this application is shown;

[0035] Figure 2 A flowchart of the gait recognition method based on quality assessment described in an embodiment of this application is shown;

[0036] Figure 3 A flowchart of the method for extracting frame-level gait feature sequences from gait image sequences according to an embodiment of this application is shown;

[0037] Figure 4 A schematic diagram of the silhouette feature extraction model described in an embodiment of this application is shown;

[0038] Figure 5 A schematic diagram of the key point feature extraction model structure described in an embodiment of this application is shown;

[0039] Figure 6 The flowchart of the method for retrieving gait features from a pre-built gait feature database based on the gait features to be identified, as described in an embodiment of this application, is shown.

[0040] Figure 7 A schematic diagram of the gait recognition device based on quality assessment described in an embodiment of this application is shown;

[0041] Figure 8A schematic diagram of the structure of the electronic device described in the embodiment of this application is shown. Detailed Implementation

[0042] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0043] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0044] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0045] In recent years, with the continuous development of artificial intelligence technology, more and more biometric features can be extracted and used for identity verification. Currently, there are face verification, fingerprint verification, voiceprint verification, and gait verification. In the process of gait verification using gait recognition models, due to the lack of a quality assessment mechanism, the gait image sequence input to the model contains frames with very poor segmentation quality. Furthermore, during gait feature extraction, frames within the same silhouette sequence are assigned the same weight, which causes the poor-quality image frames to affect the accuracy of the extracted gait features, thus impacting the effectiveness of gait verification.

[0046] Please refer to Figure 1 , Figure 1 A sequence of silhouettes was shown; in Figure 1 As can be seen, the silhouette sequence input to the gait recognition model contains frames with very poor segmentation quality, such as... Figure 1As shown in (e); during gait feature extraction, frames in the same silhouette sequence are assigned the same weight, such as... Figure 1 (d) has significantly weaker segmentation quality than Figure 1 (a)-(c), but Figure 1 The silhouette shown in (d) is typically assigned and during gait feature extraction. Figure 1 (a)-(c) have the same weights. This means that some silhouette images cannot accurately represent a person's walking posture, leading to a decrease in the discriminative power of the extracted gait features.

[0047] Based on this, this application provides a gait recognition method, apparatus, electronic device, and medium based on quality assessment. The method, for a multi-frame gait image sequence, determines the quality score of each frame's gait image based on the characteristics of multiple quality attributes in each frame; extracts frame-level gait feature sequences from the gait image sequence, and fuses these frame-level gait feature sequences based on the quality scores of each frame to obtain fused gait features to be recognized; searches a pre-built gait feature library based on these gait features to obtain at least one target gait feature, and determines the gait recognition result of the multi-frame gait images based on the at least one target gait feature; thus, quality perception is integrated into gait sequence processing and gait feature extraction, reducing the impact of low-quality gait image frames on gait features, thereby obtaining more discriminative gait features and improving the accuracy of gait verification.

[0048] Please refer to Figure 2 , Figure 2 The flowchart of the gait recognition method based on quality assessment according to an embodiment of this application is shown. Specifically, the gait recognition method based on quality assessment includes the following steps S201-S203:

[0049] S201. For a multi-frame gait image sequence, determine the quality score of the gait image frame based on the characteristics of multiple categories of quality attributes in each frame.

[0050] S202. Extract the frame-level gait feature sequence from the gait image sequence, and fuse the frame-level gait feature sequence based on the quality score of each frame of gait image to obtain the fused gait feature to be identified.

[0051] S203. Based on the gait features to be identified, a gait feature database is pre-built to obtain at least one target gait feature, and the gait recognition result of multiple frames of gait images is determined based on the at least one target gait feature.

[0052] The method incorporates quality perception into gait sequence processing and gait feature extraction, reducing the impact of low-quality gait image frames on gait features, thereby obtaining more discriminative gait features and improving the accuracy of gait verification.

[0053] In step S201, for a multi-frame gait image sequence, the quality score of the gait image is determined based on the characteristics of multiple categories of quality attributes in each frame of gait image.

[0054] The multi-frame gait image sequence includes multiple gait images, each carrying walking posture information. The walking posture information can be a human silhouette or key human point information; the key human points are points that can characterize walking posture features.

[0055] Thus, the gait images can be silhouette images or key point images.

[0056] Based on the characteristics of multiple quality attributes in each gait image frame, a quality score for that frame is determined. The quality attributes include: the sharpness of the gait image, the degree of occlusion of the silhouette and / or key points in the gait image.

[0057] If the gait images are of low quality, for example, if the gait images are not clear enough, or if... Figure 1 In (d) and (e), the silhouettes or key points in the images are occluded, which affects the discriminative power of the extracted gait features.

[0058] Therefore, this application embodiment incorporates quality assessment into gait sequence processing and gait feature extraction, reducing the weight of low-quality images, thereby obtaining gait features that are more discriminative.

[0059] When determining the quality score of a gait image based on the features of multiple quality attributes in each frame, specifically, a sequence of multiple gait images is input into a trained quality assessment module. The quality assessment module extracts the quality attribute features from each frame of the gait image and determines the quality score of the frame based on the extracted quality attribute features.

[0060] Here, the multi-frame gait images can be videos.

[0061] In this embodiment of the application, the walking sequence of each pedestrian in the input video is analyzed by methods such as segmentation and pose estimation, and the quality of each gait image in the sequence is scored according to blurring, occlusion and other conditions. The score value is normalized to between 0 and 1.

[0062] Specifically, in this embodiment of the application, the quality assessment module may adopt the following two models.

[0063] First: Quality assessment module based on uncertainty

[0064] In a gait sequence extraction model that uses methods such as video segmentation and pose estimation, an uncertainty estimation module is introduced as a quality assessment module. During the model inference phase, for each frame's forward propagation, the model parameters are randomly dropped K times, resulting in K sets of results. By calculating the entropy of the K results, an uncertainty score can be provided for each sample. A higher score means that the gait sequence extraction model considers the prediction result less reliable, and a higher uncertainty score in the entire gait image means a lower image quality, and vice versa. The uncertainty score is normalized to [0,1], and the uncertainty score within [0,1] is output as the quality score. Frames with a quality score lower than 0.1 are discarded.

[0065] Second: CNN-based quality assessment module

[0066] We manually annotate a gait sequence quality dataset, where samples are extracted gait sequences and labels are manually labeled quality scores with values ​​[0,1]. A quality evaluation module is added to the traditional segmentation network / pose estimation parsing network / human parsing network, fine-tuned on the gait sequence quality dataset, and the output is the quality score.

[0067] In this embodiment of the application, the gait recognition method based on quality assessment further includes: after determining the quality score of the gait image frame, the method further includes:

[0068] Select gait images whose quality scores meet the preset quality conditions and update the gait image sequence.

[0069] The preset quality condition is one of the following: the quality score is greater than or equal to a preset score threshold, and a preset number of gait images with the highest quality scores.

[0070] For example, the preset scoring threshold is 0.1, that is, gait images with a quality score greater than or equal to 0.1 are retained, and gait images with a quality score less than 0.1 are discarded.

[0071] In step S202, the frame-level gait feature sequence of the gait image sequence is extracted, and the frame-level gait feature sequence is fused based on the quality score of each frame of gait image to obtain the fused gait feature to be identified.

[0072] Specifically, in this embodiment, frame-level gait feature sequences are extracted from the gait image sequence, and the frame-level gait feature sequences are fused based on the quality score of each frame of gait image to obtain the fused gait features to be identified, including:

[0073] A gait feature extraction model is used to extract frame-level gait feature sequences from gait image sequences; the frame-level gait feature sequence includes multiple frame-level gait features, and each frame-level gait feature corresponds to a quality score;

[0074] The quality score is used to weight and fuse the gait features at different frame levels to obtain the fused gait features to be identified.

[0075] The quality score is used to perform weighted fusion of gait features at different frame levels, that is, the frame-level gait features are multiplied by a quality score between 0 and 1, thereby reducing the impact of poor-quality gait image frames on sequence features.

[0076] In this embodiment of the application, different fusion strategies are adopted because the silhouette image sequence and the key point image sequence contain different information.

[0077] When the gait image sequence is a silhouette image sequence, the step of extracting the frame-level gait feature sequence of the gait image sequence and fusing the frame-level gait feature sequence based on the quality score of each frame of the gait image to obtain the fused gait feature to be identified specifically includes:

[0078] Extract different types of frame-level gait feature sequences from gait image sequences;

[0079] The frame-level gait feature sequences of each type are fused based on the quality score of each gait image to obtain multiple fused gait features;

[0080] Multiple gait features are fused in a secondary manner to obtain the fused gait features to be identified.

[0081] The types of frame-level gait feature sequences include: frame-level gait feature sequences that characterize the detailed information of each frame of gait image, and frame-level gait feature sequences that characterize the context information of the gait image sequence.

[0082] For details, please refer to Figure 3 In the gait recognition method based on quality assessment, when the gait image sequence is a silhouette image sequence, the step of extracting the frame-level gait feature sequence of the gait image sequence and fusing the frame-level gait feature sequence based on the quality score of each frame of gait image to obtain the fused gait features to be recognized includes the following steps S301-S303:

[0083] S301. Extract the target gait feature sequence that represents the detailed information in each gait image sequence, and fuse the target gait feature sequence based on the quality score of each gait image to obtain the first fused gait feature;

[0084] S302. Extract the low-level gait feature sequence of the gait image sequence, and fuse the low-level gait feature sequence based on the quality score of each gait image to obtain the second fused gait feature that represents the context information of the gait image sequence.

[0085] S303. Secondary fusion of the first fused gait features and the second fused gait features yields the fused gait features to be identified.

[0086] In this way, the first fused gait feature can characterize the detailed features of each silhouette image in the silhouette image sequence, that is, it can characterize the gait detail features; the second fused gait feature can characterize the contextual features of the silhouette image sequence, that is, it can characterize the relationship between different gait features. Thus, the gait feature to be identified after the second fusion includes both sufficiently rich detailed features and the relationship between different gait features, thereby making the gait feature to be identified more discriminative and improving the reliability of gait authentication.

[0087] In this embodiment, a silhouette feature extraction model is used to extract frame-level gait feature sequences from the gait image sequence, and the frame-level gait feature sequences are fused based on the quality score of each frame of gait image to obtain the fused gait features to be identified.

[0088] Specifically, the low-level gait feature sequence of the gait image sequence is extracted, and the low-level gait feature sequence is fused based on the quality score of each gait image frame to obtain a second fused gait feature representing the contextual information of the gait image sequence, including:

[0089] The gait image sequence is input into the trained silhouette feature extraction model, and the low-level gait feature sequence of the gait image sequence is extracted through the first part of the convolutional layer of the silhouette feature extraction model;

[0090] Based on the quality score of each gait image, the underlying gait feature sequence is fused to obtain the underlying fused gait features that include gait image sequence context information;

[0091] The second part of the silhouette feature extraction model extracts the contextual information of the underlying fused gait features through the convolutional layer, thus obtaining the second fused gait features that represent the contextual information of the gait image sequence.

[0092] In this embodiment of the application, the silhouette feature extraction model can also simultaneously obtain the first fused gait feature; specifically, multiple convolutional layers of the silhouette feature extraction model extract the target gait feature sequence of the gait image sequence, and fuse the target gait feature sequence based on the quality score of each frame of gait image to obtain the first fused gait feature.

[0093] Here, at least some convolutional layers can be shared when extracting the target gait feature sequence and when extracting the underlying gait feature sequence.

[0094] In this embodiment of the application, the silhouette feature extraction model is the GaitSet model.

[0095] Please refer to Figure 4 , Figure 4 The diagram illustrates the structure of the silhouette feature extraction model. A sequence of silhouette images is input into the model, which comprises two branches: i. The first branch (blue) extracts frame-level features for each silhouette, and in the final stage, uses quality-perceived scores to weightedly fuse the frame-level features to obtain aggregate features; ii. The second branch (yellow) weights and fuses the frame-level features using quality-perceived scores at the lower level to obtain aggregate features, and then uses a CNN for further feature extraction; iii. The aggregate features obtained by the two different methods are mapped through a horizontal pyramid to obtain the final gait features. Here, the first branch performs high-level fusion, which is more conducive to preserving the details of each frame, while the second branch performs low-level fusion, which makes it easier to capture the contextual information in the silhouette image sequence.

[0096] In this embodiment of the application, when the gait image sequence is a keypoint image sequence, the frame-level gait feature sequence of the gait image sequence is extracted, and the frame-level gait feature sequence is fused based on the quality score of each frame of gait image to obtain the fused gait feature to be identified, including:

[0097] The gait image sequence is input into the key point feature extraction model, and the frame-level key point feature sequence of the gait image sequence is extracted by the key point feature extraction model.

[0098] Based on the quality score of each gait image, the features of each frame-level keypoint in the frame-level keypoint feature sequence are weighted, and the weighted frame-level keypoint feature sequence is output.

[0099] The weighted frame-level key point feature sequences are fused together, and the fused gait features to be identified are output.

[0100] In this embodiment of the application, the key point feature extraction model adopts the GaitGraph model.

[0101] Please refer to Figure 5 , Figure 5The diagram illustrates the keypoint feature extraction model structure described in this embodiment. Keypoint image sequences are input into multiple ResGCN modules within the keypoint feature extraction model. Each ResGCN module includes graph convolutional layers, 2D convolutional layers, bottleneck layers, etc., to extract frame-level keypoint feature sequences. Quality scores are used to weight the frame-level keypoint sequence features, outputting a weighted frame-level keypoint feature sequence. Average pooling is performed on the output weighted frame-level keypoint feature sequence, which is then input into a fully connected layer to output the final, fused gait features to be identified.

[0102] In step S102, the gait feature to be identified is searched in a pre-built gait feature library to obtain at least one target gait feature.

[0103] Please refer to Figure 6 Based on the gait features to be identified, a search is performed in a pre-built gait feature database to obtain at least one target gait feature, including the following steps S601-S602:

[0104] S601. Calculate the similarity between the gait feature to be identified and at least some gait features in the pre-built gait feature library;

[0105] S602. Select target gait features from the at least some gait features whose similarity meets the preset similarity conditions.

[0106] In this embodiment of the application, at least some gait features in the gait feature library can be all gait features or some gait features selected based on screening conditions, such as age, gender, city, etc.

[0107] The similarity between the gait feature to be identified and at least some gait features in the pre-built gait feature library is calculated. Specifically, the cosine similarity between the gait feature to be identified and at least some gait features is calculated by matrix multiplication.

[0108] The preset similarity condition can be a preset similarity threshold or a preset number of gait features ranked first by similarity.

[0109] For example, a preset similarity threshold of 0.7 is used to filter out target gait features with a similarity greater than or equal to 0.7.

[0110] In this embodiment, target gait features that meet the preset similarity criteria are selected and arranged in descending order of similarity, so that subsequent verification can be performed based on the similarity.

[0111] Based on the same inventive concept, this application also provides a gait recognition device based on quality assessment, which corresponds to the gait recognition method based on quality assessment. Since the principle of the device in this application is similar to the gait recognition method based on quality assessment described above, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0112] Please refer to Figure 7 , Figure 7 This illustration shows a structural diagram of the gait recognition device based on quality assessment according to an embodiment of this application. Specifically, the device includes:

[0113] The determination module 701 is used to determine the quality score of a gait image for a multi-frame gait image sequence based on the features of multiple categories of quality attributes in each frame of gait image;

[0114] The fusion module 702 is used to extract the frame-level gait feature sequence of the gait image sequence, and fuse the frame-level gait feature sequence based on the quality score of each frame of gait image to obtain the fused gait feature to be identified.

[0115] The retrieval module 703 is used to search a pre-built gait feature library based on the gait feature to be identified, to obtain at least one target gait feature, and to determine the gait recognition result of multiple frames of gait images based on the at least one target gait feature.

[0116] The gait recognition device based on quality assessment described in this application, for a multi-frame gait image sequence, determines the quality score of each gait image frame based on the features of multiple categories of quality attributes in each frame; extracts frame-level gait feature sequences from the gait image sequence, and fuses the frame-level gait feature sequences based on the quality scores of each gait image frame to obtain fused gait features to be recognized; searches a pre-built gait feature library based on the gait features to be recognized to obtain at least one target gait feature, and determines the gait recognition result of the multi-frame gait images based on the at least one target gait feature; thus, quality perception is integrated into gait sequence processing and gait feature extraction, reducing the impact of low-quality gait image frames on gait features, thereby obtaining more discriminative gait features and improving the accuracy of gait verification.

[0117] In the gait recognition device based on quality assessment described in this application embodiment, the quality attributes include: the clarity of the gait image, and the degree of occlusion of the silhouette and / or key points in the gait image.

[0118] The gait recognition device based on quality assessment described in this application embodiment further includes:

[0119] The filtering module is used to filter out gait images whose quality scores meet preset quality conditions after determining the quality score of the gait image frame, and update the gait image sequence.

[0120] In the gait recognition device based on quality assessment described in this application embodiment, when the gait image sequence is a silhouette image sequence, the fusion module, in extracting the frame-level gait feature sequence of the gait image sequence and fusing the frame-level gait feature sequence based on the quality score of each frame gait image to obtain the fused gait features to be recognized, is specifically used for:

[0121] Extract the target gait feature sequence representing the detailed information in each gait image sequence, and fuse the target gait feature sequence based on the quality score of each gait image to obtain the first fused gait feature;

[0122] The low-level gait feature sequence of the gait image sequence is extracted, and the low-level gait feature sequence is fused based on the quality score of each frame of gait image to obtain the second fused gait feature that represents the context information of the gait image sequence.

[0123] The first and second gait features are fused together to obtain the fused gait features to be identified.

[0124] In the gait recognition device based on quality assessment described in this application embodiment, the fusion module extracts the low-level gait feature sequence of the gait image sequence and fuses the low-level gait feature sequence based on the quality score of each frame of gait image to obtain a second fused gait feature representing the contextual information of the gait image sequence, including:

[0125] The gait image sequence is input into the trained silhouette feature extraction model, and the low-level gait feature sequence of the gait image sequence is extracted through the first part of the convolutional layer of the silhouette feature extraction model;

[0126] Based on the quality score of each gait image, the underlying gait feature sequence is fused to obtain the underlying fused gait features that include gait image sequence context information;

[0127] The second part of the silhouette feature extraction model extracts the contextual information of the underlying fused gait features through the convolutional layer, thus obtaining the second fused gait features that represent the contextual information of the gait image sequence.

[0128] In the gait recognition device based on quality assessment described in this application embodiment, when the gait image sequence is a keypoint image sequence, the fusion module, in extracting the frame-level gait feature sequence of the gait image sequence and fusing the frame-level gait feature sequence based on the quality score of each frame gait image to obtain the fused gait features to be recognized, is specifically used for:

[0129] The gait image sequence is input into the key point feature extraction model, and the frame-level key point feature sequence of the gait image sequence is extracted by the key point feature extraction model.

[0130] Based on the quality score of each gait image, the features of each frame-level keypoint in the frame-level keypoint feature sequence are weighted, and the weighted frame-level keypoint feature sequence is output.

[0131] The weighted frame-level key point feature sequences are fused together, and the fused gait features to be identified are output.

[0132] In the gait recognition device based on quality assessment described in this application embodiment, when the retrieval module searches a pre-built gait feature database based on the gait feature to be recognized and obtains at least one target gait feature, it is specifically used for:

[0133] Calculate the similarity between the gait feature to be identified and at least some gait features in a pre-built gait feature library;

[0134] Target gait features whose similarity meets preset similarity conditions are selected from the at least some gait features.

[0135] Based on the same inventive concept, this application also provides an electronic device corresponding to the gait recognition method based on quality assessment. Since the principle of solving the problem by the electronic device in this application is similar to the gait recognition method based on quality assessment described above in this application, the implementation of the electronic device can refer to the implementation of the method, and the repeated parts will not be described again.

[0136] Please refer to Figure 8 , Figure 8 The diagram shows the structure of the electronic device according to an embodiment of this application. Specifically, the electronic device 800 includes a processor 802, a memory 801, and a bus. The memory 801 stores machine-readable instructions executable by the processor 802. When the electronic device 800 is running, the processor 802 communicates with the memory 801 via the bus. When the machine-readable instructions are executed by the processor 802, the steps of the gait recognition method based on quality assessment are performed.

[0137] Based on the same inventive concept, this application also provides a computer-readable storage medium corresponding to the gait recognition method based on quality assessment. Since the principle of the computer-readable storage medium in this application is similar to the gait recognition method based on quality assessment described above in this application, the implementation of the computer-readable storage medium can refer to the implementation of the method, and the repeated parts will not be described again.

[0138] A computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the gait recognition method based on quality assessment.

[0139] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.

[0140] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0141] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0142] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a platform server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0143] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A gait recognition method based on quality assessment, characterized in that, The method includes: For a multi-frame gait image sequence, a quality score for each gait image is determined based on the features of multiple quality attributes in each frame. A frame-level gait feature sequence is extracted from a gait image sequence, and the frame-level gait feature sequence is fused based on the quality score of each frame of gait image to obtain the fused gait feature to be identified. When the gait image sequence is a silhouette image sequence, the process of extracting and fusing the frame-level gait feature sequence includes: extracting a target gait feature sequence representing the detailed information in each silhouette image from the silhouette image sequence; fusing the target gait feature sequence based on the quality score of each frame of gait image to obtain a first fused gait feature; inputting the silhouette image sequence into a trained silhouette feature extraction model; extracting the low-level gait feature sequence from the first part of the convolutional layer of the silhouette feature extraction model; fusing the low-level gait feature sequence based on the quality score of each frame of gait image to obtain a low-level fused gait feature including the context information of the silhouette image sequence; extracting the context information of the low-level fused gait feature from the second part of the convolutional layer of the silhouette feature extraction model to obtain a second fused gait feature; and fusing the first fused gait feature and the second fused gait feature a second time to obtain the fused gait feature to be identified. Based on the gait features to be identified, a gait feature database is searched to obtain at least one target gait feature, and the gait recognition results of multiple frames of gait images are determined based on the at least one target gait feature.

2. The gait recognition method based on quality assessment according to claim 1, characterized in that, The quality attributes include: the sharpness of the gait image, and the degree of occlusion of the silhouette and / or key points in the gait image.

3. The gait recognition method based on quality assessment according to claim 1, characterized in that, After determining the quality score of the gait image frame, the method further includes: Select gait images whose quality scores meet the preset quality conditions and update the gait image sequence.

4. The gait recognition method based on quality assessment according to claim 1, characterized in that, Based on the gait features to be identified, a search is performed in a pre-built gait feature database to obtain at least one target gait feature, including: Calculate the similarity between the gait feature to be identified and at least some gait features in a pre-built gait feature library; Target gait features whose similarity meets preset similarity conditions are selected from the at least some gait features.

5. A gait recognition device based on quality assessment, characterized in that, The device includes: The determination module is used to determine the quality score of each gait image for a multi-frame gait image sequence based on the features of multiple categories of quality attributes in each gait image; A fusion module is used to extract frame-level gait feature sequences from a gait image sequence and fuse these sequences based on the quality score of each frame to obtain fused gait features to be identified. Specifically, when the gait image sequence is a silhouette image sequence, the process of extracting and fusing the frame-level gait feature sequences includes: extracting target gait feature sequences representing detailed information in each silhouette image from the silhouette image sequence; fusing the target gait feature sequences based on the quality score of each frame to obtain a first fused gait feature; inputting the silhouette image sequence into a trained silhouette feature extraction model; extracting the low-level gait feature sequences from the silhouette image sequence through the first part of the convolutional layer of the silhouette feature extraction model; fusing the low-level gait feature sequences based on the quality score of each frame to obtain a low-level fused gait feature including context information of the silhouette image sequence; extracting the context information of the low-level fused gait feature through the second part of the convolutional layer of the silhouette feature extraction model to obtain a second fused gait feature; and fusing the first fused gait feature and the second fused gait feature a second time to obtain the fused gait features to be identified. The retrieval module is used to search a pre-built gait feature library based on the gait features to be identified, to obtain at least one target gait feature, and to determine the gait recognition result of multiple frames of gait images based on the at least one target gait feature.

6. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the gait recognition method based on quality assessment as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the gait recognition method based on quality assessment as described in any one of claims 1 to 4.

Images