Gait recognition method and apparatus

Abstract

Embodiments of the present disclosure disclose a gait recognition method and device. A specific implementation of the method comprises: acquiring a first gait image sequence and a second gait image sequence; performing three-dimensional modeling on a human body in each gait image in the first gait image sequence and the second gait image sequence to obtain a first gait model sequence and a second gait model sequence; performing view alignment on the first gait model sequence and the second gait model sequence; and performing gait recognition according to the aligned first gait model sequence and the second gait model sequence. This implementation helps to improve the accuracy of the gait recognition result.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and more particularly to a gait recognition method and apparatus. Background Art

[0002] Person re-identification, also known as pedestrian re-identification, uses computer vision technology to determine whether a specific pedestrian exists in an image or video sequence. Given an image or video of a specific pedestrian, the system searches for the specified pedestrian in images or videos across multiple devices to overcome the visual limitations of fixed cameras. Person re-identification technology can be combined with pedestrian detection or tracking technology and can be applied to fields such as intelligent video surveillance and intelligent security. However, differences between different camera devices, as well as pedestrians' appearance, clothing, scale, occlusion, posture, and viewing angle, significantly affect the accuracy of pedestrian re-identification results.

[0003] Gait recognition is an emerging biometric technology designed to identify pedestrians based on their gait. Compared to other biometric technologies, gait recognition offers the advantages of being contactless, long-range, and difficult to camouflage. In the field of intelligent video surveillance, it offers advantages over image recognition. Existing gait recognition methods primarily include model-based and shape-based. Model-based gait recognition primarily models human limb motion patterns to characterize gait parameters (such as motion trajectory, limb length, and limb bending angle), then leverages differences in motion patterns to distinguish different pedestrians. Shape-based gait recognition primarily extracts spatial and temporal features from human gait profile sequences and then calculates the similarity between these features to achieve gait matching and recognition. However, the human body is a three-dimensional object, and gait profile sequences captured from different perspectives exhibit significant differences. These intra- and inter-class feature variations caused by perspective differences significantly impact the accuracy of gait recognition results. Summary of the Invention

[0004] The embodiments of the present disclosure provide a gait recognition method and apparatus.

[0005] In a first aspect, an embodiment of the present disclosure provides a gait recognition method, which includes: acquiring a first gait image sequence and a second gait image sequence; performing three-dimensional modeling of a human body in each gait image in the first gait image sequence and the second gait image sequence to obtain a first gait model sequence and a second gait model sequence; performing perspective alignment on the first gait model sequence and the second gait model sequence; and performing gait recognition based on the aligned first gait model sequence and the second gait model sequence.

[0006] In a second aspect, an embodiment of the present disclosure provides a gait recognition device, which includes: an acquisition unit, configured to acquire a first gait image sequence and a second gait image sequence; a modeling unit, configured to perform three-dimensional modeling of a human body in each gait image in the first gait image sequence and the second gait image sequence, to obtain a first gait model sequence and a second gait model sequence; an alignment unit, configured to perform perspective alignment on the first gait model sequence and the second gait model sequence; and a recognition unit, configured to perform gait recognition based on the aligned first gait model sequence and the second gait model sequence.

[0007] In a third aspect, an embodiment of the present disclosure provides an electronic device comprising: one or more processors; a storage device for storing one or more programs; and when the one or more programs are executed by one or more processors, the one or more processors implement the method described in any implementation manner in the first aspect.

[0008] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the method described in any implementation manner in the first aspect.

[0009] The gait recognition method and device provided by the embodiments of the present disclosure perform three-dimensional modeling on two matching gait image sequences to obtain two corresponding gait model sequences. The two gait model sequences are then aligned from different perspectives before gait recognition is performed based on the aligned gait model sequences. This method and device can achieve gait matching and recognition from the same perspective, avoiding interference caused by perspective differences in gait matching and recognition, thereby helping to improve the accuracy of gait recognition results. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Other features, objects and advantages of the present disclosure will become more apparent from a reading of the detailed description of non-limiting embodiments made with reference to the following drawings:

[0011] Figure 1 is an exemplary system architecture diagram in which an embodiment of the present disclosure may be applied;

[0012] Figure 2 is a flow chart of an embodiment of a gait recognition method according to the present disclosure;

[0013] Figure 3a is a schematic diagram of a gait image according to the present disclosure;

[0014] Figure 3b is a schematic diagram of a gait model according to the present disclosure;

[0015] Figure 4is a flowchart of another embodiment of a gait recognition method according to the present disclosure;

[0016] Figure 5 is a flowchart of another embodiment of the gait recognition method according to the present disclosure;

[0017] Figure 6 is a flowchart of another embodiment of a gait recognition method according to the present disclosure;

[0018] Figure 7 is a schematic diagram of an application scenario of the gait recognition method according to an embodiment of the present disclosure;

[0019] Figure 8 is a structural diagram of an embodiment of a gait recognition device according to the present disclosure;

[0020] Figure 9 It is a schematic structural diagram of an electronic device suitable for implementing the embodiments of the present disclosure. DETAILED DESCRIPTION

[0021] The present disclosure will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are intended only to illustrate the relevant invention and are not intended to limit the invention. It should also be noted that, for ease of description, only portions relevant to the relevant invention are shown in the accompanying drawings.

[0022] It should be noted that the data acquisition involved in this disclosure (such as user images or videos, etc.) is carried out on the basis of having obtained the authorization of the relevant subjects and complies with the provisions of relevant laws and regulations.

[0023] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0024] Figure 1 An exemplary architecture 100 is shown to which an embodiment of the gait recognition method or gait recognition apparatus of the present disclosure may be applied.

[0025] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. Network 104 is a medium for providing communication links between terminal devices 101, 102, 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0026] Terminal devices 101, 102, and 103 interact with server 105 via network 104 to receive or send messages, etc. Various client applications may be installed on terminal devices 101, 102, and 103, such as browser applications, search applications, instant messaging tools, shopping applications, multimedia processing applications, and the like.

[0027] Terminal devices 101, 102, 103 can be hardware or software. When terminal devices 101, 102, 103 are hardware, they can be various electronic devices, including but not limited to smart phones, tablet computers, e-book readers, smart camera devices (such as cameras, video recorders, etc.), laptop computers and desktop computers, etc. When terminal devices 101, 102, 103 are software, they can be installed in the electronic devices listed above. They can be implemented as multiple software or software modules (for example, multiple software or software modules for providing distributed services), or they can be implemented as a single software or software module. No specific limitation is made here.

[0028] The server 105 may be a server providing various services, such as a backend server providing various services to the terminal devices 101, 102, and 103. The server 105 may process the first gait image sequence and the second gait image sequence according to the request of the terminal devices 101, 102, and 103 to obtain a gait recognition result.

[0029] It should be noted that the gait recognition method provided in the embodiments of the present disclosure is generally executed by the server 105 , and accordingly, the gait recognition device is generally provided in the server 105 .

[0030] It should also be noted that the terminal devices 101, 102, and 103 may also be installed with multimedia processing applications. The terminal devices 101, 102, and 103 may also process the first and second gait image sequences based on the multimedia processing applications to obtain gait recognition results. In this case, the gait recognition method may also be executed by the terminal devices 101, 102, and 103, and accordingly, the gait recognition device may also be installed in the terminal devices 101, 102, and 103. In this case, the exemplary system architecture 100 may not include the server 105 and the network 104.

[0031] It should be noted that the server 105 can be hardware or software. When the server 105 is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or it can be implemented as a single server. When the server 105 is software, it can be implemented as multiple software or software modules (for example, multiple software or software modules for providing distributed services), or it can be implemented as a single software or software module. No specific limitations are given here.

[0032] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0033] Continue to refer Figure 2 , which shows a process 200 of an embodiment of a gait recognition method according to the present disclosure. The gait recognition method includes the following steps:

[0034] Step 201: Acquire a first gait image sequence and a second gait image sequence.

[0035] In this embodiment, a gait image may refer to an image showing a person's walking posture. A gait image sequence may be composed of a plurality of gait images. Generally, each gait image in a gait image sequence is a gait image of the same person.

[0036] It should be noted that the gait image can be of various types. For example, a video frame can be selected from a walking video of a designated person as the gait image, or an image of the selected video frame after processing (such as sharpening, binarization, etc.) can be used as the gait image.

[0037] The first gait image sequence and the second gait image sequence can be any gait image sequence. It should be noted that, for the convenience of describing the two gait image sequences to be matched, the two gait image sequences are named as the first gait image sequence and the second gait image sequence, respectively. Those skilled in the art should understand that the first and second do not constitute a special limitation on the gait image sequence.

[0038] The execution subject of the gait recognition method (such as Figure 1 Server 105 shown in the figure) can be accessed from a local or other electronic device (such as Figure 1 The terminal devices 101, 102, 103, etc. shown in the figure obtain a first gait image sequence and a second gait image sequence.

[0039] Step 202 : Perform three-dimensional modeling on the human body in each gait image in the first gait image sequence and the second gait image sequence to obtain the first gait model sequence and the second gait model sequence.

[0040] In this embodiment, the human body in each gait image in the first gait image sequence can be three-dimensionally modeled to obtain a gait model corresponding to each gait image, and then a first gait model sequence composed of the gait models corresponding to each gait image in the first gait image sequence can be obtained.

[0041] Similarly, three-dimensional modeling can be performed on the human body in each gait image in the second gait image sequence to obtain a gait model corresponding to each gait image, and then a second gait model sequence composed of the gait models corresponding to each gait image in the second gait image sequence can be obtained.

[0042] See below Figure 3a and Figure 3b , Figure 3a is a schematic diagram of a gait image in this embodiment, Figure 3b is a schematic diagram of the gait model in this embodiment.

[0043] Specifically, for a gait image, various existing 3D modeling methods can be used to perform corresponding 3D modeling of the human body in the gait image. For example, 3D modeling methods include, but are not limited to, polygon modeling, parametric modeling, reverse modeling, NURBS modeling, and the like. As an example, existing 3D mesh reconstruction methods can be used for 3D modeling.

[0044] The 3D modeling process can be described using the following formula:

[0045] R(x k )=ROMP(x k )

[0046] Among them, X k represents the kth gait image in the gait image sequence, ROMP() represents the process of three-dimensional modeling, and R() represents the gait model obtained after three-dimensional modeling.

[0047] Step 203: perform perspective alignment on the first gait model sequence and the second gait model sequence.

[0048] In this embodiment, the perspective of the gait model sequence may refer to the shooting perspective corresponding to each gait model in the gait model sequence. The perspective of the gait model sequence may be flexibly determined according to the perspective corresponding to each gait model in the gait model sequence.

[0049] For example, the average of the viewing angles corresponding to the gait models in the gait model sequence can be determined as the viewing angle of the gait model sequence. For another example, the viewing angle that appears most frequently among the viewing angles corresponding to the gait models in the gait model sequence can be determined as the viewing angle of the gait model sequence. For another example, a viewing angle sequence composed of the viewing angles corresponding to the gait models in the gait model sequence can be determined as the viewing angle of the gait model sequence.

[0050] The perspective of the first gait model sequence is aligned with the perspective of the second gait model sequence, so that the perspectives corresponding to the aligned first gait model sequence and the second gait model sequence respectively match.

[0051] For example, the perspective corresponding to the first gait model sequence is the front shooting perspective, and the perspective corresponding to the second gait model sequence is the side shooting perspective. The side shooting perspective corresponding to the second gait model sequence can be adjusted to the front shooting perspective to complete the alignment of the first gait model sequence and the second gait model sequence. The front shooting perspective corresponding to the first gait model sequence can also be adjusted to the side shooting perspective to complete the alignment of the first gait model sequence and the second gait model sequence. The front shooting perspective corresponding to the first gait model sequence and the side shooting perspective corresponding to the second gait model sequence can also be adjusted to the same specified shooting perspective (such as the back shooting perspective, etc.) to complete the alignment of the first gait model sequence and the second gait model sequence.

[0052] If the perspectives corresponding to the first gait model sequence and the second gait model sequence include perspective sequences, various methods can be flexibly used to align the perspectives corresponding to the first gait model sequence and the second gait model sequence.

[0053] For example, if the perspective corresponding to the second gait model sequence is a perspective sequence composed of the perspectives corresponding to each gait model therein, the perspectives of the gait models corresponding to each gait model in the second gait model can be adjusted to the perspectives corresponding to the first gait model sequence, thereby completing the perspective alignment of the first gait model sequence and the second gait model sequence.

[0054] For another example, the perspective corresponding to the first gait model sequence is a perspective sequence composed of the perspectives corresponding to each gait model therein, and the perspective corresponding to the second gait model sequence is a perspective sequence composed of the perspectives corresponding to each gait model therein. Then, the perspective of each gait model in the first gait model sequence and the perspective of each gait model in the second gait model sequence can be adjusted to the same specified perspective to complete the perspective alignment of the first gait model sequence and the second gait model sequence.

[0055] Specifically, various existing perspective alignment methods can be used to align the perspectives of the first gait model sequence and the second gait model sequence. For example, the perspectives of the first gait model sequence and the second gait model sequence can be aligned by rotating the gait model.

[0056] Step 204: Perform gait recognition based on the aligned first gait model sequence and the second gait model sequence.

[0057] In this embodiment, after viewpoint alignment, the resulting aligned first gait model sequence and the aligned second gait model sequence correspond to the same viewpoint. Gait recognition can then be performed based on the aligned first and second gait model sequences from the same viewpoint to determine whether the person corresponding to the first and second gait model sequences is the same person, as a gait recognition result.

[0058] Specifically, after obtaining the aligned first gait model sequence and the aligned second gait model sequence, gait recognition can be performed using various existing gait recognition methods to obtain gait recognition results. The gait recognition results can be used to indicate whether the person in each gait image in the first gait image sequence is the same as the person in each gait image in the second gait image sequence.

[0059] In some optional implementations of this embodiment, after obtaining the aligned first gait model sequence and the second gait model sequence, gait features of the aligned first gait model sequence can be extracted to obtain the first gait features, and gait features of the aligned second gait model sequence can be extracted to obtain the second gait features. Subsequently, the similarity between the first gait features and the second gait features can be determined, and a gait recognition result can be determined based on the determined similarity.

[0060] The gait features can be obtained by flexibly applying various existing gait feature extraction methods to process the gait model sequence. For example, the gait features can be extracted based on the temporal sequence of the gait model sequence, or based on the physiological characteristics of the human body in the gait model.

[0061] Generally, if the similarity between the first gait feature and the second gait feature is greater than a preset similarity threshold, it can be assumed that the first gait image sequence and the second gait image sequence correspond to the same person. Correspondingly, if the similarity between the first gait feature and the second gait feature is not greater than the preset similarity threshold, it can be assumed that the first gait image sequence and the second gait image sequence correspond to different people.

[0062] The method provided by the above-mentioned embodiments of the present disclosure performs three-dimensional modeling on the two gait image sequences to be recognized, and then aligns the perspectives of the two corresponding gait model sequences, thereby converting gait recognition under different perspectives into gait recognition under the same perspective, avoiding the influence of perspective differences on gait recognition, and improving the accuracy of gait recognition results under different perspectives.

[0063] Further references Figure 4 , which shows a process 400 of another embodiment of a gait recognition method. The process 400 of the gait recognition method includes the following steps:

[0064] Step 401: Acquire a first gait image sequence and a second gait image sequence.

[0065] Step 402 : Perform three-dimensional modeling of the human body in each gait image in the first gait image sequence and the second gait image sequence to obtain the first gait model sequence and the second gait model sequence.

[0066] Step 403: perform perspective alignment on the first gait model sequence and the second gait model sequence.

[0067] Step 404 : convert each gait model in the aligned first gait model sequence into two dimensions to obtain a third gait image sequence.

[0068] In this embodiment, two-dimensionalization may refer to a process of converting an object larger than two dimensions into two dimensions. Specifically, various existing two-dimensionalization methods (such as two-dimensionalization functions provided by various image processing tools, binarization processing, etc.) may be used to two-dimensionalize the gait model.

[0069] For each gait model in the first gait model sequence after perspective alignment, the gait model can be two-dimensionalized to obtain a two-dimensional gait image corresponding to the gait model. Based on this, a third gait image sequence consisting of the two-dimensional gait images corresponding to each gait model in the first aligned gait model sequence is obtained.

[0070] Step 405 : convert each gait model in the aligned second gait model sequence into two dimensions to obtain a fourth gait image sequence.

[0071] In this embodiment, for each gait model in the second gait model sequence after perspective alignment, the gait model can be two-dimensionalized to obtain a two-dimensional gait image corresponding to the gait model. Based on this, a fourth gait image sequence consisting of the two-dimensional gait images corresponding to each gait model in the aligned second gait model sequence is obtained.

[0072] Step 406: Perform gait recognition based on the third gait image sequence and the fourth gait image sequence.

[0073] In this embodiment, various gait recognition methods may be used to perform gait recognition based on the third gait image sequence and the fourth gait image sequence obtained based on the two-dimensionalization to obtain a gait recognition result.

[0074] In some optional implementations of this embodiment, a pre-trained gait feature extraction network can be used to extract gait features corresponding to the third gait image sequence and the fourth gait image sequence, respectively. Then, the similarity between the gait features of the third gait image sequence and the gait features of the fourth gait image sequence is calculated, and the gait recognition result is determined based on the calculated similarity.

[0075] The gait feature extraction network can be trained using preset training data and loss functions to obtain an initial model. The training data can include gait image sequences, each labeled to indicate the corresponding person. The initial model can be flexibly constructed by technical personnel based on actual needs or based on various existing neural network models.

[0076] For example, the initial model can be built based on a deep learning model such as GaitSet. In this case, the gait image sequence input to the gait feature extraction network can be a human silhouette image sequence. The human silhouette image sequence can be obtained by performing binarization or other processing on the human image.

[0077] The loss function can be pre-set by technicians based on the actual application. For example, the loss function L can be set as follows:

[0078]

[0079] Where N represents the number of input training samples, and i corresponds to the number of input training samples. F() represents the feature extraction process of the feature extraction network, which is the gait feature output by the feature extraction network. a represents the gait image sequence corresponding to any pedestrian, X p Indicates that X a Corresponding to other gait image sequences of the same pedestrian, X n Indicates that X a Gait image sequences corresponding to different people. M represents the margin parameter. + Indicates that the value is non-negative, that is, if the result of [] is negative, L is set to 0.

[0080] The similarity between gait features can be determined based on various similarity calculation methods (such as Euclidean distance, cosine similarity, etc.). If the calculated similarity is greater than a preset similarity threshold, it can be assumed that the third gait image sequence and the fourth gait image sequence correspond to the same person, that is, the first gait image sequence and the second gait image sequence correspond to the same person. Correspondingly, if the calculated similarity is not greater than the preset similarity threshold, it can be assumed that the third gait image sequence and the fourth gait image sequence correspond to different people, that is, the first gait image sequence and the second gait image sequence correspond to different people.

[0081] The details not described in this embodiment can be found in Figure 2 The relevant descriptions in the corresponding embodiments will not be repeated here.

[0082] The method provided in the above-mentioned embodiments of the present disclosure, after obtaining two gait model sequences using 3D modeling, first converts the gait model sequences into 2D gait image sequences through 2D conversion, and then performs gait recognition based on the 2D gait image sequences. Compared with 2D perspective transformation methods such as Spatial Transformer Networks (STN), perspective transformation based on 3D models followed by 2D conversion can preserve gait information as much as possible, thereby improving gait recognition accuracy.

[0083] Further references Figure 5 , which shows a process 500 of another embodiment of a gait recognition method. The process 500 of the gait recognition method includes the following steps:

[0084] Step 501: Acquire a first gait image sequence and a second gait image sequence.

[0085] Step 502 : Perform three-dimensional modeling of the human body in each gait image in the first gait image sequence and the second gait image sequence to obtain the first gait model sequence and the second gait model sequence.

[0086] Step 503: Determine the average viewing angle corresponding to each gait model in the first gait model sequence.

[0087] In this embodiment, the average value of the viewing angles corresponding to the gait models in the first gait model sequence may be determined as the average viewing angle.

[0088] Step 504 : Adjust the viewing angle of each gait model in the second gait model sequence to align with the average viewing angle.

[0089] In this embodiment, for each gait model in the second gait model sequence, the viewing angle of the gait model may be adjusted to be aligned with the average viewing angle corresponding to the first gait model sequence.

[0090] Step 505: Perform gait recognition based on the aligned first gait model sequence and the second gait model sequence.

[0091] The details not described in this embodiment can be found in Figure 2 and Figure 4 The relevant descriptions in the corresponding embodiments will not be repeated here.

[0092] The method provided by the above-mentioned embodiment of the present disclosure can determine the average viewing angle of each gait model in one gait model sequence when performing perspective alignment of a gait model sequence, and then adjust the viewing angle of each gait model in another gait model sequence to the average viewing angle. This not only reduces the number of gait models that require perspective adjustment, thereby reducing the amount of calculation, but also preserves the gait information of people corresponding to the gait model sequences that do not require perspective adjustment as much as possible, which helps to improve the accuracy of subsequent gait matching and recognition.

[0093] Further references Figure 6 , which shows a process 600 of another embodiment of a gait recognition method. The process 600 of the gait recognition method includes the following steps:

[0094] Step 601: Acquire a first posture image sequence of a target person.

[0095] In this embodiment, the target person can be any person, and can be pre-specified by a technician according to an actual application scenario. The first gait image sequence of the target person can be composed of gait images showing the walking posture of the target person.

[0096] Step 602: Acquire a target video, and extract gait image sequences of each person in the target video to obtain at least one second gait image sequence.

[0097] In this embodiment, the target video may be any video and may be flexibly set according to actual application scenarios or application requirements.

[0098] The target video may include one or more individuals. In this case, for each person in the target video, a gait image sequence of that person can be extracted to form a second gait image sequence. Based on this, a second gait image sequence corresponding to each person in the target video can be obtained.

[0099] Step 603 : Perform three-dimensional modeling on the human body in each gait image in the first gait image sequence to obtain the first gait model sequence.

[0100] Step 604 : For each second gait image sequence in the at least one second gait image sequence, perform three-dimensional modeling of the human body in each gait image in the second gait image sequence to obtain a second gait model sequence corresponding to the second gait image sequence.

[0101] Step 605: Perform perspective alignment on the obtained at least one second gait model sequence and the first gait model sequence.

[0102] As an example, the view alignment process can be described using the following formula:

[0103]

[0104] Where P() represents the perspective alignment operation of the two objects. g represents the second gait model sequence, and q represents the first gait model sequence. X g i,k θ represents the kth gait model in the i-th second gait model sequence. g i,k θ represents the viewing angle of the kth gait model in the i-th second gait model sequence. q k represents the view angle of the kth gait model in the first step model sequence. K1 represents the length of the first step model sequence. R() represents the rotation operation of the view angle. | () represents the second gait model sequence after view alignment.

[0105] The length of the first gait model sequence and the length of each second gait model sequence may be the same or different.

[0106] Step 606 : extracting gait features of the aligned first gait model sequence, and extracting gait features corresponding to at least one aligned second gait model sequence.

[0107] In this embodiment, each aligned second gait model sequence may be first converted into two-dimensional form to obtain corresponding two-dimensional gait image sequences, and then corresponding gait features may be extracted based on each obtained two-dimensional gait image sequence.

[0108] As an example, the process of two-dimensionalization through binarization and rendering operations can be described by the following formula:

[0109]

[0110] Among them, X g i,k R represents the kth gait model in the i-th second gait model sequence. | () represents the second gait model sequence after view alignment. X() represents the renderer. B() represents the image binarization process. G | () represents the two-dimensional image obtained after two-dimensional processing.

[0111] Step 607: For each aligned second gait model sequence in at least one aligned second gait model sequence, determine the similarity between the gait features of the aligned second gait model sequence and the gait features of the aligned first gait model sequence, and obtain the similarity corresponding to the aligned second gait model sequence.

[0112] Step 608 : Determine a gait recognition result of the target video according to the similarities corresponding to the aligned second gait model sequences in the at least one aligned second gait model sequence.

[0113] In this embodiment, after determining the similarity between the gait features of each aligned second gait model sequence and the gait features of the aligned first gait model sequence, various methods can be flexibly employed to determine the gait recognition result for the target video. The gait recognition result for the target video can be used to indicate whether the target video includes the target person.

[0114] For example, the maximum similarity value may be selected first, and then it may be determined whether the maximum similarity value is greater than a preset similarity threshold. If so, the person corresponding to the aligned second gait model sequence corresponding to the maximum similarity value may be considered to be the target person.

[0115] The details not described in this embodiment can be found in Figure 2 、 Figure 4 and Figure 5 The relevant descriptions in the corresponding embodiments will not be repeated here.

[0116] Continue to see Figure 7 , Figure 7 This is an exemplary application scenario 700 of the gait recognition method according to this embodiment. Figure 7 In the application scenario, the execution subject can obtain the video 702 captured by the camera 701, and extract the gait image sequence 7031 of the pedestrian "A" and the gait image sequence 7032 of the pedestrian "B" from the video 702.

[0117] Then, 3D modeling can be performed on each gait image in the gait image sequence 7031 of pedestrian "A" to obtain a corresponding gait model sequence 7041. Simultaneously, 3D modeling can be performed on each gait image in the gait image sequence 7032 of pedestrian "B" to obtain a corresponding gait model sequence 7042.

[0118] In addition, the execution entity can pre-acquire a gait image sequence 705 of the target person "C" and perform three-dimensional modeling on each gait image to obtain a corresponding gait model sequence 706. The perspective of gait model sequence 7041 can then be adjusted to that of gait model sequence 706 to obtain an aligned gait model sequence 7071, and the perspective of gait model sequence 7042 can be adjusted to that of gait model sequence 706 to obtain an aligned gait model sequence 7072, thereby completing perspective alignment.

[0119] Then, each gait model in the aligned gait model sequence 7071 can be converted into two dimensions to obtain a corresponding two-dimensional gait image sequence 7081, and the pre-trained gait feature extraction network 709 can be used to extract gait features 7001 from the two-dimensional gait image sequence 7081. Simultaneously, each gait model in the aligned gait model sequence 7072 can be converted into two dimensions to obtain a corresponding two-dimensional gait image sequence 7082, and the gait feature extraction network 709 can be used to extract gait features 7002 from the two-dimensional gait image sequence 7082.

[0120] At the same time, the gait feature extraction network 709 is used to extract the gait features 710 of the gait image sequence 705, and then the similarity 711 between the gait features 7001 of pedestrian "A" and the gait features of the target person "C" is calculated to be 95%, and the similarity 712 between the gait features 7002 of pedestrian "B" and the gait features of the target person "C" is calculated to be 60%. Based on this, a gait recognition result 713 can be obtained to indicate that the pedestrian "A" in the video 702 is the target person "C".

[0121] The method provided by the above-mentioned embodiments of the present disclosure specifies a gait image sequence of a target person, and simultaneously extracts the gait image sequences of each pedestrian in the target video. Then, the gait model sequences corresponding to the gait image sequences of each pedestrian in the target video are respectively aligned with the gait model sequences corresponding to the gait image sequence of the target person, and gait matching is performed, so as to determine whether the target video includes the specified target person, thereby realizing accurate pedestrian recognition across perspectives.

[0122] Further references Figure 8 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a gait recognition device, which is similar to Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0123] like Figure 8 As shown, the gait recognition device 800 provided in this embodiment includes an acquisition unit 801, a modeling unit 802, an alignment unit 803, and a recognition unit 804. The acquisition unit 801 is configured to acquire a first gait image sequence and a second gait image sequence; the modeling unit 802 is configured to perform three-dimensional modeling of the human body in each gait image in the first gait image sequence and the second gait image sequence to obtain a first gait model sequence and a second gait model sequence; the alignment unit 803 is configured to perform perspective alignment on the first gait model sequence and the second gait model sequence; and the recognition unit 804 is configured to perform gait recognition based on the aligned first gait model sequence and the second gait model sequence.

[0124] In this embodiment, the specific processing of the acquisition unit 801, the modeling unit 802, the alignment unit 803 and the recognition unit 804 and the technical effects thereof can be referred to in the respective embodiments. Figure 2 The relevant descriptions of step 201, step 202, step 203 and step 204 in the corresponding embodiment are not repeated here.

[0125] In some optional implementations of this embodiment, the above-mentioned recognition unit 804 is further configured to: extract the gait features corresponding to the aligned first gait model sequence and the second gait model sequence respectively to obtain the first gait feature and the second gait feature; determine the similarity between the first gait feature and the second gait feature; and determine the gait recognition result based on the similarity.

[0126] In some optional implementations of this embodiment, the above-mentioned recognition unit 804 is further configured to: two-dimensionalize each gait model in the aligned first gait model sequence to obtain a third gait image sequence; two-dimensionalize each gait model in the aligned second gait model sequence to obtain a fourth gait image sequence; and perform gait recognition based on the third gait image sequence and the fourth gait image sequence.

[0127] In some optional implementations of this embodiment, the above-mentioned recognition unit 804 is further configured to: use a pre-trained gait feature extraction network to respectively extract gait features corresponding to the third gait image sequence and the fourth gait image sequence; determine the similarity of the gait features corresponding to the third gait image sequence and the fourth gait image sequence, respectively, and determine a gait recognition result based on the determined similarity.

[0128] In some optional implementations of this embodiment, the alignment unit 803 is further configured to: determine the average viewing angle corresponding to each gait model in the first gait model sequence; and adjust the viewing angle of each gait model in the second gait model sequence to align with the average viewing angle.

[0129] In some optional implementations of this embodiment, the acquisition unit 801 is further configured to: acquire a first gait image sequence of the target person; acquire a target video, and extract the gait image sequences of each person in the target video respectively to obtain at least one second gait image sequence; and the modeling unit 802 is further configured to: perform three-dimensional modeling on the human body in each gait image in the first gait image sequence to obtain a first gait model sequence; for each second gait image sequence in at least one second gait image sequence, perform three-dimensional modeling on the human body in each gait image in the second gait image sequence to obtain a second gait model sequence.

[0130] In some optional implementations of this embodiment, the alignment unit 803 is further configured to: perform perspective alignment on the obtained at least one second gait model sequence and the first gait model sequence respectively.

[0131] The device provided by the above-mentioned embodiment of the present disclosure acquires a first gait image sequence and a second gait image sequence through an acquisition unit; a modeling unit performs three-dimensional modeling of the human body in each gait image in the first gait image sequence and the second gait image sequence to obtain a first gait model sequence and a second gait model sequence; an alignment unit performs perspective alignment on the first gait model sequence and the second gait model sequence; and a recognition unit performs gait recognition based on the aligned first gait model sequence and the second gait model sequence. This can achieve gait matching and recognition under the same perspective, avoid interference with gait matching and recognition caused by perspective differences, and thus help improve the accuracy of gait recognition results.

[0132] Reference below Figure 9 , which shows an electronic device (eg, Figure 1 A schematic diagram of the structure of the server in (900). Figure 6 The server shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0133] like Figure 9 As shown, the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. Various programs and data required for the operation of the electronic device 900 are also stored in the RAM 903. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0134] Typically, the following devices may be connected to the I / O interface 905: an input device 906 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 908 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 909. The communication device 909 may allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 9 The electronic device 900 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead. Figure 9Each block shown in the figure may represent one device, or may represent multiple devices as needed.

[0135] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network via the communication device 909, or installed from the storage device 908, or installed from the ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0136] It should be noted that the computer-readable medium described in the embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device, or device. In the embodiments of the present disclosure, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0137] The computer-readable medium may be included in the electronic device, or may exist independently and not be incorporated into the electronic device. The computer-readable medium carries one or more programs. When executed by the electronic device, the one or more programs cause the electronic device to: acquire a first gait image sequence and a second gait image sequence; perform three-dimensional modeling of the human body in each gait image in the first gait image sequence and the second gait image sequence to obtain a first gait model sequence and a second gait model sequence; perform perspective alignment on the first gait model sequence and the second gait model sequence; and perform gait recognition based on the aligned first gait model sequence and the second gait model sequence.

[0138] Computer program code for performing the operations of embodiments of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0139] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0140] The units involved in the embodiments described in the present disclosure may be implemented by software or by hardware. The units described may also be provided in a processor. For example, they may be described as: a processor including an acquisition unit, a modeling unit, an alignment unit, and a recognition unit. The names of these units do not, in some cases, constitute limitations on the units themselves. For example, the acquisition unit may also be described as a "unit for acquiring a first gait image sequence and a second gait image sequence."

[0141] The above description is merely a preferred embodiment of the present disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also encompass other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned inventive concept. For example, a technical solution formed by mutually replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in the embodiments of the present disclosure.

Claims

1. A gait recognition method, comprising: Obtain the target person's first step image sequence; Acquire a target video, and respectively extract a gait image sequence of each person in the target video to obtain at least one second gait image sequence; Performing three-dimensional modeling of a human body in each gait image in the first gait image sequence and the second gait image sequence to obtain a first gait model sequence and a second gait model sequence; Performing perspective alignment on the first gait model sequence and the second gait model sequence, comprising: determining an average perspective corresponding to each gait model in the first gait model sequence; and adjusting the perspective of each gait model in the second gait model sequence to align with the average perspective; Gait recognition is performed based on the first gait model sequence and the second gait model sequence aligned with the average viewing angle to obtain a gait recognition result; the gait recognition result is used to indicate whether the target person is included in the target video.

2. The method according to claim 1, wherein The step of performing gait recognition according to the first gait model sequence and the second gait model sequence aligned with the average viewing angle to obtain a gait recognition result includes: extracting gait features corresponding to the first gait model sequence and the second gait model sequence aligned with the average view angle respectively, to obtain the first gait feature and the second gait feature; determining a similarity between the first gait feature and the second gait feature; The gait recognition result is determined according to the similarity.

3. The method according to claim 1, wherein The step of performing gait recognition according to the first gait model sequence and the second gait model sequence aligned with the average viewing angle to obtain a gait recognition result includes: converting each gait model in the first gait model sequence into two dimensions to obtain a third gait image sequence; converting each gait model in the second gait model sequence aligned with the average viewing angle into two dimensions to obtain a fourth gait image sequence; Gait recognition is performed according to the third gait image sequence and the fourth gait image sequence to obtain the gait recognition result.

4. The method according to claim 3, wherein: The performing gait recognition according to the third gait image sequence and the fourth gait image sequence to obtain the gait recognition result includes: extracting gait features corresponding to the third gait image sequence and the fourth gait image sequence respectively using a pre-trained gait feature extraction network; Determine the similarity between the gait features corresponding to the third gait image sequence and the fourth gait image sequence, and determine the gait recognition result according to the determined similarity.

5. The method according to any one of claims 1 to 4, wherein: The three-dimensional modeling of the human body in each gait image in the first gait image sequence and the second gait image sequence to obtain the first gait model sequence and the second gait model sequence includes: Performing three-dimensional modeling on the human body in each gait image in the first gait image sequence to obtain a first gait model sequence; For each second gait image sequence in the at least one second gait image sequence, three-dimensional modeling is performed on the human body in each gait image in the second gait image sequence to obtain a second gait model sequence.

6. The method according to claim 5, wherein: The performing perspective alignment on the first gait model sequence and the second gait model sequence comprises: Perspective alignment is performed on the obtained at least one second gait model sequence and the first gait model sequence.

7. A gait recognition device, comprising: The acquisition unit is configured to: acquire a first posture image sequence of the target person; Acquire a target video, and respectively extract a gait image sequence of each person in the target video to obtain at least one second gait image sequence; A modeling unit is configured to perform three-dimensional modeling of a human body in each gait image in the first gait image sequence and the second gait image sequence to obtain a first gait model sequence and a second gait model sequence; an alignment unit configured to perform perspective alignment on the first gait model sequence and the second gait model sequence, comprising: determining an average perspective corresponding to each gait model in the first gait model sequence; and adjusting the perspective of each gait model in the second gait model sequence to align with the average perspective; The recognition unit is configured to perform gait recognition based on the first gait model sequence and the second gait model sequence aligned with the average viewing angle to obtain a gait recognition result; the gait recognition result is used to indicate whether the target person is included in the target video.

8. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer-readable medium having a computer program stored thereon, wherein: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images