Gait recognition device, gait recognition method, and computer readable storage medium

Abstract

The invention discloses a gait recognition device, a gait recognition method and a computer readable storage medium. The gait recognition device may comprise: a probability distribution acquisition unit for acquiring a first part probability distribution of each frame included in a first video clip, the first part probability distribution indicating a probability that each pixel point of the frame relates to each body part; the feature extraction unit is used for extracting a global feature of the first video clip, extracting a local feature of the first video clip based on probability distribution of a first part of the first video clip, and fusing the global feature and the local feature to obtain a fused feature as a gait feature of the first video clip; and the judging unit is used for judging whether a first predetermined object in the first video clip and a second predetermined object in the second video clip belong to the same person or not according to the gait feature of the first video clip and the gait feature of the second video clip.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular to a gait recognition device, a gait recognition method and a computer-readable storage medium. Background Art

[0002] With the continuous development of science and technology, biometric authentication technology has been widely used in public security and other fields. Commonly used biometric authentication technologies include gait recognition, iris, face, ReID (person re-identification) technology, etc. Compared with iris, face, ReID and other biometric authentication technologies, gait recognition technology has the advantages of long recognition distance (for example, no need to get close), full-view recognition, insensitivity to light, difficult to disguise (for example, plastic surgery, makeup, changing clothes, etc. can also be recognized), no need for cooperation (for example, no need to align the retina, face, etc.), low requirements for collection equipment (for example, ordinary cameras can be used to collect videos), etc., and has a wider and more flexible application scenario.

[0003] In gait recognition technology, gait features with distinguishing capabilities can be constructed by extracting information such as walking patterns and appearance contours, thereby determining whether people in different videos are the same person. In order to shield the influence of factors such as complex video background, environment, and clothing, the gait recognition model can extract gait features based on silhouette or pose data.

[0004] The silhouette-based gait recognition method performs better than the posture-based gait recognition method in the ordinary clothing and backpack scenarios, while the posture-based gait recognition method performs better than the silhouette-based method in the thick clothing scenario. In terms of the average performance of various scenarios, the silhouette-based gait recognition method has more advantages.

[0005] It should be noted that the above introduction to the technical background is only for the convenience of providing a clear and complete description of the technical solutions of the present application and for the convenience of understanding by those skilled in the art. It cannot be considered that the above technical solutions are well known to those skilled in the art simply because these solutions are described in the background technology part of the present application. Summary of the invention

[0006] A brief overview of the disclosure is given below in order to provide a basic understanding of certain aspects of the disclosure. However, it should be understood that this overview is not an exhaustive overview of the disclosure. It is not intended to identify the key or important parts of the disclosure, nor is it intended to limit the scope of the disclosure. Its purpose is simply to give certain concepts of the disclosure in a simplified form as a prelude to a more detailed description given later.

[0007] An object of the present disclosure is to provide an improved gait recognition device, a gait recognition method, a training device, a training method and a computer-readable storage medium, which make it possible, for example, to improve the accuracy of gait recognition results.

[0008] According to a first aspect of the present disclosure, a gait recognition device is provided, comprising: a probability distribution acquisition unit, configured to acquire a first part probability distribution of each frame included in a first video clip, wherein the first part probability distribution indicates the probability that each pixel point of the frame relates to each body part; a feature extraction unit, configured to extract global features of the first video clip using a global feature extraction layer of a gait model, extract local features of the first video clip based on the first part probability distribution of the first video clip using a local feature extraction layer of the gait model, and fuse the global features and the local features to obtain the fused features as the gait features of the first video clip; and a judgment unit, configured to judge whether a first predetermined object in the first video clip and a second predetermined object in the second video clip belong to the same person according to the gait features of the first video clip and the gait features of the second video clip. The gait features of the second video clip are obtained by fusing the global features and local features of the second video clip, and the local features of the second video clip are obtained based on the second part probability distribution of each frame included in the second video clip, wherein the second part probability distribution indicates the probability that each pixel point of the frame relates to each body part.

[0009] According to a second aspect of the present disclosure, a gait recognition method is provided, comprising: obtaining a first part probability distribution of each frame included in a first video clip, wherein the first part probability distribution indicates the probability that each pixel point of the frame relates to each body part; extracting global features of the first video clip using a global feature extraction layer of a gait model, extracting local features of the first video clip based on the first part probability distribution of the first video clip using a local feature extraction layer of the gait model, and fusing the global features and the local features to obtain the fused features as the gait features of the first video clip; and judging whether a first predetermined object in the first video clip and a second predetermined object in the second video clip belong to the same person according to the gait features of the first video clip and the gait features of the second video clip. The gait features of the second video clip are obtained by fusing the global features and local features of the second video clip, and the local features of the second video clip are obtained based on the second part probability distribution of each frame included in the second video clip, and the second part probability distribution indicates the probability that each pixel point of the frame relates to each body part.

[0010] According to other aspects of the present disclosure, a computer program code and a computer program product for implementing the method according to the present disclosure are also provided, as well as a computer-readable storage medium having the computer program code for implementing the method according to the present disclosure recorded thereon.

[0011] Other aspects of the embodiments of the present disclosure are given in the following description, wherein the detailed description is used to fully disclose the preferred embodiments of the embodiments of the present disclosure without imposing limitations thereon. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The present disclosure may be better understood by referring to the detailed description given below in conjunction with the accompanying drawings, wherein the same or similar reference numerals are used throughout the drawings to represent the same or similar components. The accompanying drawings, together with the following detailed description, are included in and form a part of the present specification to further illustrate the preferred embodiments of the present disclosure and to explain the principles and advantages of the present disclosure. Among them:

[0013] Figure 1 is a block diagram showing a functional configuration example of a gait recognition device according to an embodiment of the present disclosure;

[0014] Figure 2A and Figure 2B is a schematic diagram showing an example of the architecture of a gait recognition device according to an embodiment of the present disclosure;

[0015] Figure 3A and Figure 3B is a schematic diagram showing the geometric partitioning method;

[0016] Figure 4 An example of a visual representation of a first part probability distribution is shown;

[0017] Figure 5 showing an additional example of a visual representation of a first location probability distribution;

[0018] Figure 6 is a schematic diagram showing another architecture example of a gait recognition device according to an embodiment of the present disclosure;

[0019] Figure 7 is a schematic diagram showing a specific example of a gait recognition device according to an embodiment of the present disclosure;

[0020] Figure 8 is a schematic diagram showing a specific example of a feature extraction unit according to an embodiment of the present disclosure;

[0021] Fig. 9 A comparison between a gait recognition device according to an embodiment of the present disclosure and the prior art is shown;

[0022] Fig.10 is a flowchart showing an example of the process of a gait recognition method according to an embodiment of the present disclosure;

[0023] Fig.11 is a block diagram showing a functional configuration example of a training device according to an embodiment of the present disclosure;

[0024] Fig.12 is a flowchart showing an example of a process of a training method according to an embodiment of the present disclosure;

[0025] Fig.13 is a block diagram showing a functional configuration example of an information processing system according to an embodiment of the present disclosure;

[0026] Fig.14 is a flowchart showing an example of the flow of an information processing method according to an embodiment of the present disclosure; and

[0027] Fig.15 : is a block diagram showing an example structure of a personal computer that can be employed in the embodiments of the present disclosure. DETAILED DESCRIPTION

[0028] Exemplary embodiments of the present disclosure will be described below in conjunction with the accompanying drawings. For the sake of clarity and conciseness, not all features of the actual implementation are described in the specification. However, it should be understood that many implementation-specific decisions must be made in the process of developing any such actual implementation in order to achieve the developer's specific goals, such as meeting those constraints related to the system and business, and these constraints may vary from implementation to implementation. In addition, it should be understood that although the development work may be very complex and time-consuming, it is only a routine task for those skilled in the art who benefit from the content of this disclosure.

[0029] It is also necessary to explain here that in order to avoid obscuring the present disclosure due to unnecessary details, only the device structure and / or processing steps closely related to the scheme according to the present disclosure are shown in the accompanying drawings, while other details that are not closely related to the present disclosure are omitted.

[0030] Embodiments according to various aspects of the present disclosure are described in detail below with reference to the accompanying drawings.

[0031] According to the first embodiment of the present disclosure

[0032] First, refer to Figure 1 and FIG. 2A to FIG. 2B An implementation example of the gait recognition device 100 according to an embodiment of the present disclosure is described. Figure 1 : is a block diagram showing a functional configuration example of the gait recognition device 100 according to an embodiment of the present disclosure. FIG. 2A to FIG. 2Bis a schematic diagram showing an example of the architecture of the gait recognition device 100 according to an embodiment of the present disclosure.

[0033] like Figure 1 and FIG. 2A to FIG. 2B As shown, the gait recognition device 100 according to an embodiment of the present disclosure may include a probability distribution acquisition unit 102 , a feature extraction unit 104 and a judgment unit 106 .

[0034] The probability distribution acquisition unit 102 may be configured to acquire a first part probability distribution for each frame included in the first video clip. The first part probability distribution may indicate the probability that each pixel of the frame relates to each body part. The term "video clip" used in this article may refer to a continuously shot video, a clip captured from a continuously shot video, etc. In addition, as commonly used in the art, the term "frame" refers to an image that constitutes a video clip. In the following, for ease of understanding and description, "frame" is sometimes referred to as "image".

[0035] The feature extraction unit 104 can be configured to obtain the gait features of the first video clip through the following operations: extracting the global features of the first video clip using the global feature extraction layer of the gait model, extracting the local features of the first video clip based on the first part probability distribution of the first video clip using the local feature extraction layer of the gait model, and fusing the global features and the local features to obtain the fused features as the gait features of the first video clip.

[0036] The judging unit 106 may be configured to judge whether a first predetermined object in the first video segment and a second predetermined object in the second video segment belong to the same person based on the gait feature of the first video segment and the gait feature of the second video segment. For example, the first predetermined object may be a first human body or a part of the first human body, and the second predetermined object may be a second human body or a part of the second human body.

[0037] For example, the gait feature of the second video clip may be obtained by fusing the global feature and the local feature of the second video clip. For example, the local feature of the second video clip may be obtained based on the second part probability distribution of each frame included in the second video clip. The second part probability distribution may indicate the probability that each pixel point of the frame relates to each body part.

[0038] In existing silhouette-based gait recognition methods, binary silhouettes are usually used to extract human information from images. However, binary silhouettes lack structural information of human body parts and are prone to cause local gait pattern loss due to self-occlusion, which limits the improvement of gait recognition model performance.

[0039] As described above, in the gait recognition device 100 according to an embodiment of the present disclosure, local features can be extracted based on the probability that each pixel point of each frame of the video clip involves each body part, thereby reducing or avoiding the loss of local gait patterns due to self-occlusion, thereby improving the accuracy of recognition based on the gait features finally obtained.

[0040] In addition, the gait recognition device 100 according to the embodiment of the present disclosure extracts local features of the video segment based on each pixel point of each frame of the video segment. Figure 3A and Figure 3B As shown, by using the geometric (horizontal or block) division method, in the embodiment of the present disclosure, it is possible to achieve spatial and temporal alignment of local features with specific semantic body parts, thereby improving the accuracy of recognition based on the gait features finally obtained.

[0041] <Probability Distribution Acquisition Unit 102>

[0042] In some embodiments, the probability distribution acquisition unit 102 can acquire the first part probability distribution in various ways. For example, the first part probability distribution can be acquired using a human parsing model. For example, the output of the human parsing model can be a probability distribution graph whose size is consistent with the size of the input image. The value of each pixel point in the probability distribution graph represents the probability that the corresponding pixel point of the input image involves each body part. For example, the value P(i, j) of each pixel point can be expressed by the following formula (1):

[0043] P(i,j)=[P 0 (i,j),…,P k (i,j),…,P K (i,j)], where Formula (1)

[0044] In formula (1), k is the index number of the semantic category of the body part. Where k = 0 is used to represent the background category, and k>0 is used to represent the defined semantic category of the body part. K represents the total number of defined body parts. (i, j) represents the coordinate position of the pixel point. P k (i, j) represents the probability that the pixel point is related to category k.

[0045] For example, Figure 4 An example of a visual representation of the first part probability distribution is shown. Figure 4In the figure, the human body is divided into K=10 parts, where k=1 corresponds to the head, k=2 corresponds to the trunk, k=3 corresponds to the left upper arm, k=4 corresponds to the right upper arm, k=5 corresponds to the left lower arm, k=6 corresponds to the right lower arm, k=7 corresponds to the left upper leg, k=8 corresponds to the right upper leg, k=9 corresponds to the left lower leg, and k=10 corresponds to the right lower leg. The present application is not limited to this, and the body parts of the human body can also be defined in other ways according to the needs of the business scenario.

[0046] exist Figure 4 In the probability map corresponding to body part k (k is greater than 0), the value of each pixel can be equal to the probability P of the pixel in the corresponding input image involving part k. k (i,j). Figure 4 It can be seen that different body parts are clearly distinguished. Figure 4 The probability map corresponding to the background category k=0 is also shown in FIG. 1 , where the value of each pixel point can be equal to (1-the probability P of the pixel point in the corresponding input image involving the background category) 0 (i, j)), which can represent the probability that the pixel point is related to the first predetermined object.

[0047] Figure 5 Another example of a visual representation of the first part probability distribution is shown. Figure 5 In each of the figures shown, the value of each pixel represents the information entropy of the pixel in the input image. The information entropy can be expressed by the following formula (2):

[0048] H k (i,j)=-(P k (i,j)log2P k (i,j)+(1-P k (i,j))log2(1-P k (i,j)))

[0049] Formula (2)

[0050] In formula (2), H k (i, j) may represent the value at the pixel point (i, j) in the image corresponding to category k.

[0051] For example, in data processing, the probability distribution map can be represented by a plurality of two-dimensional matrices corresponding to K categories respectively. The elements in each two-dimensional matrix can indicate the probability that the pixel corresponding to the element in the input image is related to the category corresponding to the two-dimensional matrix. For example, in the two-dimensional matrix corresponding to category k, the value of each element can be the probability that the pixel corresponding to the element is related to category k or the corresponding information entropy, such as the information entropy calculated by the above formula (2). In some examples, the probability distribution map can be represented by a three-dimensional matrix.

[0052] In some embodiments, the human body parsing model can be a class segmentation model.

[0053] In some embodiments, before using the human body analysis model to obtain the probability distribution of the first part, the human body analysis model can be trained. The annotation data used to train the human body analysis model can be defined according to the required human body parts without considering the clothing worn. The specific training method of the human body analysis model can refer to the relevant technology and will not be described here.

[0054] <Feature Extraction Unit 104>

[0055] In some embodiments, the global feature extraction layer of the gait recognition model may include one or more three-dimensional (3D) convolutional layers. Figure 2A As shown, the feature extraction unit 104 can input the first video segment (i.e., the frame sequence constituting the first video segment) into the global feature extraction layer, and perform a 3D convolution operation on the input frame sequence through the global feature extraction layer to extract global features.

[0056] In some embodiments, Figure 2B As shown, the feature extraction unit 104 can input the first object probability distribution of the first video clip into the global feature extraction layer, and perform a 3D convolution operation on the input first object probability distribution through the global feature extraction layer to extract global features. The first object probability distribution can indicate the probability that each pixel point of each frame of the first video clip involves the first predetermined object (i.e., the first human body or a part of the first human body). For example, the first object probability distribution can be represented by a sequence of first object probability distribution graphs corresponding to the frame sequence of the first video clip, respectively. Each first object probability distribution graph can indicate the probability that each pixel point in the frame corresponding to the first object probability distribution graph involves the first predetermined object. For example, the value of a pixel point (i, j) in the first object probability distribution graph can be (1-P 0 (i, j)) or the information entropy H calculated by the above formula (2) 0 (i,j). Figure 4 and Figure 5 The graph shown corresponding to k=0 shows an example of a visual representation of the first object probability distribution, and it can be seen that the uncertainty of the contour of the human body is well preserved.

[0057] The existing binary silhouette method has the problem of contour noise. Specifically, since the boundary between the foreground (human body) and the background is uncertain, the judgment of whether the pixel point near the boundary is the foreground or the background is often wrong, resulting in an inaccurate silhouette contour. For silhouette extraction based on the segmentation model, the foreground and background probabilities of each pixel point near the boundary are close to 0.5. Binary silhouette is equivalent to forcibly using a threshold of 0.5 to determine whether the pixel point is the foreground or the background when the inaccurate foreground and background probabilities are close to 0.5. This forced category judgment is prone to loss of detail information. Therefore, binary silhouette limits the further improvement of the performance of the gait recognition model.

[0058] As described above, in some embodiments of the present disclosure, global features are extracted based on a first object probability distribution indicating the probability that each pixel point of each frame is related to a first predetermined object, thereby retaining more contour details and further improving the accuracy of recognition based on gait features.

[0059] In some embodiments, the feature extraction unit 104 may utilize a local feature extraction layer to perform feature extraction on the video segment based on the first part probability distribution to obtain local features of the video segment.

[0060] In some embodiments, the feature extraction unit 104 can use the local feature extraction layer to obtain the weight of each body part based on the first part probability distribution, obtain the part local feature corresponding to each body part based on the weight and the global feature extracted by the global feature extraction layer, and fuse the part local features to obtain the fused part local feature as the local feature. The local features obtained in this way can better characterize the features corresponding to each body part, thereby further improving the recognition accuracy based on the gait features finally obtained.

[0061] In the text, the configuration for acquiring the weight of each body part may be referred to as a "part weight acquisition sublayer".

[0062] For example, the feature extraction unit 104 may fuse the weight of each body part with the global feature extracted by the global feature extraction layer by dot multiplication to obtain a part local feature corresponding to each body part (hereinafter referred to as a "first part local feature"). For example, the feature extraction unit 104 may fuse the first part local features corresponding to each body part by addition to obtain the fused first part local features as the local features.

[0063] In some embodiments, the feature extraction unit 104 may use the feature extraction sublayer to extract the first part local features corresponding to each body part obtained based on the weight and the global features to obtain the second part local features. In this case, the feature extraction unit 104 may fuse the second part local features of each body part to obtain the fused second part local features as the local features of the first video clip, but the fusion method is not limited thereto, and an appropriate fusion method may be selected according to the specific representation method of the second part local features.

[0064] In some embodiments, the feature extraction sublayer may include multiple feature extraction sublayers corresponding to multiple body parts respectively. The structural parameters of the multiple feature extraction sublayers may be the same.

[0065] In some embodiments, multiple body parts can share the feature extraction sublayer.

[0066] In some embodiments, Figure 6 As shown, the gait recognition device 100 may include multiple feature extraction units 104. The fused features obtained by each feature extraction unit 104 may be used as input to the next feature extraction unit 104. In this case, the fused features obtained by the last feature extraction unit 104 may be used as the gait features of the first video clip. By including multiple feature extraction units 104, the recognition accuracy based on the obtained gait features may be further improved.

[0067] The following will be combined Figure 7 and Figure 8 The operation of the feature extraction unit is further explained. Figure 7 is an architecture diagram showing a specific example of the gait recognition device 100 . Figure 8 is an architectural diagram showing a specific example of the feature extraction unit 104.

[0068] like Figure 7 As shown, the entire human body sequence of the video clip can be Perform a 3D convolution (with a kernel of (3,3,3)) and LTA operation to extract shallow spatiotemporal features Y LTA Among them, LTA is also a 3D convolution, and its convolution kernel is (3,1,1), which is used to aggregate feature information in the time dimension. Figure 7 The entire human body sequence is shown in FIG. 1 , which is a first object probability distribution map sequence output by the probability distribution acquisition unit 102. However, in some embodiments, the entire human body sequence may be a sequence of frames constituting the first video segment.

[0069] As mentioned above, for the first object probability distribution map sequence For each figure, the value of the pixel point (i, j) in the figure can be (1 - P 0 (i,j)) or the information entropy H 0 (i,j) calculated by the above formula (2). Correspondingly, for each figure in the first part probability distribution map sequence (where k is a natural number and 0 < k ≤ K), the value of the pixel point (i, j) in the figure can be P k (i,j)) or the information entropy H k (i,j).

[0070] Although it is shown in a visual form in Figure 7 and Figure 8 from to However, in actual data processing, to can be represented by multiple two-dimensional matrices or three-dimensional matrices respectively. The value of each element in each two-dimensional matrix or three-dimensional matrix can be the value of the corresponding pixel point.

[0071] As Figure 7 shown, the obtained spatio-temporal feature Y LTA can be input into the feature extraction unit 104a. The feature extraction unit 104a can extract global features from the spatio-temporal feature Y LTA using the global feature extraction layer. For example, as Figure 8 shown, the global feature extraction layer can include a 3D convolutional layer (3, 3, 3) for performing a convolutional operation on the input spatio-temporal feature to extract the global feature Y Global . In Figure 8 , Y i can represent the input of the global feature extraction layer of the feature extraction unit. As Figure 7 shown, for the feature extraction unit 104a, Y i can be the spatio-temporal feature Y LTA , and for the feature extraction units 104b and 104c, Y i can be Y2 and Y3 respectively.

[0072] Although the configuration for obtaining the spatio-temporal feature Y Figure 7 is shown in LTA and is not included in the feature extraction unit, in some embodiments, the configuration for obtaining the spatio-temporal feature Y LTA can be included in the feature extraction unit, for example, included in the first feature extraction unit 104a. Additionally, although in Figure 7, three feature extraction units 104a, 104b and 104c are shown, but in actual applications, more or fewer feature extraction units can be set as needed. In the following, when there is no need to specifically distinguish the feature extraction units 104a, 104b and 104c, they are collectively referred to as "feature extraction units 104".

[0073] In some embodiments, Figure 8 As shown, the feature extraction unit 104 can use the part weight acquisition sublayer to perform an adaptive average pooling operation on the first part probability distribution to aggregate the spatiotemporal information, thereby obtaining an aggregated feature with the same spatiotemporal size as the corresponding global feature. For example, the feature extraction unit 104 can also use a loss layer to constrain the values ​​of the aggregated features of each body part to the interval range of 0 to 1 through a softmax function, and use the constrained aggregated feature M 1 ,…,M K Used as weights to extract each body part.

[0074] The feature extraction unit 104 can aggregate the feature M 1 ,…,M K and the global feature Y Global To merge, for example Figure 8 As shown in the figure, the aggregate feature M 1 ,…,M K and the global feature Y Global Perform a dot product operation to obtain the first part local features corresponding to multiple body parts 1 to K respectively

[0075]

[0076] In some embodiments, Figure 8 As shown, the local features of the first part can be further extracted by the feature extraction sublayer. Perform feature extraction to obtain the local features of the second part For example, the feature extraction sublayer may include the second part local features The corresponding multiple 3D convolution kernels (3,3,3) are used to local features of the second part Perform convolution operation to extract the local features of the second part Multiple 3D convolution kernels (3,3,3) can share structural parameters.

[0077] In some embodiments, the local features of the first part can be convolved by the same 3D convolution kernel (3,3,3) Perform feature extraction to obtain the local features of the second part

[0078] After obtaining the second local feature Afterwards, the feature extraction unit 104 may extract the second local feature Fusion is performed to obtain the local features Y of the video clip local Since different body parts are distributed differently in space, the second local features corresponding to multiple body parts can be aggregated by simple element-wise addition. As the final local feature Y local .

[0079] The feature extraction unit 104 can extract the local feature Y local and the global feature Y Global The fused feature is obtained as the output of the feature extraction unit 104. For example, the last feature extraction unit 104c can extract the local feature Y in a longitudinal splicing manner. local and the global feature Y Global Fusion is performed, that is, the local feature Y is fused along the height direction local and the global feature Y Global The two are spliced ​​(such as Figure 8 As shown in GSLConvB in ), to obtain the fused features, such as Figure 8 In The remaining feature extraction units 104a and 104b can be fused by element-wise addition operation (e.g. Figure 8 ) to obtain fused features, such as Figure 8 In

[0080] Although in Figure 8 The “space-time mask alignment” operation and the “local feature extraction” operation are outlined by different dashed boxes, but in some embodiments, the “local feature extraction” operation may include the “space-time mask alignment” operation.

[0081] In some embodiments, Figure 7 As shown, a pooling unit may be provided between the feature extraction unit 104a and the feature extraction unit 104b to perform a pooling operation on the fused feature Y1 obtained by the feature extraction unit 104a, so as to aggregate spatial (height, width) features. Figure 7 , the pooling unit is not included in the feature extraction unit, but in some embodiments, the pooling unit may be included in the feature extraction unit. In addition, the number of pooling units is not limited to Figure 7 As shown, other numbers of pooling units may be arranged between adjacent pooling units according to actual needs.

[0082] <Judgment Unit 106>

[0083] In some embodiments, the judgment unit 106 may calculate similarity information based on the first gait feature and the second gait feature, and judge whether the first human body and the second human body belong to the same person based on the similarity information. For example, the judgment unit 106 may calculate the similarity information using the technology described in reference 1 (Gait Recognition via Effective Global-Local Feature Representation and Local Temporal Aggregation), and thus will not be described in detail.

[0084] For example, when the calculated similarity information is greater than a predetermined threshold, it can be determined that the first person and the second person belong to the same person; when the calculated similarity information is less than the predetermined threshold, it can be determined that the first person and the second person do not belong to the same person.

[0085] In some embodiments, when there are multiple pieces of similarity information greater than a predetermined threshold, the second human body corresponding to the highest similarity can be determined to be the same person as the first human body; or, the identification information of multiple second human bodies corresponding to the multiple pieces of similarity information can be output for user reference. For example, the identification information of multiple second human bodies corresponding to the multiple pieces of similarity information can be output in order from high to low similarity information, and so on.

[0086] In some embodiments, the gait feature of the second video clip may be obtained by the gait recognition device 100. For example, the probability distribution acquisition unit 102 may also acquire the second part probability distribution of the second video clip, and the feature extraction unit 104 may also extract the gait feature of the second video clip. For example, the probability distribution acquisition unit 102 may acquire the second part probability distribution in a manner similar to or the same as that of acquiring the first part probability distribution, and the feature extraction unit 104 may extract the gait feature of the second video clip in a manner similar to or the same as that of extracting the gait feature of the first video clip. As an example, the probability distribution acquisition unit 102 may include two subunits for acquiring the first part probability distribution and the second part probability distribution, respectively, and / or the feature extraction unit 104 may include two subunits for extracting the gait feature of the first video clip and the gait feature of the second video clip, respectively. As another example, the probability distribution acquisition unit 102 may acquire the first part probability distribution and the second part probability distribution through the same subunit, and / or the feature extraction unit 104 may extract the gait feature of the first video clip and the gait feature of the second video clip through the same subunit.

[0087] In some embodiments, the gait feature of the second video clip may be obtained by another device different from the gait recognition device 100. For example, the other device may send the obtained gait feature of the second video clip directly or indirectly via other devices to the gait recognition device 100. By way of example and not limitation, the other device may store the obtained gait feature of the second video clip in a storage unit for access by the gait recognition device 100.

[0088] The gait recognition device according to the embodiment of the present disclosure can be applied to various scenarios, such as a 1-to-many (i.e., 1-to-N) scenario, or a 1-to-1 scenario. Among them, the 1-to-N scenario is, for example: performing gait recognition on a first video clip and multiple second video clips. For example, the gait features of the person in the query video clip (i.e., the first video clip) and the gait features of the person already recorded in the storage unit (i.e., the gait features of the second video clip) are compared to determine whether the person in the query video is included in the recorded persons or which one of the recorded persons (for example, the one (or more) corresponding persons with a similarity greater than a threshold and the highest ranking are the result of gait recognition).

[0089] An example of a one-to-one scenario is: gait recognition is performed on a first video clip and a second video clip. For example, the gait features of two people in different video clips are compared to determine whether they are the same person (for example, if the similarity between the two gaits is greater than a threshold, then they are determined to be the same person). In some examples, the label of the person in the second video clip may be known, while in other examples, the label of the person in the second video clip may not be known.

[0090] Fig. 9 FIG. 1 shows a comparison between the gait recognition device 100 according to an embodiment of the present disclosure and the prior art in reference 1. Fig. 9 It can be seen that in the three situations of normal (NM), backpack (BG) and clip-on (CL), the recognition accuracy of the gait recognition device 100 is improved by 2.01%, 2.84% and 2.23% respectively compared with the prior art in reference 1.

[0091] According to the second embodiment of the present disclosure

[0092] Corresponding to the above gait recognition device 100 according to the embodiment of the present disclosure, the present disclosure may also provide a gait recognition method. Fig.10 is a flowchart showing an example of the flow of a gait recognition method according to an embodiment of the present disclosure.

[0093] like Fig.10As shown, the gait recognition method 1000 according to the embodiment of the present disclosure may start at S1001 and end at S1008. The gait recognition method 1000 may include a probability distribution acquisition step S1002, a feature extraction step S1004 and a determination step S1006.

[0094] In the probability distribution acquisition step S1002, the first part probability distribution of each frame included in the first video clip can be obtained. The first part probability distribution can indicate the probability that each pixel point of the frame relates to each body part. For example, the probability distribution acquisition step S1002 can be implemented by the probability distribution acquisition unit 102 described above, so it is only briefly described below.

[0095] In the feature extraction step S1004, the gait feature of the first video segment can be obtained by the following operations: extracting the global feature of the first video segment using the global feature extraction layer of the gait model, extracting the local feature of the first video segment based on the probability distribution of the first part of the first video segment using the local feature extraction layer of the gait model, and fusing the global feature and the local feature to obtain the fused feature as the gait feature of the first video segment. For example, the feature extraction step S1004 can be implemented by the feature extraction unit 104 described above, so it is only briefly described below.

[0096] In the judgment step S1006, it can be judged whether the first predetermined object in the first video clip and the second predetermined object in the second video clip belong to the same person based on the gait features of the first video clip and the gait features of the second video clip. For example, the gait features of the second video clip can be obtained by fusing the global features and local features of the second video clip. For example, the local features of the second video clip can be obtained based on the second part probability distribution of each frame included in the second video clip. The second part probability distribution can indicate the probability that each pixel point of the frame involves each body part. For example, the judgment step S1006 can be implemented by the judgment unit 106 described above, so it will not be described in detail below.

[0097] Similar to the gait recognition device 100 according to the embodiment of the present disclosure, the gait recognition method 1000 according to the embodiment of the present disclosure can improve the accuracy of recognition based on gait features.

[0098] In some embodiments, in the feature extraction step S1004, the first object probability distribution of the first video clip can be input into the global feature extraction layer of the gait model, and a 3D convolution operation is performed on the input first object probability distribution through the global feature extraction layer to extract global features, thereby reducing or avoiding contour noise and further improving the accuracy of recognition based on gait features.

[0099] In some embodiments, in the feature extraction step S1004, the weight of each body part can be obtained based on the first part probability distribution using the local feature extraction layer, the part local feature corresponding to each body part is obtained based on the weight and the global feature extracted by the global feature extraction layer, and the part local features are fused to obtain the fused part local features as the local features. The local features obtained in this way can better characterize the features corresponding to each body part, thereby further improving the recognition accuracy based on the gait features finally obtained.

[0100] In some embodiments, in the feature extraction step S1004, the feature extraction sublayer can be used to extract the first part local features corresponding to each body part obtained based on the weight and the global features to obtain the second part local features, and the second part local features of each body part are fused to obtain the fused part local features as the local features of the corresponding video clip.

[0101] According to the third embodiment of the present disclosure

[0102] Examples of the gait recognition device 100 and the gait recognition method 1000 according to the embodiments of the present disclosure have been described above. According to the present disclosure, a training device for training a gait model may also be provided. Fig.11 1 is a block diagram illustrating a functional configuration example of a training device 1100 for training a gait model according to an embodiment of the present disclosure.

[0103] like Fig.11 As shown, the training device 1100 according to an embodiment of the present disclosure may include a second probability distribution acquisition unit 1102 , a second feature extraction unit 1104 and a model training unit 1106 .

[0104] The second probability distribution acquisition unit 1102 may be configured to acquire the first part probability distribution of each frame in the training video segment. The first part probability distribution may indicate the probability that each pixel of the frame relates to each body part. For example, the second probability distribution acquisition unit 1102 may acquire the first part probability distribution of each frame in a manner similar to the first probability distribution acquisition unit 102 described above, so the specific details may refer to the description above.

[0105] The second feature extraction unit 1104 may be configured to: set a global feature extraction layer for the training video segment to extract the global features of the training video segment, set a local feature extraction layer for the training video segment to extract the local features of the training video segment based on the first part probability distribution, and fuse the global features and the local features to obtain the fused features. For example, the global feature extraction layer may include one or more three-dimensional convolutional layers, and the local feature extraction layer may include one or more three-dimensional convolutional layers.

[0106] The model training unit 1106 can be configured to train the gait model based on the fused features to obtain a trained gait model. The trained gait model can be used to determine whether the first predetermined object in the first video clip and the second predetermined object in the second video clip belong to the same person. For example, the model training unit 1106 can use cross-entropy loss and triplet loss to train the gait model to optimize the structural parameters of the gait model. The specific method of the model training unit 1106 can be found in the above-mentioned reference 1, and will not be described in detail herein.

[0107] As described above, the training device 1100 according to an embodiment of the present disclosure can extract local features of the training video clip based on the first part probability distribution indicating the probability that each pixel point of the frame involves each body part, and train the gait model based on the fused local features and global features, thereby improving the recognition accuracy of the trained gait model.

[0108] In some embodiments, the second probability distribution acquisition unit 1102 can also obtain a first object probability distribution for each frame in the training video clip, indicating the probability that each pixel point in the frame involves a predetermined object, and the second feature extraction unit 1104 can extract global features of the training video clip based on the first object probability distribution, thereby further improving the recognition accuracy of the trained gait model.

[0109] In some embodiments, the training device 1100 may include multiple second feature extraction units 1104. The fused features obtained by each second feature extraction unit 1104 may be used as input to the next second feature extraction unit 1104. For example, the model training unit 1106 may train the gait model based on the fused features obtained by the last second feature extraction unit 1104 among the multiple second feature extraction units 1104.

[0110] In some embodiments, at least one second feature extraction unit 1104 among the plurality of second feature extraction units 1104 is further configured to set a pooling layer for the training video clip to pool the fused features acquired by the second feature extraction unit 1104 .

[0111] In some embodiments, the second feature extraction unit 1104 can use the local feature extraction layer to obtain the weight of each body part based on the first part probability distribution, obtain the part local feature corresponding to each body part based on the weight and the global feature extracted by the global feature extraction layer (hereinafter referred to as the "first part local feature"), and fuse the part local features to obtain the fused part local features as local features. The local features obtained in this way can better characterize the features corresponding to each body part, thereby further improving the recognition accuracy of the trained gait model.

[0112] In some embodiments, the second feature extraction unit 1104 may use the feature extraction sublayer to extract the first part local features corresponding to each body part obtained based on the weight and the global features to obtain the second part local features. In this case, the second feature extraction unit 1104 may fuse the second part local features of each body part to obtain the fused part local features as the local features of the corresponding video clip.

[0113] According to the fourth aspect of the present disclosure,

[0114] Corresponding to the training device 1100 for training a gait model according to the above embodiment of the present disclosure, the present disclosure may also provide a training method 1200 for training a gait model.

[0115] Fig.12 FIG. 1 is a flow chart showing an example of a process of a training method for training a gait model according to an embodiment of the present disclosure. Fig.12 As shown, the training method 1200 according to an embodiment of the present disclosure may start at S1201 and end at S1208. The training method 1200 may include a second probability distribution acquisition step S1202, a second feature extraction step S1204, and a model training step S1206.

[0116] In the second probability distribution acquisition step S1202, the first part probability distribution of each frame in the training video segment can be obtained. The first part probability distribution can indicate the probability that each pixel point of the frame relates to each body part. For example, the second probability distribution acquisition step S1202 can be performed by the second probability distribution acquisition unit 1102 described above, so the specific details can be found in the above description.

[0117] In the second feature extraction step S1204, a global feature extraction layer may be set for the training video segment to extract the global features of the training video segment, a local feature extraction layer may be set for the training video segment to extract the local features of the training video segment based on the first part probability distribution, and the global features and the local features may be fused to obtain fused features. For example, the second feature extraction step S1204 may be performed by the second feature extraction unit 1104 described above, so the specific details may refer to the above description.

[0118] In the model training step S1206, the gait model can be trained based on the fused features obtained by the second feature extraction step S1204 to obtain a trained gait model. The trained prediction model can be used to determine whether the first predetermined object in the first video clip and the second predetermined object in the second video clip belong to the same person. For example, the model training step S1206 can be performed by the model training unit 1106 described above, so the specific details can be found in the above description.

[0119] Similar to the training device 1100 according to an embodiment of the present disclosure, the training method 1200 can extract local features of the training video clip based on a first part probability distribution indicating the probability that each pixel point of the frame involves each body part, and train the gait model based on the fused local features and global features, thereby improving the recognition accuracy of the trained gait model.

[0120] According to the fifth aspect of the present disclosure,

[0121] The present disclosure may also provide an information processing system. Fig.13 13 is a block diagram showing a configuration example of an information processing system 1300 according to an embodiment of the present disclosure.

[0122] like Fig.13 As shown, the information processing system 1300 may include a training device 1100 and a gait recognition device 100. For example, the training device 1100 may train a gait model using a training video clip to obtain a trained gait model, and the gait recognition device 100 may perform gait recognition using the trained gait model obtained by the training device 1100, thereby improving gait recognition accuracy.

[0123] According to the sixth aspect of the present disclosure,

[0124] The present disclosure may also provide an information processing method 1400 corresponding to the above-mentioned information processing system 1300 . Fig.14 14 is a flowchart showing an example of the flow of the information processing method 1400 according to an embodiment of the present disclosure.

[0125] like Fig.14As shown, the information processing method 1400 may start at S1401 and end at S1416. Specifically, the information processing method 1400 may include a third probability distribution acquisition step S1402, a third feature extraction step S1404, a model training step S1406, a fourth probability distribution acquisition step S1408, a fourth feature extraction step S1410 and a judgment step S1412.

[0126] In the third probability distribution acquisition step S1402, the first part probability distribution of each frame in the training video segment can be obtained. The first part probability distribution can indicate the probability that each pixel point of the frame relates to each body part. For example, the third probability distribution acquisition step S1402 can be performed by the second probability distribution acquisition unit 1102 described above, so the specific details can be found in the above description.

[0127] In the third feature extraction step S1404, a global feature extraction layer may be set for the training video segment to extract the global features of the training video segment, a local feature extraction layer may be set for the training video segment to extract the local features of the training video segment based on the first part probability distribution, and the global features and the local features may be fused to obtain fused features. For example, the third feature extraction step S1404 may be performed by the second feature extraction unit 1104 described above, so the specific details may refer to the above description.

[0128] In the model training step S1406, the gait model can be trained based on the fused features obtained by the third feature extraction step S1404 to obtain a trained gait model. The trained prediction model can be used to determine whether the first predetermined object in the first video clip and the second predetermined object in the second video clip belong to the same person. For example, the model training step S1406 can be performed by the model training unit 1106 described above, so the specific details can be found in the above description.

[0129] In the fourth probability distribution acquisition step S1408, the first part probability distribution of each frame included in the first video clip can be obtained. The first part probability distribution can indicate the probability that each pixel point of the frame relates to each body part. For example, the fourth probability distribution acquisition step S1408 can be implemented by the probability distribution acquisition unit 102 described above, so it is only briefly described below.

[0130] In the fourth feature extraction step S1410, the gait feature of the first video clip can be obtained by the following operations: extracting the global feature of the first video clip using the global feature extraction layer of the trained gait model obtained by the model training step S1406, extracting the local feature of the first video clip based on the first part probability distribution of the first video clip using the local feature extraction layer of the trained gait model, and fusing the global feature and the local feature to obtain the fused feature as the gait feature of the first video clip. For example, the fourth feature extraction step S1410 can be implemented by the feature extraction unit 104 described above, so it is only briefly described below.

[0131] In the judgment step S1412, it can be determined whether the first predetermined object in the first video segment and the second predetermined object in the second video segment belong to the same person according to the gait feature of the first video segment and the gait feature of the second video segment. For example, the judgment step S1412 can be implemented by the judgment unit 106 described above, so it is only briefly described below.

[0132] It should be pointed out that although the above describes the functional configuration and operation of the gait recognition device and method, the training device and method for training the gait model, and the information processing system and method according to the embodiments of the present disclosure, this is only an example and not a limitation, and those skilled in the art may modify the above embodiments according to the principles of the present disclosure, for example, the functional modules and operations in each embodiment may be added, deleted or combined, and such modifications shall fall within the scope of the present disclosure.

[0133] In addition, it should be pointed out that the method embodiments here correspond to the above-mentioned device and / or system embodiments. Therefore, for the contents not described in detail in the method embodiments, please refer to the description of the corresponding parts in the device and / or system embodiments, and the description will not be repeated here.

[0134] In addition, the present disclosure also provides a storage medium and a program product. It should be understood that the machine-executable instructions in the storage medium and the program product according to the embodiments of the present disclosure can also be configured to execute the above-mentioned gait recognition method, the training method for training the gait model, and the information processing method, so the contents not described in detail here can refer to the description of the previous corresponding parts, and will not be repeated here.

[0135] Accordingly, the storage medium for carrying the program product including the machine executable instructions is also included in the disclosure of the present invention, including but not limited to a floppy disk, an optical disk, a magneto-optical disk, a memory card, a memory stick, and the like.

[0136] In addition, it should be noted that the above series of processes, devices and systems can also be implemented by software and / or firmware. In the case of being implemented by software and / or firmware, from a storage medium or a network to a computer with a dedicated hardware structure, such as Fig.15 The general-purpose personal computer 1500 shown has programs constituting the software installed therein, and when the various programs are installed therein, the computer can execute various functions and the like.

[0137] exist Fig.15 In the embodiment, a central processing unit (CPU) 1501 executes various processes according to a program stored in a read-only memory (ROM) 1502 or a program loaded from a storage device 1508 to a random access memory (RAM) 1503. In the RAM 1503, data required when the CPU 1501 executes various processes and the like is also stored as needed.

[0138] The CPU 1501, the ROM 1502, and the RAM 1503 are connected to one another via a bus 1504. To the bus 1504, an input / output interface 1505 is also connected.

[0139] The following components are connected to the input / output interface 1505: an input device 1506 including a keyboard, a mouse, etc.; an output device 1507 including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage device 1508 including a hard disk, etc.; and a communication device 1509 including a network interface card such as a LAN card, a modem, etc. The communication device 1509 performs communication processing via a network such as the Internet.

[0140] A drive 1510 is also connected to the input / output interface 1505 as needed. A removable medium 1511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive 1510 as needed so that a computer program read therefrom is installed into the storage device 1508 as needed.

[0141] In the case where the above-described series of processing is realized by software, a program constituting the software is installed from a network such as the Internet or a storage medium such as the removable medium 1511 .

[0142] It should be understood by those skilled in the art that such storage media is not limited to Fig.15The removable medium 1511 shown has a program stored therein and is distributed separately from the device to provide the program to the user. Examples of the removable medium 1511 include magnetic disks (including floppy disks (registered trademark)), optical disks (including compact disk read-only memory (CD-ROM) and digital versatile disks (DVD)), magneto-optical disks (including minidiscs (MD) (registered trademark)), and semiconductor memories. Alternatively, the storage medium may be a ROM 1502, a hard disk included in the storage device 1508, or the like, in which the program is stored and distributed to the user together with the device containing them.

[0143] The preferred embodiments of the present disclosure are described above with reference to the accompanying drawings, but the present disclosure is certainly not limited to the above examples. Those skilled in the art may obtain various changes and modifications within the scope of the appended claims, and it should be understood that these changes and modifications will naturally fall within the technical scope of the present disclosure.

[0144] For example, a plurality of functions included in one unit in the above embodiments may be implemented by separate devices. Alternatively, a plurality of functions implemented by a plurality of units in the above embodiments may be implemented by separate devices, respectively. In addition, one of the above functions may be implemented by a plurality of units. Needless to say, such a configuration is included in the technical scope of the present disclosure.

[0145] In this specification, the steps described in the flowchart include not only the processing performed in time series in the order described, but also the processing performed in parallel or individually rather than necessarily in time series. In addition, even in the steps processed in time series, it goes without saying that the order can be appropriately changed.

[0146] Additionally, the technology according to the present disclosure may also be configured as follows.

[0147] Note 1. A gait recognition device, comprising:

[0148] A probability distribution acquisition unit is configured to acquire a first part probability distribution of each frame included in the first video clip, wherein the first part probability distribution indicates the probability that each pixel point of the frame relates to each body part,

[0149] a feature extraction unit configured to extract global features of the first video segment using a global feature extraction layer of a gait model, extract local features of the first video segment based on a first part probability distribution of the first video segment using a local feature extraction layer of the gait model, and fuse the global features and the local features to obtain fused features as gait features of the first video segment, and

[0150] a judging unit configured to judge whether a first predetermined object in the first video segment and a second predetermined object in the second video segment are the same person according to a gait feature of the first video segment and a gait feature of the second video segment,

[0151] Among them, the gait features of the second video clip are obtained by fusing the global features and local features of the second video clip, and the local features of the second video clip are obtained based on the second part probability distribution of each frame included in the second video clip, and the second part probability distribution indicates the probability that each pixel point of the frame involves each body part.

[0152] Note 2. The gait recognition device according to Note 1, wherein:

[0153] The probability distribution acquisition unit further acquires a first object probability distribution of each frame included in the first video segment, wherein the first object probability distribution indicates a probability that each pixel point of the frame is related to the first predetermined object.

[0154] The feature extraction unit extracts a global feature of the first video segment based on a first object probability distribution with respect to the first video segment.

[0155] Note 3. The gait recognition device according to Note 1, wherein:

[0156] The gait recognition device comprises a plurality of feature extraction units, wherein the fused features acquired by each feature extraction unit are used as inputs of the next feature extraction unit.

[0157] The fused features acquired by the last feature extraction unit are used as gait features of the first video clip.

[0158] Note 4. The gait recognition device according to Note 3, wherein:

[0159] At least one of the multiple feature extraction units is further configured to pool the fused features acquired by the feature extraction unit using a pooling layer of the gait model.

[0160] Note 5. The gait recognition device according to Note 3, wherein:

[0161] The last feature extraction unit fuses the corresponding global features and local features in a vertical splicing manner.

[0162] The remaining feature extraction units among the multiple feature extraction units fuse the corresponding global features and local features in an additive manner.

[0163] Note 6. The gait recognition device according to Note 1, wherein:

[0164] The local feature extraction layer includes a part weight acquisition sublayer and a feature extraction sublayer.

[0165] Extracting the local features includes: using the part weight acquisition sublayer to obtain the weight of each body part, obtaining a first part local feature corresponding to each body part based on the weight and the global feature, using the feature extraction sublayer to extract a second part local feature based on the first part local feature, and fusing the second part local feature to obtain the fused second part local feature as the local feature.

[0166] Note 7. The gait recognition device according to Note 6, wherein:

[0167] The first part probability distribution of each frame is represented by a plurality of two-dimensional matrices corresponding to a plurality of body parts, respectively.

[0168] The feature extraction sublayer includes a plurality of feature extraction sublayers corresponding to the plurality of body parts respectively, and the structural parameters of the plurality of feature extraction sublayers are the same.

[0169] Note 8. The gait recognition device according to any one of Notes 1 to 7, wherein:

[0170] The probability distribution acquisition unit further acquires a second part probability distribution about the second video segment, and

[0171] The feature extraction unit further extracts gait features of the second video clip.

[0172] Note 9. A gait recognition method, comprising:

[0173] Obtaining a first part probability distribution of each frame included in the first video clip, wherein the first part probability distribution indicates the probability that each pixel point of the frame relates to each body part,

[0174] extracting global features of the first video segment using a global feature extraction layer of a gait model, extracting local features of the first video segment based on a first part probability distribution of the first video segment using a local feature extraction layer of the gait model, and fusing the global features and the local features to obtain fused features as gait features of the first video segment, and

[0175] determining, based on the gait features of the first video clip and the gait features of the second video clip, whether a first predetermined object in the first video clip and a second predetermined object in the second video clip are the same person,

[0176] Among them, the gait features of the second video clip are obtained by fusing the global features and local features of the second video clip, and the local features of the second video clip are obtained based on the second part probability distribution of each frame included in the second video clip, and the second part probability distribution indicates the probability that each pixel point of the frame involves each body part.

[0177] Supplement 10. The gait recognition method according to Supplement 9 further comprises: obtaining a first object probability distribution of each frame included in the first video segment, the first object probability distribution indicating a probability that each pixel point of the frame relates to the first predetermined object,

[0178] Extracting the global feature of the first video segment includes: extracting the global feature based on a first object probability distribution about the first video segment.

[0179] Note 11. The gait recognition method according to Note 9, wherein:

[0180] The local feature extraction layer includes a part weight acquisition sublayer,

[0181] Extracting the local features includes: using the part weight acquisition sublayer to obtain the weight of each body part, obtaining the part local features corresponding to each body part based on the weight and the global features, and fusing the part local features to obtain the fused part local features as the local features.

[0182] Note 12. The gait recognition method according to Note 9, wherein:

[0183] The local feature extraction layer includes a part weight acquisition sublayer and a feature extraction sublayer.

[0184] Extracting the local features includes: using the part weight acquisition sublayer to obtain the weight of each body part, obtaining a first part local feature corresponding to each body part based on the weight and the global feature, using the feature extraction sublayer to extract a second part local feature based on the first part local feature, and fusing the second part local feature to obtain the fused second part local feature as the local feature.

[0185] Note 13. The gait recognition method according to Note 12, wherein:

[0186] The first part probability distribution of each frame is represented by a plurality of two-dimensional matrices corresponding to a plurality of body parts, respectively.

[0187] The feature extraction sublayer includes a plurality of feature extraction sublayers corresponding to the plurality of body parts respectively, and the structural parameters of the plurality of feature extraction sublayers are the same.

[0188] Note 14: The gait recognition method according to any one of Notes 9 to 13, further comprising:

[0189] obtaining a second part probability distribution for the second video clip, and

[0190] Gait features of the second video clip are extracted.

[0191] Note 15. A computer-readable storage medium storing program instructions, which is used to execute the gait recognition method described in any one of Notes 9 to 14 when the program instructions are executed by a computer.

Claims

1. A gait recognition device, comprising: A probability distribution acquisition unit is configured to acquire a first part probability distribution of each frame included in the first video clip, wherein the first part probability distribution indicates the probability that each pixel point of the frame relates to each body part, a feature extraction unit configured to extract global features of the first video segment using a global feature extraction layer of a gait model, extract local features of the first video segment based on a first part probability distribution of the first video segment using a local feature extraction layer of the gait model, and fuse the global features and the local features to obtain fused features as gait features of the first video segment, and a judging unit configured to judge whether a first predetermined object in the first video segment and a second predetermined object in the second video segment are the same person according to a gait feature of the first video segment and a gait feature of the second video segment, Among them, the gait features of the second video clip are obtained by fusing the global features and local features of the second video clip, and the local features of the second video clip are obtained based on the second part probability distribution of each frame included in the second video clip, and the second part probability distribution indicates the probability that each pixel point of the frame involves each body part.

2. The gait recognition device according to claim 1, wherein: The probability distribution acquisition unit further acquires a first object probability distribution of each frame included in the first video segment, wherein the first object probability distribution indicates a probability that each pixel point of the frame is related to the first predetermined object. The feature extraction unit extracts a global feature of the first video segment based on a first object probability distribution with respect to the first video segment.

3. The gait recognition device according to claim 1, wherein: The gait recognition device comprises a plurality of feature extraction units, wherein the fused features acquired by each feature extraction unit are used as inputs of the next feature extraction unit. The fused features acquired by the last feature extraction unit are used as gait features of the first video clip.

4. The gait recognition device according to claim 3, wherein: At least one of the multiple feature extraction units is further configured to pool the fused features acquired by the feature extraction unit using a pooling layer of the gait model.

5. The gait recognition device according to claim 3, wherein: The last feature extraction unit fuses the corresponding global features and local features in a vertical splicing manner. The remaining feature extraction units among the multiple feature extraction units fuse the corresponding global features and local features in an additive manner.

6. The gait recognition device according to claim 1, wherein: The local feature extraction layer includes a part weight acquisition sublayer and a feature extraction sublayer. Extracting the local features includes: using the part weight acquisition sublayer to obtain the weight of each body part, obtaining a first part local feature corresponding to each body part based on the weight and the global feature, using the feature extraction sublayer to extract a second part local feature based on the first part local feature, and fusing the second part local feature to obtain the fused second part local feature as the local feature.

7. The gait recognition device according to claim 6, wherein: The first part probability distribution of each frame is represented by a plurality of two-dimensional matrices corresponding to a plurality of body parts, respectively. The feature extraction sublayer includes a plurality of feature extraction sublayers corresponding to the plurality of body parts respectively, and the structural parameters of the plurality of feature extraction sublayers are the same.

8. The gait recognition device according to any one of claims 1 to 7, wherein: The probability distribution acquisition unit further acquires a second part probability distribution about the second video segment, and The feature extraction unit further extracts gait features of the second video clip.

9. A gait recognition method, comprising: Obtaining a first part probability distribution of each frame included in the first video clip, wherein the first part probability distribution indicates the probability that each pixel point of the frame relates to each body part, extracting global features of the first video segment using a global feature extraction layer of a gait model, extracting local features of the first video segment based on a first part probability distribution of the first video segment using a local feature extraction layer of the gait model, and fusing the global features and the local features to obtain fused features as gait features of the first video segment, and determining, based on the gait features of the first video clip and the gait features of the second video clip, whether a first predetermined object in the first video clip and a second predetermined object in the second video clip are the same person, Among them, the gait features of the second video clip are obtained by fusing the global features and local features of the second video clip, and the local features of the second video clip are obtained based on the second part probability distribution of each frame included in the second video clip, and the second part probability distribution indicates the probability that each pixel point of the frame involves each body part. 10 . A computer-readable storage medium storing instructions, wherein when the instructions are executed by a computer, the computer executes the gait recognition method according to claim 9 .

Images