Cross-dressing pedestrian gait recognition method and system based on feature direction compression

Abstract

The present invention discloses a cross-dressing pedestrian gait recognition method and system based on feature direction compression. The method includes obtaining a plurality of gait sequences of different dressing types, denoted as the first sequences; performing data augmentation on the dressed parts in each gait sequence to obtain the second sequences; determining the direction of the dressing change of the second sequences relative to the first sequences in the feature space; determining the weights of the fully connected layers in the gait recognition model according to the direction; obtaining the gait sequence to be recognized, and inputting the gait sequence to be recognized into the gait recognition model to obtain the cross-dressing pedestrian gait recognition result. The method of the present invention can cope with complex dressing situations in actual scenarios, and can ensure the accuracy of subsequent cross-dressing gait recognition without a large amount of training data. At the same time, the present invention statistically analyzes the direction of dressing change in the feature space, and then uniformly compresses the direction of dressing change in the feature space, having the advantages of accurate recognition results, high accuracy, and short time consumption.

Description

Technical Field

[0001] The present invention discloses a cross-dressing pedestrian gait recognition method and system based on feature direction compression, belonging to the technical field of gait recognition. Background Art

[0002] Gait is a biological and behavioral characteristic of natural persons, reflecting the variation law of body shape in space-time. Gait recognition is a method of identifying a person's identity by their walking manner. Compared with other biometric technologies, gait recognition has the advantages of non-contact, long-distance, and not being easily disguised.

[0003] Existing gait recognition methods mainly identify based on the human body contour in gait images. However, the recognition method based on the human body contour is affected by clothing, such as the thickness, length, style, etc. of the clothing, which will significantly affect the human contour, thus resulting in a low recognition ability of the recognition algorithm in the case of cross-dressing. Summary of the Invention

[0004] The purpose of this application is to provide a cross-dressing pedestrian gait recognition method and system based on feature direction compression to solve the technical problem of the low recognition ability of existing recognition algorithms in the case of cross-dressing.

[0005] The first aspect of the present invention provides a cross-dressing pedestrian gait recognition method based on feature direction compression, including:

[0006] Obtain multiple gait sequences of different clothing types, denoted as the first sequences;

[0007] Perform data augmentation on the clothing parts in each of the gait sequences to obtain second sequences;

[0008] Determine the direction of the clothing change of the second sequences relative to the first sequences in the feature space;

[0009] Determine the weights of the fully connected layer in the gait recognition model according to the direction of the clothing change in the feature space, and the direction of the clothing change in the feature space is compressed in the weights of the fully connected layer;

[0010] Obtain the gait sequence to be recognized, and input the gait sequence to be recognized into the gait recognition model to obtain the cross-dressing pedestrian gait recognition result.

[0011] Preferably, performing data augmentation on the clothing parts in each of the gait sequences specifically includes:

[0012] Perform morphological dilation processing on the clothing parts in each of the gait sequences, and the clothing parts include the upper body clothing parts and the lower body clothing parts.

[0013] Preferably, determining the direction of the dressing change of the second sequence relative to the first sequence in the feature space specifically includes:

[0014] Extracting the features of the first sequence and the features of the second sequence respectively;

[0015] According to the features of the first sequence and the features of the second sequence, determining the direction of the dressing change of the second sequence relative to the first sequence in the feature space.

[0016] Preferably, according to the features of the first sequence and the features of the second sequence, determining the direction of the dressing change of the second sequence relative to the first sequence in the feature space specifically includes:

[0017] Determining the direction of the dressing change of the second sequence relative to the first sequence in the feature space according to the first formula, and the first formula is:

[0018]

[0019] In the formula, v is the direction of the dressing change in the feature space, x i is the feature of the i-th gait sequence in the first sequence and x i ∈[x 1 , x 2 , …, x N , x' i is the feature of the i-th gait sequence in the second sequence and x' i ∈[x' 1 , x' 2 , …, x' N , x i and x' i are both n-dimensional vectors, N is the number of gait sequences, and normalize() is a normalization function.

[0020] Preferably, determining the weights of the fully connected layer in the gait recognition model according to the direction of the dressing change in the feature space specifically includes:

[0021] According to the direction of the dressing change in the feature space, constructing an orthogonal matrix by using the Schmidt orthogonalization method;

[0022] Constructing a diagonal matrix, the value of which in the direction of the dressing change in the feature space is 0, and the values in other directions are 1;

[0023] According to the orthogonal matrix and the diagonal matrix, determining the weights of the fully connected layer in the gait recognition model.

[0024] Preferably, according to the direction of the clothing change in the feature space, an orthogonal matrix is constructed by using the Schmidt orthogonalization method, which specifically includes:

[0025] Obtain n - 1 random vectors, where the random vectors are [u 1 , u 2 , …, u n-1 , and n is the feature dimension of the gait sequence;

[0026] Merge the random vectors with the direction of the clothing change in the feature space to obtain an n×n matrix, where the n×n matrix is [v, u 1 , u 2 , …, u n-1 ;

[0027] Normalize each random vector in the n×n matrix to obtain an orthogonal matrix.

[0028] Preferably, normalizing each random vector in the n×n matrix to obtain an orthogonal matrix specifically includes:

[0029] Normalize each random vector in the n×n matrix by using the second formula to obtain an orthogonal matrix, and the second formula is:

[0030]

[0031] where u i is the i-th random vector, is the transpose of u i , and normalize() is the normalization function.

[0032] Preferably, according to the orthogonal matrix and the diagonal matrix, determine the weights of the fully connected layer in the gait recognition model, which specifically includes:

[0033] Determine the weights of the fully connected layer in the gait recognition model by using the third formula, and the third formula is:

[0034] W = PΛP T

[0035] where W is the weight of the fully connected layer, P is the orthogonal matrix, P T is the transpose of the orthogonal matrix, and Λ is the diagonal matrix.

[0036] Preferably, the normalization function is as the fourth formula, and the fourth formula is:

[0037]

[0038] Where y is the parameter of the normalization function, and ||y|| is the norm of y.

[0039] The second aspect of the present invention provides a cross-dressing pedestrian gait recognition system based on feature direction compression, including:

[0040] A sequence acquisition module, which is used to acquire a plurality of gait sequences of different dressing types, denoted as the first sequence;

[0041] A data augmentation module, which is used to perform data augmentation on the dressed parts in each of the gait sequences to obtain a second sequence;

[0042] A direction determination module, which is used to determine the direction of the dressing change of the second sequence relative to the first sequence in the feature space;

[0043] A weight determination module, which is used to determine the weights of the fully connected layer in the gait recognition model according to the direction of the dressing change in the feature space, and the direction of the dressing change in the feature space is compressed in the weights of the fully connected layer;

[0044] A gait recognition module, which is used to acquire a gait sequence to be recognized, input the gait sequence to be recognized into the gait recognition model, and obtain a cross-dressing pedestrian gait recognition result.

[0045] The cross-dressing pedestrian gait recognition method and system based on feature direction compression of the present invention have the following beneficial effects compared with the prior art:

[0046] The cross-dressing pedestrian gait recognition method based on feature direction compression of the present invention uses morphological dilation processing to simulate the dressing changes of different types (upper and lower garments), which can cope with complex dressing situations in actual scenarios. Without a large amount of training data, it can ensure the accuracy of subsequent cross-dressing gait recognition. At the same time, the cross-dressing pedestrian gait recognition method based on feature direction compression of the present invention points out that the key to cross-dressing lies in the linear transformation of the gait features by the fully connected layer. Specifically, it is the compression of the corresponding matrix in the cross-dressing feature direction. Therefore, the present invention statistically analyzes the direction of dressing change in the feature space, and improves the weights of the fully connected layer according to the statistical results, so as to uniformly compress the direction of dressing change in the feature space, thereby improving the cross-dressing gait recognition ability of the gait recognition model in actual scenarios, and having the advantages of accurate recognition results, high accuracy, and short time consumption.

[0047] The cross-dressing pedestrian gait recognition system based on feature direction compression of the present invention has a simple structure and can quickly obtain accurate gait recognition results for cross-dressing. Description of the Drawings

[0048] Figure 1 It is a schematic flowchart of the cross-dressing pedestrian gait recognition method based on feature direction compression in an embodiment of the present invention;

[0049] Figure 2 It is a schematic structural diagram of the cross-dressing pedestrian gait recognition system based on feature direction compression in an embodiment of the present invention.

[0050] In the figure, 101 is a sequence acquisition module; 102 is a data enhancement module; 103 is a direction determination module; 104 is a weight determination module; 105 is a gait recognition module. Detailed implementation manners

[0051] In the following description, specific details such as specific system structures and technologies are proposed for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present invention. However, those skilled in the art should clearly understand that the present invention can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present invention.

[0052] As Figure 1 shown, the cross-dressing pedestrian gait recognition method based on feature direction compression in an embodiment of the present invention includes:

[0053] Step 1: Obtain a plurality of gait sequences of different dressing types, denoted as the first sequences.

[0054] The plurality of gait sequences in the embodiments of the present invention are gait sequences of M individuals, and each person's gait sequence includes a plurality of gait subsequences of different dressing types.

[0055] The different dressing types in the embodiments of the present invention include upper body dressing types and lower body dressing types.

[0056] Step 2: Perform data enhancement on the dressed parts in each gait sequence to obtain the second sequences, specifically including:

[0057] Perform morphological dilation processing on the dressed parts in each gait sequence, where the dressed parts include upper body dressed parts and lower body dressed parts.

[0058] Morphological dilation is an operation of finding local maxima. Specifically, the dressed parts (including upper body dressed parts and lower body dressed parts) of each gait image in the gait sequence are convolved with a kernel. The kernel can be of any shape and size, and it has a separately defined reference point (anchor point). The kernel can be regarded as a template or a mask. The kernels used in the embodiments of the present invention are 3×3 and 5×5 square kernels.

[0059] Morphological dilation is an operation of finding local maxima. The kernel is convolved with the dressed part of each gait image, that is, the maximum value of the pixel points in the area covered by the kernel is calculated, and this maximum value is assigned to the pixel specified by the reference point. In this way, the highlighted area (dressed part) in the image will gradually grow, causing changes in the dressed part in the original gait image.

[0060] In the embodiment of the present invention, morphological dilation processing is performed on the dressed part in each gait sequence, so as to simulate different types of dressing changes to cope with complex dressing situations in actual scenarios.

[0061] Step 3: Determine the direction of the dressing change of the second sequence relative to the first sequence in the feature space, specifically including:

[0062] Step 31: Extract the features of the first sequence and the features of the second sequence respectively.

[0063] In the embodiment of the present invention, the features of the first sequence and the features of the second sequence are extracted respectively, specifically by using the convolutional layer in the gait recognition model to extract the features of the first sequence and the features of the second sequence respectively.

[0064] Step 32: Determine the direction of the dressing change of the second sequence relative to the first sequence in the feature space according to the features of the first sequence and the features of the second sequence, specifically: for each type of dressing change, subtract the features of the first sequence from the features of the second sequence, and calculate the average and normalization to obtain the direction of all types of dressing changes in the feature space. More specifically:

[0065] Determine the direction of the dressing change of the second sequence relative to the first sequence in the feature space according to formula (1), which is specifically the direction of all types of dressing changes in the feature space:

[0066]

[0067] In formula (1), v is the direction of all types of dressing changes in the feature space, x i is the feature of the i-th gait sequence in the first sequence and x i ∈[x 1 , x 2 , …, x N , x' i is the feature of the i-th gait sequence in the second sequence and x' i ∈[x' 1 , x' 2 , …, x' N , x i and x' i are both n-dimensional vectors, N is the number of gait sequences, and normalize() is the normalization function.

[0068] The normalization function in the embodiments of the present invention is as shown in formula (2):

[0069]

[0070] In formula (2), y is a parameter of the normalization function, ||y|| is the norm of y, and the normalization function represents normalizing the modulus length of y.

[0071] Further, formula (1) can be written as:

[0072]

[0073] The purpose of averaging the feature differences in the embodiments of the present invention is to statistically analyze the common features of various dressing changes, and the reason for normalization is that this patent only cares about the features of dressing changes rather than the amplitude.

[0074] Step 4: Determine the weights of the fully connected layer in the gait recognition model according to the direction of the dressing change in the feature space. The weights of the fully connected layer compress the direction of the dressing change in the feature space, specifically including:

[0075] Step 41: Construct an orthogonal matrix P using the Schmidt orthogonalization method according to the direction of the dressing change in the feature space, specifically including:

[0076] Step 411: Obtain n - 1 random vectors, and the random vectors are [u 1 , u 2 , …, u n-1 , where n is the dimensionality of the feature vector of the gait sequence;

[0077] Step 412: Combine the random vectors with the direction v of the dressing change in the feature space to obtain an n×n matrix, and the n×n matrix is [v, u 1 , u 2 , …, u n-1 ;

[0078] Step 413: Normalize each random vector in the n×n matrix to obtain an orthogonal matrix, specifically including:

[0079] Using formula (3), start normalizing each random vector in the n×n matrix from u 1 to obtain the orthogonal matrix P = [v, u′ 1 , u′ 2 , …, u′ n-1 :

[0080]

[0081] In formula (3), u iis the i-th random vector, is the transpose of u i , and normalize() is a normalization function, specifically:

[0082]

[0083] where y is a parameter of the normalization function, ||y|| is the norm of y, and the normalization function represents normalizing the magnitude of y.

[0084] Furthermore, formula (3) can be written as:

[0085]

[0086] Step 42: Construct a diagonal matrix Λ, where the values of the diagonal matrix in the direction of the clothing change in the feature space are 0, and the values in other directions are 1. Specifically:

[0087]

[0088] Step 43: Determine the weights of the fully connected layer in the gait recognition model according to the orthogonal matrix and the diagonal matrix, specifically including:

[0089] Use formula (4) to determine the weights of the fully connected layer in the gait recognition model:

[0090] W = PΛP T (4)

[0091] where W is the weight of the fully connected layer, P is the orthogonal matrix, and P T is the transpose of the orthogonal matrix, and Λ is the diagonal matrix.

[0092] The purpose of constructing the orthogonal matrix in the embodiments of the present invention is to perform uncorrelated descriptions on the gait feature space. The purpose of constructing the diagonal matrix is to compress the clothing features (because the eigenvalues corresponding to the cross-clothing directions are 0) while retaining other useful features (corresponding eigenvalues are 1). The purpose of calculating the weights of the fully connected layer using formula (4) is to compress the features in the above-mentioned feature directions, thereby achieving cross-clothing recognition.

[0093] Step 5: Obtain the gait sequence to be recognized, and input the gait sequence to be recognized into the gait recognition model to obtain the cross-clothing pedestrian gait recognition result.

[0094] In the embodiments of the present invention, input the obtained gait sequence to be recognized into the gait recognition model trained using the above first sequence and second sequence, and the cross-clothing pedestrian gait recognition result can be obtained.

[0095] Cross-dressing recognition is a major challenge in gait recognition tasks. Existing gait recognition methods for cross-dressing are limited to learning from data, which requires a large amount of training data in the model training stage to ensure the accuracy of recognition results. However, a large amount of training data has the problems of time-consuming and laborious collection process and incomplete collection. Further, when existing gait recognition methods implement cross-dressing recognition, they only focus on the upper body clothing, while the clothing changes in the actual scenario are diverse (including changing lower body clothing or other types of clothing). If only the upper body clothing is considered and the lower body clothing is not considered, it will lead to the problem of low recognition accuracy.

[0096] The cross-dressing pedestrian gait recognition method based on feature direction compression of the present invention uses morphological dilation processing to simulate different types of clothing (upper body and lower body) changes, which can handle complex clothing situations in the actual scenario. Without a large amount of training data, it can ensure the accuracy of subsequent cross-dressing gait recognition. At the same time, the cross-dressing pedestrian gait recognition method based on feature direction compression of the present invention points out that the key to cross-dressing lies in the linear transformation of gait features by the fully connected layer. Specifically, it is the compression of the corresponding matrix in the cross-dressing feature direction. Therefore, the present invention statistically analyzes the direction of clothing changes in the feature space, and improves the weights of the fully connected layer according to the statistical results, so as to uniformly compress the direction of clothing changes in the feature space, thereby enhancing the cross-dressing gait recognition ability of the gait recognition model in the actual scenario, and having the advantages of accurate recognition results, high accuracy, and short time consumption.

[0097] The second aspect of the present invention provides a cross-dressing pedestrian gait recognition system based on feature direction compression, including a sequence acquisition module 101, a data enhancement module 102, a direction determination module 103, a weight determination module 104, and a gait recognition module 105;

[0098] Among them, the sequence acquisition module 101 is used to acquire multiple gait sequences of different clothing types, denoted as the first sequence;

[0099] The data enhancement module 102 is used to perform data enhancement on the clothing parts in each gait sequence to obtain the second sequence;

[0100] The direction determination module 103 is used to determine the direction of clothing changes of the second sequence relative to the first sequence in the feature space;

[0101] The weight determination module 104 is used to determine the weights of the fully connected layer in the gait recognition model according to the direction of clothing changes in the feature space, and the weights of the fully connected layer compress the direction of clothing changes in the feature space;

[0102] The gait recognition module 105 is used to acquire the gait sequence to be recognized, input the gait sequence to be recognized into the gait recognition model, and obtain the cross-dressing pedestrian gait recognition result.

[0103] The gait recognition system for cross-dressing pedestrians based on feature direction compression of the present invention has a simple structure and can quickly obtain accurate gait recognition results for cross-dressing.

[0104] The above are only several embodiments of the present application, and do not impose any form of limitation on the present application. Although the present application is disclosed above with preferred embodiments, it is not intended to limit the present application. Any person skilled in the art, without departing from the scope of the technical solution of the present application, makes some changes or modifications using the technical content disclosed above, which are equivalent to equivalent implementation cases and all fall within the scope of the technical solution.

Claims

1. A cross-dressing pedestrian gait recognition method based on feature direction compression, characterized in that, it includes: Obtain multiple gait sequences of different dressing types, denoted as the first sequences; Perform data augmentation on the dressed parts in each of the gait sequences to obtain second sequences; Determine the direction of the dressing change in the second sequence relative to the first sequence in the feature space; Determine the weights of the fully connected layer in the gait recognition model according to the direction of the dressing change in the feature space, and the weights of the fully connected layer compress the direction of the dressing change in the feature space; Obtain the gait sequence to be recognized, input the gait sequence to be recognized into the gait recognition model, and obtain the cross-dressing pedestrian gait recognition result; Determine the direction of the dressing change in the second sequence relative to the first sequence in the feature space, specifically including: Extract the features of the first sequence and the features of the second sequence respectively; According to the features of the first sequence and the features of the second sequence, determine the direction of the dressing change in the second sequence relative to the first sequence in the feature space, specifically including: Determine the direction of the dressing change in the second sequence relative to the first sequence in the feature space according to the first formula, and the first formula is: where \(v\) is the direction of the clothing change in the feature space, \(x\) i is the feature of the \(i\)-th gait sequence in the first sequence and \(x\) i \(\in [x\) 1 , x\) 2 , \(\cdots, x\) N , \(x'\) i is the feature of the \(i\)-th gait sequence in the second sequence and \(x'\) i \(\in [x'\) 1 , x'\) 2 , \(\cdots, x'\) N , \(x\) i and \(x'\) i are both \(n -\)dimensional vectors, \(N\) is the number of gait sequences, and \(normalize()\) is a normalization function; Determine the weights of the fully connected layer in the gait recognition model according to the direction of the dressing change in the feature space, specifically including: Construct an orthogonal matrix using the Schmidt orthogonalization method according to the direction of the dressing change in the feature space; Construct a diagonal matrix, the value of the diagonal matrix in the direction of the dressing change in the feature space is 0, and the values in other directions are 1; Determine the weights of the fully connected layer in the gait recognition model according to the orthogonal matrix and the diagonal matrix.

2. The cross-dressing pedestrian gait recognition method based on feature direction compression according to claim 1, characterized in that, Performing data augmentation on the dressed parts in each of the gait sequences specifically includes: Perform morphological dilation processing on the dressed parts in each of the gait sequences, and the dressed parts include upper body dressed parts and lower body dressed parts.

3. The cross-dressing pedestrian gait recognition method based on feature direction compression according to claim 1, characterized in that, Construct an orthogonal matrix using the Schmidt orthogonalization method according to the direction of the dressing change in the feature space, specifically including: Obtain n-1 random vectors, where the random vectors are [u 1 , u 2 , …, u n-1 , and n is the feature dimension of the gait sequence; Merge the random vector with the direction of the clothing change in the feature space to obtain an n×n matrix, where the n×n matrix is [v, u 1 , u 2 , …, u n-1 ; Normalize each random vector in the \(n\times n\) matrix to obtain an orthogonal matrix.

4. The cross-dressing pedestrian gait recognition method based on feature direction compression according to claim 3, characterized in that, Normalize each random vector in the \(n\times n\) matrix to obtain an orthogonal matrix, specifically including: Normalize each random vector in the \(n\times n\) matrix using the second formula to obtain an orthogonal matrix, and the second formula is: where u i is the i-th random vector, is the transpose of the said u i , and normalize() is a normalization function.

5. The cross-dressing pedestrian gait recognition method based on feature direction compression according to claim 4, characterized in that, Determine the weights of the fully connected layer in the gait recognition model according to the orthogonal matrix and the diagonal matrix, specifically including: Determine the weights of the fully connected layer in the gait recognition model using the third formula, and the third formula is: W = P Λ P T Where W is the weight of the fully connected layer, P is the orthogonal matrix, and P T is the transpose of the orthogonal matrix, and Λ is the diagonal matrix.

6. The cross-dressing pedestrian gait recognition method based on feature direction compression according to claim 4, wherein, the normalization function is as shown in the fourth formula, and the fourth formula is: In the formula, y is a parameter of the normalization function, and ||y|| is the norm of y.

7. A cross-dressing pedestrian gait recognition system based on feature direction compression using the cross-dressing pedestrian gait recognition method according to any one of claims 1-6, wherein, it includes: a sequence acquisition module for acquiring a plurality of gait sequences of different dressing types, denoted as the first sequence; a data augmentation module for performing data augmentation on the dressed parts in each of the gait sequences to obtain a second sequence; a direction determination module for determining the direction of the dressing change of the second sequence relative to the first sequence in the feature space; a weight determination module for determining the weights of the fully connected layer in the gait recognition model according to the direction of the dressing change in the feature space, and the weights of the fully connected layer compress the direction of the dressing change in the feature space; a gait recognition module for acquiring a gait sequence to be recognized, inputting the gait sequence to be recognized into the gait recognition model, and obtaining a cross-dressing pedestrian gait recognition result.

Images