A method for individual identification based on frontal perspective gait data

Abstract

The present invention discloses an individual identification method based on frontal perspective gait data, comprising the following steps: collecting a human body frontal perspective contour sequence; extracting three types of time-varying data and curves, namely, area features, length features, and angle features of the human body lower limb contour under the frontal perspective; constructing radial basis function neural networks to extract gait dynamics features under the three types of time-varying gait data, and using a deep transfer learning mechanism to perform secondary feature extraction; using the differences in gait dynamics between different individuals, calculating the feature fusion weights of the three types of features after secondary feature extraction; and inputting the weighted fused features into a fully connected network for training and classification. The present invention adopts a combination of gait dynamics features and a deep transfer learning mechanism to perform transfer feature learning and intelligent fusion on gait data under the frontal perspective, which is simple, convenient, and easy to operate, and can effectively improve the recognition rate under the frontal perspective.

Description

Technical Field

[0001] The present invention belongs to the technical field of pattern recognition, and in particular relates to an individual identity recognition method based on frontal perspective gait data. Background Art

[0002] Gait recognition, as an emerging biometric recognition technology, has incomparable advantages such as uniqueness, non-contact and long-distance, and can be widely used in video surveillance. In specific monitoring scenarios, existing gait recognition algorithms rely heavily on side view data, most of which start from finding side view gait features and side view image processing. Although many valuable works have emerged, these methods are limited by the site problems caused by side view data collection, and often require a large data collection area. This is also one of the important reasons why gait recognition technology cannot be applied in practice. If the human gait movement and its characteristic information can be extracted and modeled based on the front view gait data, and the extraction and modeling results can be used as gait labels for gait recognition, it will obviously provide great help to the practical process of gait recognition technology. To this end, we propose an individual identity recognition method based on front view gait data to solve the problems mentioned in the above background technology. Summary of the invention

[0003] The purpose of the present invention is to provide a method for individual identity recognition based on frontal perspective gait data to solve the problems raised in the above background technology.

[0004] To achieve the above object, the present invention provides the following technical solution: a method for identifying an individual based on frontal perspective gait data, comprising the following steps:

[0005] Step 1: Obtain a human walking gait contour map, and extract three types of time-varying data and curves, namely, area features, length features, and angle features of the human lower limb contour from a frontal perspective;

[0006] The walking image sequence from the frontal perspective is subjected to background subtraction and morphological processing to obtain the binary gait contour map of the human body. The binary human body contour is divided into three equal-height sub-regions from top to bottom according to the height of the human body contour, and the bottom region is selected as the region of interest. The contour area features, length features and angle features of the region of interest of each frame are extracted. The contour area ratio of the lower limbs and legs is extracted for each frame. Difference in area of ​​lower limb contours The triangle area of ​​the lower limbs and legs Composed gait area data. Extract the vertical length ratio of the lower limbs and legs to the ground in each frame Length of the line connecting the lower extreme points of the lower legs The length difference between the lower extreme point of the lower leg contour and the midpoint of the upper boundary line The gait length data is composed. The angle between the line connecting the lower extreme points of the lower limbs and the ground is extracted for each frame. The vertical angle between the lower extreme point of the left leg and the midpoint of the upper limit line The vertical angle between the lower extreme point of the right leg and the midpoint of the upper limit line The gait angle data is formed. The periodic or quasi-periodic variation curves of three types of time-varying data, namely, area characteristics, length characteristics and angle characteristics of the human lower limb contour under the frontal perspective, are obtained.

[0007] Step 2: construct a radial basis function (RBF) neural network to approximate and identify the nonlinear gait dynamics information inherent in the three types of gait time-varying data, and store it in a constant neural network weight matrix;

[0008] For each type of time-varying data mentioned above, a nonlinear dynamic model is established. Where x=[x1,···,x n ] T ∈R n are the three extracted area / length / angle features, p is the system constant parameter value; F(x; p) = [f1(x; p), ···, f n (x; p)] T represents the intrinsic nonlinear gait dynamics term, v(x;p)=[v1(x;p),…,v n (x; p)] T is the introduced modeling uncertainty; the modeling uncertainty v(x; p) and the gait nonlinear dynamics term F(x; p) are combined into one term: And defined as the total gait nonlinear dynamics term; constructing an RBF neural network under a deterministic learning mechanism to identify nonlinear gait dynamics Select the converged weights and take the numerical average to obtain the weights of the corresponding neural network constants. Stored as a constant neural network weight matrix.

[0009] Step 3: Use the three types of gait dynamics features as inputs of the pre-trained deep learning neural network model, perform transfer feature learning, complete secondary feature extraction, and perform adaptive weighted fusion and splicing of the features extracted by the three single networks;

[0010] The three types of gait dynamics features are respectively input into one or more of the pre-trained deep learning neural network models Inception-v3, Resnet152 and Inception-Resnet-v2, and transfer feature learning is performed in them alone or in combination to complete secondary feature extraction. In the pattern training phase, the corresponding data of the test pattern is compared with the obtained secondary data features to generate a set of recognition errors in the form of L1 norm, which is used as a standard to measure the weight proportion of each secondary feature in the training gait pattern. The recognition error value is in the interval [-10,10], and the weight is 80; the recognition error value is in the interval [-20,-10) and (10,20], and the weight is 60; the recognition error value is in the remaining intervals, and the weight is 20. According to the obtained weight value, the secondary features obtained by the pre-training model are spliced: Q = a × length data transfer learning feature value + b × area data transfer learning feature value + c × angle data transfer learning feature value. Among them, a, b, c are weights, and Q is the spliced ​​gait feature.

[0011] Step 4: Construct a fully connected network to accurately classify and identify the test pattern based on the differences in nonlinear dynamic characteristics between the training pattern and the test pattern, and realize gait recognition from a frontal perspective.

[0012] The spliced ​​gait features are used to represent different walking individuals. According to the differences between the training mode and the test mode, a two-layer fully connected network, a five-layer fully connected network and a seven-layer fully connected network are used to accurately classify and identify the test mode according to the majority voting method of the recognition results, thereby realizing gait recognition from a frontal perspective.

[0013] Compared with the prior art, the present invention has the following beneficial effects:

[0014] 1. Compared with the traditional gait recognition method from the side perspective, the present invention uses gait data from the front perspective for modeling and analysis. Therefore, in practical applications, a larger data collection area is not required, and it can be more practical and operable in specific monitoring scenarios such as entrance access.

[0015] 2. Use nonlinear neural network recognition technology to directly approximate and identify the unknown nonlinear gait system dynamics under the human body contour shape information, and use deep transfer learning mechanism for secondary feature extraction, so that the proposed features are more distinguishable between different individuals, thereby further improving the recognition rate of gait recognition.

[0016] 3. The weighted fusion splicing mechanism is used to splice and fuse the deep transfer learning features, which has the advantages of simple calculation, low feature dimension, small amount of calculation, and fast speed. It has also achieved good results in practice, and as more data is added to the database, its recognition effect will continue to improve. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a flow chart of an individual identity recognition method based on frontal perspective gait data proposed by the present invention;

[0018] Figure 2 2 is a schematic diagram of human body contour parameter extraction in an embodiment, in which the human body contour image is divided into three areas from top to bottom according to the image height value;

[0019] Figure 3 is a schematic diagram of human body area feature extraction in an embodiment;

[0020] Figure 4 is a schematic diagram of human body length feature extraction in an embodiment;

[0021] Figure 5 Schematic diagram of human body angle feature extraction in the embodiment. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] Example:

[0024] The present invention provides Figure 1 The method for identifying an individual based on frontal perspective gait data shown in the figure comprises the following steps:

[0025] Step 1: Obtain a human walking gait contour map, and extract three types of time-varying data and curves: area features, length features, and angle features of the human lower limb contour from a frontal perspective;

[0026] The gait database used in this embodiment is the CASIA-B gait database. The database contains a total of 124 people, each of whom has 6 different frontal perspective gait sequences. This embodiment uses four of the sequences as training modes and the remaining two sequences as test modes. The present invention performs background subtraction on the walking videos of each person in the training set and the test set in turn, performs morphological processing on the gait images that have been separated from the background, and obtains a binary human body contour map.

[0027] like Figure 2As shown in the figure, the binary human body contour is divided into three equal height sub-regions from top to bottom according to the height of the human body contour, and the bottom region is selected as the region of interest. The contour area features, length features and angle features of the region of interest in each frame are extracted. The number of pixels in each region is defined as the area value of the region, such as Figure 3 As shown, the ratio of the lower limbs and legs contour area of ​​each frame is extracted Difference in area of ​​lower limb contours The triangle area of ​​the lower limbs and legs Composition of gait area data. Figure 4 As shown, the ratio of the vertical length of the lower limbs to the ground is extracted for each frame. Length of the line connecting the lower extreme points of the lower legs The length difference between the lower extreme point of the lower leg contour and the midpoint of the upper boundary line Composed of gait length data. Figure 5 The angle between the lower extreme points of the lower limbs and the ground is extracted as shown in the figure The vertical angle between the lower extreme point of the left leg and the midpoint of the upper limit line The vertical angle between the lower extreme point of the right leg and the midpoint of the upper limit line The gait angle data is formed. The periodic or quasi-periodic variation curves of three types of time-varying data, namely, area characteristics, length characteristics and angle characteristics of the human lower limb contour under the frontal perspective, are obtained.

[0028] Step 2: construct a radial basis function (RBF) neural network to approximate and identify the nonlinear gait dynamics information inherent in the three types of gait time-varying data, and store it in a constant neural network weight matrix;

[0029] For each type of time-varying data mentioned above, a nonlinear dynamic model is established. Where x=[x1,···,x n ] T ∈R n are the three extracted area / length / angle features, p is the system constant parameter value; F(x; p) = [f1(x; p), ···, f n (x; p)] T represents the intrinsic nonlinear gait dynamics term, v(x;p)=[v1(x;p),…,v n (x; p)] T is the introduced modeling uncertainty; the modeling uncertainty v(x; p) and the gait nonlinear dynamics term F(x; p) are combined into one term: And defined as the total gait nonlinear dynamics term. Design the following neural network identifier:

[0030]

[0031] in, is the state variable of the RBF neural network identifier; A = diag[0.5,…,0.5] is a diagonal matrix, a i =0.5 is the designed constant; It is a dynamic RBF neural network used to approximate the unknown nonlinear dynamics of gait S(x)=[S1(||X-ξ1||,…,S N (||X-ξ n ||] T is a Gaussian radial basis function; ξ i is the center point of the neuron; RBF neurons are evenly distributed in the area [-1,1]×[-1,1]×[-1,1] and the width is 0.15. The adjustment law of the RBF neural network weights is as follows:

[0032]

[0033] in, is the state error, Г i =1.5,σ i =10 is the adjustment parameter of the adjustment law, and the weight of the dynamic RBF neural network Initial value of

[0034] The local accurate modeling of the nonlinear dynamics of gait is expressed as follows:

[0035]

[0036] Among them, ε i1 is the approximation error. Local accurate modeling is the approximation of the intrinsic nonlinear dynamics along the three-dimensional gait profile parameter trajectory by the RBF neural network, while the nonlinear dynamic terms far from the trajectory are not approximated. The approximation results are stored in the constant neural network weight matrix, that is, the weights converge to the optimal constant value within a period of time, and the converged weights are selected to take the numerical average to obtain the corresponding neural network constant weights.

[0037]

[0038] Among them, [t a ,t b ] represents a period of time after the neural network weights complete the transition process of converging to their optimal values, so that Constant-valued neural network Make a locally exact approximation:

[0039]

[0040] Among them, ε i2 is the approximation error. The nonlinear gait dynamics characteristics are expressed in the form of a constant neural network weight matrix. for storage.

[0041] Step 3: Use the three types of gait dynamics features as the input of the pre-trained deep learning neural network model, perform transfer feature learning, complete secondary feature extraction, and perform adaptive weighted fusion and splicing of the features extracted by the three single networks;

[0042] The three types of gait dynamics features are input into one or more of the pre-trained deep learning neural network models Inception-v3, Resnet152 and Inception-Resnet-v2, and transfer feature learning is performed in them alone or in combination to complete secondary feature extraction. In the pattern training phase, the data corresponding to the test pattern is compared with the obtained secondary data features to generate a set of recognition errors in the form of L1 norm, which is used as a standard to measure the weight proportion of each secondary feature in the training gait pattern. The recognition error value is in the range of [-10,10], and the weight is 80; the recognition error value is in the range of [-20,-10) and (10,20], and the weight is 60; the recognition error value is in the rest of the range, and the weight is 20.

[0043] In this embodiment, according to the gait data of the CASIA-B database and the obtained weight value, the secondary features obtained by the pre-training model are spliced: Q = 80×length data transfer learning feature value + 20×area data transfer learning feature value + 60×angle data transfer learning feature value. Q is the spliced ​​gait feature.

[0044] Step 4: Construct a fully connected network to accurately classify and identify the test pattern based on the differences in nonlinear dynamic characteristics between the training pattern and the test pattern, and realize gait recognition from a frontal perspective.

[0045] The concatenated gait features are used to characterize different walking individuals. According to the difference between the training mode and the test mode, a two-layer fully connected network, a five-layer fully connected network, and a seven-layer fully connected network are used, and the following loss function is used:

[0046]

[0047] The test pattern is accurately classified and identified according to the majority voting method of the recognition results, and gait recognition from a frontal perspective is achieved. In the recognition experiment on the CASIA-B database, the recognition efficiency can reach up to 93%.

[0048] To sum up, compared with the prior art, 1. Compared with the traditional gait recognition method from the side perspective, the present invention uses gait data from the front perspective for modeling and analysis. Therefore, in practical applications, a larger data collection area is not required, and it can be more practical and operational in specific monitoring scenarios such as entrance access.

[0049] 2. Use nonlinear neural network recognition technology to directly approximate and identify the unknown nonlinear gait system dynamics under the human body contour shape information, and use deep transfer learning mechanism for secondary feature extraction, so that the proposed features are more distinguishable between different individuals, thereby further improving the recognition rate of gait recognition.

[0050] 3. The weighted fusion splicing mechanism is used to splice and fuse the deep transfer learning features, which has the advantages of simple calculation, low feature dimension, small amount of calculation, and fast speed. It has also achieved good results in practice, and as more data is added to the database, its recognition effect will continue to improve.

[0051] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for identifying an individual based on frontal perspective gait data, characterized in that: The steps include: Step 1: Obtain a human walking gait contour map, and extract three types of time-varying data and curves, namely, area features, length features, and angle features of the human lower limb contour from a frontal perspective; In step 1, the method for extracting time-varying data of human body contour under the frontal perspective is: performing background subtraction and morphological processing on the walking image sequence under the frontal perspective to obtain a binary gait contour image of human body walking; Divide the binary human body contour into three equal height sub-regions from top to bottom according to the height of the human body contour, and select the bottom region as the region of interest; Extract three types of time-varying data and curves, namely, contour area features, length features and angle features of the region of interest in each frame; Extract the ratio of the lower limbs and legs contour area in each frame , Difference in the contour area of ​​the lower limbs and legs , the triangle area of ​​the lower limbs and legs , forming the gait area data; Extract the vertical length ratio of the lower limbs to the ground of each frame , Length of the line connecting the lower extreme points of the lower legs , the length difference between the lower extreme point of the lower leg contour and the midpoint of the upper boundary line , constituting the gait length data; Extract the angle between the lower extreme points of the lower limbs and the ground in each frame , the vertical angle between the lower extreme point of the left leg and the midpoint of the upper boundary line , the vertical angle between the lower extreme point of the right leg and the midpoint of the upper boundary line , forming gait angle data; Obtain the periodic or quasi-periodic variation curves of three types of time-varying data, namely, area characteristics, length characteristics and angle characteristics of the human lower limb contour under the frontal perspective; Step 2: construct a radial basis function neural network to approximate and identify the nonlinear gait dynamics information inherent in the three types of gait time-varying data using a neural network, and store it in a constant value neural network weight matrix; In step 2, the nonlinear gait dynamics information inherent in the three types of gait time-varying data is identified by neural network. The specific method is: for each type of time-varying data mentioned above, its nonlinear dynamics model is established respectively. ; in, are the three extracted area, length or angle features, is the constant parameter value of the system; represents the intrinsic nonlinear gait dynamics term, is the modeling uncertainty introduced; Modeling uncertainty and the nonlinear dynamics of gait Combined into one: , and defined as the total gait nonlinear dynamics term; an RBF neural network under a deterministic learning mechanism is constructed to identify nonlinear gait dynamics. The nonlinear gait dynamics characteristics are expressed in the form of a constant neural network weight matrix Store, select the converged weights and take the numerical average to obtain the weights of the corresponding neural network constants ; Step 3: Use the three types of gait dynamics features as inputs of the pre-trained deep learning neural network model, perform transfer feature learning, complete secondary feature extraction, and perform adaptive weighted fusion and splicing of the features extracted by the three single networks; In step 3, the transfer feature learning method is as follows: the three types of gait dynamics features are respectively input into one or more of the pre-trained deep learning neural network models Inception-v3, Resnet152 and Inception-Resnet-v2, and transfer feature learning is performed in them alone or in combination to complete secondary feature extraction; In the pattern training phase, the data corresponding to the test pattern is compared with the obtained secondary data features to generate a set of recognition errors in the form of L1 norm, which is used as a standard to measure the weight proportion of each secondary feature in the training gait pattern; When the recognition error is in the interval [-10, 10], the weight is 80; when the recognition error is in the interval [-20, -10) and (10, 20], the weight is 60; when the recognition error is in the rest of the intervals, the weight is 20; According to the obtained weight value, the secondary features obtained by the pre-training model are spliced: Q = a × length data transfer learning feature value + b × area data transfer learning feature value + c × angle data transfer learning feature value, where a, b, c are weights, and Q is the spliced ​​gait feature; Step 4: Construct a fully connected network to accurately classify and identify the test pattern based on the differences in nonlinear dynamic characteristics between the training pattern and the test pattern, and realize gait recognition from a frontal perspective.

2. The method for identifying an individual based on frontal perspective gait data according to claim 1, characterized in that: In step four, a fully connected network is constructed to perform gait recognition from a frontal perspective. Specifically, the spliced ​​gait features are used to characterize different walking individuals. According to the differences between the training mode and the test mode, a two-layer fully connected network, a five-layer fully connected network, and a seven-layer fully connected network are used. The test mode is accurately classified and identified by majority voting according to the recognition results, thereby realizing gait recognition from a frontal perspective.

Images