A visual gait recognition method and system based on deterministic learning

Abstract

The present invention proposes a visual gait recognition method and system based on deterministic learning. The method collects a gait sequence consisting of multiple frames, extracts three-dimensional gait variables from each frame of the gait sequence, forms a three-dimensional gait variable sequence, and plots a three-dimensional gait phase space trajectory. The three-dimensional gait variable sequence is input into a trained RBF neural network to obtain a gait dynamic feature sequence and time-invariant knowledge. The time-invariant knowledge is used to construct a gait pattern library, and the dynamic trajectory is plotted. The three-dimensional gait variable sequence of the gait sequence to be identified is extracted, and a state estimator is constructed using the time-invariant knowledge in the gait pattern library. The dynamics of the gait sequence to be identified are quickly compared with the dynamics of each gait in the gait pattern library to obtain the final gait recognition result. The present invention can accurately reflect the spatiotemporal characteristics of each individual, keenly capture changes in gait patterns, improve the accuracy of gait recognition, and has strong robustness across perspectives.

Description

Technical Field

[0001] The present invention belongs to the field of biometric recognition, and in particular relates to a visual gait recognition method and system based on deterministic learning. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Gait is a typical behavioral characteristic that describes the way an individual walks and can be used to identify individuals. Vision-based gait recognition methods are currently a hot topic of research. Using visual acquisition devices such as cameras, it is possible to capture a subject's gait from a distance without making contact with the subject. Therefore, vision-based gait recognition technology can be used for remote identification and other non-contact applications. Furthermore, numerous studies have demonstrated the important application value of gait in the diagnosis, rehabilitation assessment, and treatment efficacy analysis of degenerative diseases.

[0004] Vision-based gait recognition uses a camera to obtain individual gait videos or sequences, which can be roughly divided into two categories: contour-based and model-based methods. Contour-based methods usually establish an effective representation for gait sequences and directly extract high-dimensional gait features with discriminative capabilities from them. Model-based methods tend to combine knowledge of human anatomy, fit gait sequences with walking models, and extract effective gait features from the models.

[0005] Due to the lack of effective representation of gait patterns, most methods only use statistical methods or Fourier transform methods to extract time-invariant gait features from gait sequences or gait models. However, this transformation loses the effective temporal relationship between gaits and makes it difficult to explain the recognition results.

[0006] The spatiotemporal relationship of gait plays an important role in gait recognition systems. In recent years, Wang et al. proposed a deterministic learning method for accurately identifying dynamic systems and, based on this, proposed a rapid recognition mechanism for dynamic patterns. Zeng et al., based on the deterministic learning method, used spatiotemporal features extracted from a five-link biped model to propose a model-based dynamic recognition method. Deng et al. further analyzed the five-link biped model and proved that changes in lower limb joint angles encompass most gait dynamics. However, the feature dimensions selected by the above methods are still relatively high, making it difficult to intuitively display the changing process of gait feature trajectories, and even more difficult to visualize the gait modeling process and recognition results. Summary of the Invention

[0007] To overcome the shortcomings of the above-mentioned existing technologies, the present invention provides a visual gait recognition method and system based on deterministic learning. The method extracts three-dimensional gait variables reflecting the movement of the left and right limbs from a five-link model. According to deterministic learning theory, the method accurately identifies the gait dynamics characteristics locally along the three-dimensional gait phase space trajectory, obtains a time-invariant representation of the gait dynamics characteristics, and displays the gait dynamics in the visual space. The method can accurately reflect the spatiotemporal characteristics of each individual, keenly capture changes in gait patterns, improve the accuracy of gait recognition, and has strong robustness across perspectives.

[0008] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0009] A first aspect of the present invention provides a visual gait recognition method based on deterministic learning;

[0010] A visual gait recognition method based on deterministic learning, comprising:

[0011] Collect a gait sequence consisting of multiple frames, extract three-dimensional gait variables from each frame of the gait sequence, form a three-dimensional gait variable sequence, and draw a three-dimensional gait phase space trajectory;

[0012] The three-dimensional gait variable sequence is input into the RBF neural network, and the gait dynamic feature sequence and time-invariant knowledge are obtained according to the deterministic learning. The time-invariant knowledge is used to construct a gait pattern library and draw the dynamic trajectory;

[0013] The three-dimensional gait variable sequence of the gait sequence to be identified is extracted, and a state estimator is constructed using the time-invariant knowledge in the gait pattern library. The dynamics of the gait sequence to be identified are quickly compared with the dynamics of each gait in the gait pattern library to obtain the final gait recognition result.

[0014] Furthermore, the three-dimensional gait variables are: the angle between the left thigh and the vertical direction, the angle between the left calf and the vertical direction, and the relative distance between the knee joints.

[0015] Furthermore, in each frame of the gait sequence, a five-link bipedal model is used to characterize human motion.

[0016] Furthermore, the five-link biped model is represented by the Lagrange equations, and the limb angle vector is θ = [θ1, θ2, θ3, θ4, θ5] T ;

[0017] Among them, θ1, θ2, θ3, θ4, and θ5 are the angles between the left calf, left thigh, human trunk, right thigh, and right calf and the vertical direction respectively;

[0018] Extract θ1 and θ2 from the five-link biped model as the angle between the left calf and the vertical direction and the angle between the left thigh and the vertical direction in the three-dimensional gait variables.

[0019] Furthermore, based on the triangular constraint between the distance l between the knee joints and the left and right thigh lengths l2 and l4 in the five-link biped model, the distance l between the knee joints is calculated according to the law of cosines.

[0020] Furthermore, the main steps of the gait dynamics feature sequence extraction process are as follows:

[0021] Construct RBF neural network;

[0022] The RBF neural network is used to perform dynamic modeling on the training gait pattern and obtain the gait dynamic feature sequence;

[0023] Initialize all neural network weights to 0 and update them according to the deterministic learning theory until they converge to the optimal value, obtaining the timely invariant knowledge of the gait dynamics feature sequence;

[0024] The learned time-invariant knowledge is used to construct a gait pattern library.

[0025] Furthermore, a dynamic pattern rapid recognition method based on a deterministic learning mechanism is used to achieve rapid gait recognition. The specific steps are as follows:

[0026] A series of dynamic estimators are constructed using the time-invariant knowledge in the gait pattern library;

[0027] extracting a three-dimensional gait variable sequence of a gait sequence to be identified;

[0028] According to the three-dimensional gait variable sequence and the dynamic estimator, a residual system representing the difference between the gait to be identified and the gait pattern is obtained;

[0029] The average L1 norm of the estimation error, which is proportional to the difference between gait patterns, is calculated, and the gait pattern with the smallest error is the recognition result.

[0030] A second aspect of the present invention provides a visual gait recognition system based on deterministic learning.

[0031] A visual gait recognition system based on deterministic learning, including a three-dimensional gait variable extraction module, a gait dynamic feature extraction module and a fast gait recognition module;

[0032] The three-dimensional gait variable extraction module is configured to: collect a gait sequence consisting of multiple frames, extract three-dimensional gait variables from each frame of the gait sequence, form a three-dimensional gait variable sequence, and draw a three-dimensional gait phase space trajectory;

[0033] The gait dynamics feature extraction module is configured to: input the three-dimensional gait variable sequence into the trained RBF neural network to obtain the gait dynamics feature sequence and time-invariant knowledge, construct a gait pattern library based on the time-invariant knowledge, and draw the dynamic trajectory;

[0034] The fast gait recognition module is configured to: extract the three-dimensional gait variable sequence of the gait sequence to be recognized, use the time-invariant knowledge in the gait pattern library to build a state estimator, and quickly compare the dynamics of the gait sequence to be recognized with the dynamics of each gait in the gait pattern library to obtain the final gait recognition result.

[0035] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of the visual gait recognition method based on deterministic learning as described in the first aspect of the present invention.

[0036] The fourth aspect of the present invention provides an electronic device, comprising a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps of a visual gait recognition method based on deterministic learning as described in the first aspect of the present invention are implemented.

[0037] One or more of the above technical solutions have the following beneficial effects:

[0038] The three-dimensional gait variable sequence extracted from a multi-frame gait sequence can accurately reflect the spatiotemporal characteristics of each individual and keenly capture changes in gait patterns. In particular, the relative distance between knee joints, as a gait motion variable, retains gait dynamic and structural information, improving the accuracy of gait recognition.

[0039] Benefiting from the natural visualization properties of three-dimensional variables, the present invention provides an intuitive display of three-dimensional gait variables and gait dynamic characteristics in a visualization space, which can intuitively describe the individual's gait movement and the implied gait change information;

[0040] Compared with existing methods, the method provided by the present invention uses fewer features to achieve sufficiently good recognition results and has strong robustness across viewing angles. Although the feature dimension is reduced, it does not cause a decrease in recognition performance. The three-dimensional gait variable sequence selected by the present invention has extremely strong resolution ability. At the same time, the visual analysis process provides a powerful tool for gait analysis.

[0041] Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0043] Figure 1 A flow chart of the method of the first embodiment;

[0044] Figure 2 It is a five-link bipedal model of the human body;

[0045] Figure 3 is the periodic change curve of three-dimensional gait variables;

[0046] Figure 4 is the three-dimensional gait phase space trajectory;

[0047] Figure 5 is the dynamic trajectory;

[0048] Figure 6 The three-dimensional gait variable sequence, three-dimensional gait phase space trajectory and dynamic trajectory of different individuals;

[0049] Figure 7 Identify residual curves for gait;

[0050] Figure 8 is the gait state curve;

[0051] Figure 9 To test gait;

[0052] Figure 10 Comparison of the three-dimensional gait phase space trajectory and dynamic trajectory of all gaits of object e;

[0053] Figure 11 This is a system structure diagram of the second embodiment. DETAILED DESCRIPTION

[0054] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present disclosure belongs.

[0055] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0056] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0057] Example 1

[0058] This embodiment discloses a visual gait recognition method based on deterministic learning;

[0059] like Figure 1 As shown, a visual gait recognition method based on deterministic learning includes:

[0060] S1: Collect a gait sequence consisting of multiple frames, extract three-dimensional gait variables from each frame of the gait sequence, form a three-dimensional gait variable sequence, and draw a three-dimensional gait phase space trajectory;

[0061] Microsoft Kinect is used to collect multi-frame gait sequences, each frame including the coordinates of 21 joint points. The coordinates of these 21 joint points are used to construct a five-link biped model.

[0062] The three-dimensional gait variables extracted from the five-link biped model directly reflect the motion characteristics of the gait in the visualization space. The motion of the human lower limbs can be simplified into a five-link biped model, including: trunk, thigh and calf; each limb in the model can be regarded as a rigid body with restricted motion, such as Figure 2 As shown, the limb movement angle θ i (i=1,2,3,4,5) reflects most of the gait dynamics. The five-link biped model can be expressed by the following Lagrangian equation:

[0063]

[0064] Among them, θ = [θ1, θ2, θ3, θ4, θ5] T represents the limb angle vector, θ1, θ2, θ3, θ4, θ5 are the angles between the left calf, left thigh, human trunk, right thigh, right calf and the vertical direction respectively; D(θ) is a 5×5 positive definite inertia matrix; H(θ) is a 5×5 eccentric Coriolis matrix; G(θ), Tθ ,θ, They are 5×1 gravity matrix, generalized torque matrix, generalized coordinate matrix, velocity matrix and acceleration matrix respectively; the human body trunk remains basically unchanged during normal walking, so it can be considered that θ3=0,

[0065] The relative distance between the knee joints, as a gait motion variable, preserves gait dynamic and structural information. In the five-link bipedal model, there are geometric constraints between limb angles and lengths. In particular, there is a strict triangular constraint between the distance l between the knee joints and the left and right thigh lengths l2 and l4. According to the law of cosines, we can know that: θ2 and θ4 satisfy 0≤θ2+θ4<π.

[0066] Among them, θ2 and θ4 represent the left thigh angle and the right thigh angle respectively; for a single individual, l2 and l4 remain unchanged, let k2=2l2l4, and we can deduce that:

[0067] θ2=arccos(k1-1 / k2)-θ4 (2)

[0068] In θ1+θ2∈[0,π), θ2 can be uniquely determined by l and θ4, and vice versa.

[0069] Formula 1 can be transformed into Substitute Equation 2 into the transformed Equation 1 and assume that φ = [θ1, l, θ4, θ5] T , we can get a simplified representation of the five-link biped model:

[0070]

[0071] Among them, D(φ) and H(φ) are both 4×4 matrices; G(φ), T φ , φ are 4×1 matrices respectively. Therefore, the gait dynamics can be expressed as a nonlinear function of the lower limb motion φ, that is, the lower limb angle and knee joint relative distance parameters reflect the main information of gait dynamics.

[0072] Gait changes between gait sequence frames can be represented by the movement of only one limb. The angles of the lower limbs are similar, satisfying θ2(t) = θ4(Tt) and θ1(t) = θ5(Tt), where T is the gait period. Therefore, gait dynamics can be represented by the changes in the limb angles and the relative distance between the knee joints on either side, that is, gait dynamics can be expressed as F(φ) = F(θ1, θ2, l) or F(φ) = F(θ4, θ5, l).

[0073] The present invention selects the angle between the left thigh and the vertical direction, the angle between the left calf and the vertical direction, and the relative distance between the knee joints as the time-varying features to characterize the gait. Therefore, the gait process can be expressed as where F is the dynamics as the state of the system.

[0074] In the specific implementation process, Kinect provided by Microsoft is used to extract the 3D coordinates of each joint of the human body in the camera coordinate system, and the relative distance between the knee joints is extracted according to Formula 4. The periodic change curve is as follows Figure 3 -a shown;

[0075]

[0076] Among them, (x lknee ,y lknee ,z lknee )、(x rknee ,y rknee ,z rknee ) are the 3D coordinates of the left and right knee joints, respectively. Similarly, the angles of the thigh and calf relative to the vertical direction are calculated according to Formula 5. The change curves are as follows: Figure 3 -b, 3-c.

[0077]

[0078] in, Represent the direction vectors of the thigh and calf respectively, Indicates the vertical direction of a person walking.

[0079] According to the above three motion characteristics, the three-dimensional gait phase space trajectory is drawn to form a visual periodic or quasi-periodic parameter trajectory that intuitively describes the gait motion, such as Figure 4 As shown, this is conducive to intuitive and effective gait analysis.

[0080] S2: Input the three-dimensional gait variable sequence into the trained RBF neural network to obtain the gait dynamic feature sequence and time-invariant knowledge. The time-invariant knowledge is used to construct a gait pattern library and draw the dynamic trajectory.

[0081] By dynamically modeling the three-dimensional gait variables, dynamic information reflecting the gait dynamics is obtained. The three-dimensional gait variables reflect the gait changes between frames in the gait sequence. According to the deterministic learning theory, the RBF neural network is used to dynamically model the training gait pattern along the selected three-dimensional gait phase space trajectory, and the dynamic information reflecting the gait changes is extracted as the visual gait dynamic features of the training gait pattern.

[0082] The training gait pattern can be expressed as follows:

[0083]

[0084] where x = [x1, x2, x3] T ∈R 3 is the kinematic gait variable representing the gait of the system state; p is the system constant parameter that produces different gait patterns; F(x; p) represents the gait dynamics, and v(x; p) represents the modeling uncertainty.

[0085] Based on the deterministic learning algorithm, the RBF neural network is used to accurately model the training gait along the three-dimensional gait phase space trajectory, and the gait dynamics hidden behind the time-varying signal are obtained as gait dynamic features. Thanks to the natural visualization properties of three-dimensional variables, the obtained gait dynamics information can be intuitively displayed in the visualization space. The main steps of the gait dynamics feature extraction process are as follows:

[0086] 1) Construct the RBF neural network as shown below:

[0087]

[0088] Among them, X represents the input time-varying features, W represents the neural network weight, s i Represents a type of radial basis function, choose Gaussian function μ i Represents points regularly distributed in the state space, i = 1,…,N, the network has a total of 1728 neurons, and the centers of each neuron are evenly distributed in the area [1.0, -0.1] × [1.0, -0.1] × [1.0, -0.1], with an interval of η = 0.1.

[0089] 2) Use RBF neural network to perform dynamic modeling on the training gait pattern and obtain gait representation:

[0090]

[0091] in, represents the state estimation of the neural network model, x i is the input of the RBF neural network, representing the motion parameters of the gait, Represents the RBF neural network used to approximate unknown dynamics, and the design parameter a i =0.05.

[0092] 3) Initialize all neural network weights to 0 and update the weights according to the following rules:

[0093]

[0094] in, are the state estimation error, weight error and ideal weight vector respectively; in this embodiment, the parameters are set as follows: Γ i =20(i=1,2,3),σ i =0.155(i=1,2,3).

[0095] State estimation error The derivative of satisfies According to the deterministic learning theory, the weights of the RBF neural network will quickly converge to their optimal values, and the average value over a period of time after the weights converge is used. t b >t a >0 indicates the final convergence result of the weight. So far, the RBF neural network has been used to obtain the gait dynamics φ i Locally accurate modeling of (x;p) As a gait representation:

[0096]

[0097] Among them, ε i represents the approximation error, and the learned time-invariant knowledge constitutes a gait pattern library representing the training gait patterns, which is used for subsequent fast gait recognition.

[0098] As a three-dimensional visualization of gait dynamics features, the dynamic trajectory obtained by RBF neural network is as follows: Figure 5 As shown, the dynamic information of gait is described intuitively and vividly.

[0099] The three-dimensional gait kinematic variables and the gait dynamics information extracted from them can intuitively describe the gait motion and the implied gait change information in the three-dimensional visualization space; the space that people can intuitively see and understand is at most three-dimensional, and the three-dimensional motion gait variables and gait dynamics information extracted from the five-link model meet this requirement. The three-dimensional motion features constitute the spatial state trajectory describing the gait motion. The gait dynamics features extracted from the gait motion using the deterministic learning method constitute the dynamic trajectory describing the gait dynamics. Different individuals have different gait patterns, such as Figure 6 By comparing the three-dimensional gait variable sequences of different individuals (such as Figure 6 -a), three-dimensional gait phase space trajectory (such as Figure 6 -b) and dynamic trajectories (e.g. Figure 6 -c), it can be seen that the gait trajectories of different individuals have significantly different topological structures, that is, different individuals have different gait dynamics characteristics.

[0100] S3: Extract the three-dimensional gait variable sequence of the gait sequence to be identified, use the time-invariant knowledge in the gait pattern library to build a state estimator, and quickly compare the dynamics of the gait sequence to be identified with the dynamics of each gait in the gait pattern library to obtain the final gait recognition result.

[0101] This embodiment adopts a dynamic pattern rapid recognition method with a deterministic learning mechanism to achieve rapid gait recognition. The specific recognition process is as follows:

[0102] 1) Utilize the time-invariant knowledge acquired during gait training Construct a series of dynamic estimators for training gait sequences:

[0103]

[0104] Where k=1,…,M represents the kth dynamic estimator, represents the state of the estimator, and B = diag{0.05,0.05,0.05} is the designed diagonal matrix, which is the same for all estimators and is consistent with the design parameters a during the training phase. i Stay consistent.

[0105] 2) extracting a three-dimensional gait variable sequence from the gait sequence;

[0106] 3) Based on the three-dimensional gait variable sequence and the dynamic estimator, a residual system is obtained that represents the difference between the test gait and the training gait:

[0107]

[0108] in, represents the synchronization or estimation error, where b i =0.05.

[0109] 4) Calculate the estimation error proportional to the difference between gait patterns The average L1 norm of After a finite time t f After that, if s≠k, for all t>t f If established, The corresponding object is the recognition result. For the recognition result, the phase space trajectory and dynamic trajectory of the gait pattern can be intuitively compared in the visualization space to analyze the differences between different gait patterns.

[0110] Experimental Part 1: Various types of experiments were conducted on the self-built 3D skeleton gait database

[0111] Using Microsoft's second-generation depth camera KinectV2, a gait database with five perspectives was established. The database contains 47 people, including 17 women and 30 men. The data in the database were collected in an indoor treadmill environment. During the collection process, speed and perspective were strictly controlled, but there were no strict restrictions on shoes and clothing factors. The Kinect was approximately 90 cm from the ground; on a semicircle with a radius of 240 cm and the geometric center of the treadmill as the center, a Kinect was placed at 45° intervals starting from the left, for a total of five Kinects; each subject walked on the treadmill at a speed of 3 km / h at two time periods of the day, and a total of five gait sequences were collected with a sampling rate of approximately 30 fps; each subject had 25 gait sequences from five perspectives, each containing approximately 300 frames. The following two types of experiments were conducted on this self-built database.

[0112] Gait recognition performance on self-built database

[0113] The first type of experiment verified the effectiveness of the selected gait dynamics features from the left side of the eye. Only gait data from the left side of the eye were used for the training and test sets. Three sets of gait sequences from each subject's left side of the eye were randomly selected as the training set, and the remaining two sets of sequences were used as the test set. Thus, the training set included 47*3=141 sets of gait sequences. Three-dimensional gait variables, including the angle of the left thigh relative to the vertical, the angle of the left calf relative to the vertical, and the relative distance between the knee joints, were extracted. Gait dynamics features were extracted using an RBF neural network. A fast gait recognition method, namely a fast gait recognition mechanism, was used to achieve rapid and reliable gait recognition. To obtain convincing results, the above experiment was randomly repeated 20 times. The average of the CCR (Constraint Characterization Rate) across all experiments was used as the final recognition result. Comparison results with existing methods are shown in Table 1. The experimental results show that, compared with existing methods, the method described in this paper can achieve sufficiently good recognition results using fewer features.

[0114] Table 1 Recognition results under normal conditions of the self-built database

[0115]

[0116] Gait recognition performance across perspectives in a self-built database

[0117] To evaluate the performance of the proposed algorithm across viewpoints, experiments were conducted on a self-built database, as shown in Table 2. Three gait sequences from each viewpoint were randomly selected and added to the training set. A deterministic learning method was used to establish a gait dynamics model and a gait pattern library. The remaining gait sequences were used as the test set. For each test gait sequence from each viewpoint, a dynamic pattern fast recognition mechanism was used to achieve fast gait recognition. The average CCR of 20 random experiments was used as the experimental result, as shown in Table 2. The experimental results show that the method of this embodiment has strong robustness across viewpoints.

[0118] Table 2 Recognition results from multiple perspectives of the self-built database

[0119]

[0120] Experimental Part 2: Experiments on the UPCV Database

[0121] The method of this embodiment was verified by experimental results on the public gait database UPCV. UPCV includes 150 gait sequences from 30 subjects. The gait data was acquired using Microsoft's first-generation depth camera Kinect, including the coordinates of 21 joint points. The Kinect sensor was approximately 2 meters above the ground and had an inclination of approximately 30-40 degrees with the walking direction. Each subject walked in a straight line toward the Kinect at a comfortable speed at three time periods of the day. Five sets of gait sequences were collected at a sampling rate of 30fps, with each gait sequence containing approximately 55 to 120 frames.

[0122] In this experiment, three-dimensional gait variables were first obtained from a five-link bipedal model. Three sets of three-dimensional gait variable sequences for each subject were randomly selected as the training set, and the remaining two sets were used as the test set. That is, the training set contained 90 gait sequences and the test set contained 60 gait sequences. Then, the dynamic features of each gait were extracted based on the dynamic modeling method, and gait recognition was performed using a fast recognition mechanism. The above experiment was randomly repeated 20 times, and the average recognition rate of all experiments was used as a convincing experimental result. It was compared with other algorithms on this database. The results are shown in Table 3.

[0123] Table 3 Comparison of CCR (%) between different methods and the method proposed in this invention

[0124]

[0125] The visualization analysis method based on deterministic learning proposed in this invention can also be used to perform visualization analysis on the gait recognition results of the public database. A group of test gaits with recognition errors is randomly selected, such as the gait e-1 of individual e. The residual system generates a residual curve of the test gait e-1 and the training gait over time, such as Figure 7According to the minimum residual principle, the residual between the test gait e-1 and the training gait f-2 is the smallest, as shown in Figure 7 The “-” curve in , so individual e is misidentified as individual f. Figure 8 In the gait dynamic characteristic state curve shown, it can be observed that the thigh and calf angle characteristic curves have strong similarities, and the distance characteristics between the knee joints are quite different; however, this makes it difficult to fully compare the similarities between different gaits. The gait phase space trajectory and dynamic trajectory in three-dimensional space can intuitively and comprehensively reflect the differences between gaits, such as Figure 9 As shown in the figure, “-” belongs to the training gait and “---” belongs to the test gait. From the figure, we can see that there are obvious differences in the phase space trajectory and dynamic trajectory between the test gait e-1 and the training gait sequence f-2.

[0126] At the same time, the phase space trajectory of all gaits of individual e is drawn (such as Figure 10 -a) and dynamic trajectories (e.g. Figure 10 -b). From Figure 10 It can be seen that the gait sequence e-1 of individual e is significantly different from other gaits in both phase space trajectory and dynamic trajectory, which indicates that the existing gait data cannot fully represent the object e. Therefore, in order to improve the recognition performance, more gaits under this condition need to be collected and added to the training set.

[0127] Experimental results show that the method of this embodiment performs better than existing methods on the UPCV database. Compared with our original method, although the feature dimension is reduced, it does not cause a decrease in recognition performance. This proves that the three-dimensional spatiotemporal features selected in this embodiment have extremely strong resolution capabilities. At the same time, the visual analysis process once again proves that the method of this embodiment provides a powerful tool for gait analysis.

[0128] Example 2

[0129] This embodiment discloses a visual gait recognition system based on deterministic learning;

[0130] like Figure 2 As shown, a visual gait recognition system based on deterministic learning includes a three-dimensional gait variable extraction module, a gait dynamic feature extraction module and a fast gait recognition module;

[0131] The three-dimensional gait variable extraction module is configured to: collect a gait sequence consisting of multiple frames, extract three-dimensional gait variables from each frame of the gait sequence, form a three-dimensional gait variable sequence, and draw a three-dimensional gait trajectory;

[0132] The gait dynamics feature extraction module is configured to: input the three-dimensional gait sequence into the trained RBF neural network to obtain the gait dynamics feature sequence and time-invariant knowledge, construct a gait pattern library based on the time-invariant knowledge, and draw the dynamic trajectory;

[0133] The fast gait recognition module is configured to extract the three-dimensional gait variable sequence of the gait sequence to be recognized, and quickly compare it with the time-invariant knowledge in the gait pattern library to obtain the final gait recognition result.

[0134] Example 3

[0135] The purpose of this embodiment is to provide a computer-readable storage medium.

[0136] A computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of a visual gait recognition method based on deterministic learning as described in Example 1 of the present disclosure.

[0137] Example 4

[0138] The purpose of this embodiment is to provide an electronic device.

[0139] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps of a visual gait recognition method based on deterministic learning as described in Example 1 of the present disclosure are implemented.

[0140] The steps involved in the apparatuses of Examples 2, 3, and 4 above correspond to those of Method Example 1. For detailed implementations, please refer to the relevant description of Example 1. The term "computer-readable storage medium" should be understood to mean a single medium or multiple media containing one or more instruction sets; it should also be understood to include any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and causing the processor to perform any method of the present invention.

[0141] Those skilled in the art will appreciate that the modules or steps of the present invention described above can be implemented using a general-purpose computer device. Alternatively, they can be implemented using program code executable by a computing device, which can then be stored in a storage device and executed by the computing device. Alternatively, they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0142] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without any creative work are still within the scope of protection of the present invention.

Claims

1. A visual gait recognition method based on deterministic learning, characterized in that: include: Collect a gait sequence consisting of multiple frames, extract three-dimensional gait variables from each frame of the gait sequence, form a three-dimensional gait variable sequence, and draw a three-dimensional gait phase space trajectory; The three-dimensional gait variable sequence is input into the trained RBF neural network to obtain the gait dynamic feature sequence and time-invariant knowledge. The time-invariant knowledge is used to construct a gait pattern library and draw the dynamic trajectory. Extract the three-dimensional gait variable sequence of the gait sequence to be identified, build a state estimator using the time-invariant knowledge in the gait pattern library, and quickly compare the dynamics of the gait sequence to be identified with the gait patterns in the gait pattern library to obtain the final gait recognition result; The three-dimensional gait variables are: the angle between the left thigh and the vertical direction, the angle between the left calf and the vertical direction, and the relative distance between the knee joints.

2. The visual gait recognition method based on deterministic learning according to claim 1, characterized in that: In each frame of the gait sequence, a five-link bipedal model is used to characterize human motion.

3. The visual gait recognition method based on deterministic learning according to claim 2, characterized in that: The five-link biped model is represented by the Lagrange equation, and the limb angle vector is ; in, They are the angles between the left calf, left thigh, human trunk, right thigh, right calf and the vertical direction; Extracted from the five-link biped model As three-dimensional gait variables, the angle between the left calf and the vertical direction and the angle between the left thigh and the vertical direction.

4. The visual gait recognition method based on deterministic learning according to claim 2, characterized in that: The distance between knee joints in the five-link biped model The length of the left and right thighs 、 The triangular constraints between them are calculated according to the cosine theorem to obtain the distance between the knee joints. .

5. The visual gait recognition method based on deterministic learning according to claim 1, characterized in that: The main steps of the gait dynamics feature sequence extraction process are as follows: Construct RBF neural network; The RBF neural network is used to perform dynamic modeling on the training gait pattern and obtain the gait dynamic feature sequence; Initialize all neural network weights to 0 and update them according to the deterministic learning theory until they converge to the optimal value, obtaining the timely invariant knowledge of the gait dynamics feature sequence; The learned time-invariant knowledge is used to construct a gait pattern library.

6. The visual gait recognition method based on deterministic learning according to claim 1, characterized in that: To obtain the gait recognition results, the specific steps are: A series of dynamic estimators are constructed using the time-invariant knowledge in the gait pattern library; extracting a three-dimensional gait sequence of a gait sequence to be identified; According to the three-dimensional gait sequence and the dynamic estimator, a residual system representing the difference between the gait to be identified and the gait pattern is obtained; The average L1 norm of the estimation error, which is proportional to the difference between gait patterns, is calculated, and the gait pattern with the smallest error is the recognition result.

7. A visual gait recognition system based on deterministic learning, characterized by: It includes three-dimensional gait variable extraction module, gait dynamic feature extraction module and fast gait recognition module; The three-dimensional gait variable extraction module is configured to: collect a gait sequence consisting of multiple frames, extract three-dimensional gait variables from each frame of the gait sequence, form a three-dimensional gait variable sequence, and draw a three-dimensional gait phase space trajectory; The gait dynamics feature extraction module is configured to: input the three-dimensional gait sequence into the trained RBF neural network to obtain the gait dynamics feature sequence and time-invariant knowledge, construct a gait pattern library based on the time-invariant knowledge, and draw the dynamic trajectory; The fast gait recognition module is configured to: extract a three-dimensional gait variable sequence of the gait sequence to be recognized, construct a state estimator using the time-invariant knowledge in the gait pattern library, and quickly compare the dynamics of the gait sequence to be recognized with the gait patterns in the gait pattern library to obtain the final gait recognition result; The three-dimensional gait variables are: the angle between the left thigh and the vertical direction, the angle between the left calf and the vertical direction, and the relative distance between the knee joints.

8. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps of the visual gait recognition method based on deterministic learning as described in any one of claims 1 to 6 are implemented.

9. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the visual gait recognition method based on deterministic learning as described in any one of claims 1 to 6 are implemented.

Images