A multi-person gait recognition model

By introducing a two-way long and short-term memory network and attention mechanism into the multi-person gait recognition model, combined with multiple data sources, the data quality and accuracy problems of multi-person gait recognition in urban environments are solved, and the adaptability and accuracy.

CN118468084BActive Publication Date: 2024-11-19BEIJING INFORMATION SCI & TECH UNIV
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202410643842.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-23
Publication Date
2024-11-19
Estimated Expiration
2044-05-23

AI Technical Summary

Technical Problem

The existing multi-person gait recognition model faces the problems of poor quality of identification data, low accuracy of gait recognition results, and difficulty in adapting to complex and changing environments in urban environments.

Method used

Using a bidirectional long and short-term memory network, innovative improvements to the attention mechanism are combined with sole pressure sensor data and human body acceleration data, and multi-dimensional feature extraction and fusion are performed through the full connection layer, BiLSTM layer and TimeDistributed attention layer to improve The accuracy and robustness of the model.

Benefits of technology

It significantly improves the accuracy of gait recognition and the adaptability of the model, enhances personalized recognition ability and robustness, improves data quality and computing efficiency, and enhances the generalization ability of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118468084B_ABST
    Figure CN118468084B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of biometrics, specifically to a multi-person gait recognition model, which includes the following steps: collecting plantar pressure sensor data and human body acceleration data, obtaining gait recognition initial data based on the plantar pressure sensor data and the human body acceleration data, processing the gait recognition initial data, and obtaining gait recognition target data; obtaining the output result of the full connection layer, the output result of the first BiLSTM layer, and the output result of the second BiLSTM layer based on the gait recognition target data; performing residual addition on the output result of the full connection layer and the output result of the first BiLSTM layer to obtain a first residual output result; obtaining a second residual output result based on the first residual output result and the output result of the second BiLSTM layer; performing attention analysis on the second residual output result based on the TimeDistributed attention layer to obtain a final attention output result, and recognizing the gait of multiple people according to the final attention output result. The model of the present invention provides a solution for action recognition.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] The present invention relates to the field of biometric identification technology, in particular to a multi-person gait recognition model. Background Art

[0002] With the development of artificial intelligence, multi-person gait recognition has always been a hot research topic. Although the existing video image data provides rich data information for multi-person gait recognition research, the unstable lighting in natural environment conditions and the need for privacy protection still pose a considerable challenge to multi-person gait recognition technology. The current gait recognition method based on wearable sensor data provides a new direction for thinking, but the lack of sensor accuracy and the complexity of the model dimension are still difficult problems that the existing multi-person gait recognition model needs to solve.

[0003] In order to deeply explore the problem of multi-person gait recognition in urban environments, the existing multi-person gait recognition model in urban environments is optimized. Through data optimization, analysis and mining, the accuracy and generalization ability of multi-person gait recognition technology in practical applications are improved, providing more solid support for the research and practice in the field of biometric technology, and contributing more to the development of urban security, intelligent monitoring and other fields. Summary of the invention

[0004] In view of the defects of existing methods and the shortcomings in practical applications, in order to solve the problems faced by multi-person gait recognition models in current urban environments, such as poor recognition data quality, low accuracy of gait recognition results, and difficulty in adapting to complex and changing environments, the present invention introduces a bidirectional long short-term memory network and makes innovative improvements to the attention mechanism. The present invention has achieved remarkable results in improving the accuracy and robustness of the model. In the first aspect, the present invention provides a multi-person gait recognition model, which includes the following steps: collecting plantar pressure sensor data and human body acceleration data, obtaining gait recognition initial data based on the plantar pressure sensor data and the human body acceleration data, processing the gait recognition initial data, and obtaining gait recognition target data; obtaining the full connection layer output result, the first BiLSTM layer output result and the second BiLSTM layer output result based on the gait recognition target data; adding the residuals of the full connection layer output result and the first BiLSTM layer output result to obtain the first residual output result; obtaining the second residual output result based on the first residual output result and the second BiLSTM layer output result; performing attention analysis on the second residual output result based on the TimeDistributed attention layer to obtain the final attention output result, and identifying the gait of multiple people according to the final attention output result. The present invention combines plantar pressure sensor data and human body acceleration data to capture gait features from multiple dimensions, making the gait recognition initial data more comprehensive and rich, and multi-dimensional data fusion not only improves the accuracy of gait recognition, but also enhances the model's adaptability to complex gait changes.

[0005] Optionally, the collecting of plantar pressure sensor data and human body part acceleration data includes: setting 16 plantar sensors in the multi-person gait recognition model, and collecting plantar pressure sensor data based on the 16 plantar sensors; presetting human body three-axis acceleration parts in the multi-person gait recognition model; and collecting human body part acceleration data based on the human body three-axis acceleration parts. The present invention sets 16 plantar sensors and presetting human body three-axis acceleration parts to collect plantar pressure sensor data and human body part acceleration data, which not only improves the comprehensiveness and accuracy of the data, but also enhances the personalized recognition ability, robustness and adaptability of the model, and provides more reliable and effective data support for the multi-person gait recognition model.

[0006] Optionally, the processing of the initial gait recognition data and obtaining the gait recognition target data includes: the multi-person gait recognition model uses a filter to perform data smoothing on the initial gait recognition data to obtain the first gait recognition data; the multi-person gait recognition model uses a sliding time window method to divide the first gait recognition data to obtain the second gait recognition data; the multi-person gait recognition model uses PCA technology to optimize the second gait recognition data to obtain gait recognition target data. The present invention performs smoothing, time window division and PCA technology optimization on the initial gait recognition data, and based on this, the multi-person gait recognition model can significantly improve data quality, feature extraction effectiveness, computational efficiency and model generalization ability.

[0007] Optionally, the fully connected layer output result, the first BiLSTM layer output result and the second BiLSTM layer output result obtained based on the gait recognition target data include: the multi-person gait recognition model obtains the fully connected layer output result based on the gait recognition target data and the fully connected layer; the multi-person gait recognition model obtains the first BiLSTM layer output result based on the gait recognition target data and the first BiLSTM layer; the multi-person gait recognition model obtains the second BiLSTM layer output result based on the gait recognition target data and the second BiLSTM layer. The gait recognition target data of the present invention obtains output results through the fully connected layer, the first BiLSTM layer and the second BiLSTM layer respectively, which helps to extract and fuse various types of features and improve the robustness, adaptability and generalization ability of the model.

[0008] Optionally, the performing of attention analysis on the second residual output result based on the TimeDistributed attention layer to obtain the final attention output result includes: obtaining a forward hidden state result of the second residual output result based on the TimeDistributed attention layer; and obtaining a backward hidden state result of the second residual output result based on the TimeDistributed attention layer. The present invention combines the forward hidden state result with the backward hidden state result, thereby being able to more comprehensively capture the temporal dependency in the gait data, which helps to improve the actual performance of the model in the multi-person gait recognition model.

[0009] Optionally, the forward hidden state result satisfies the following relationship:

[0010] H f =F(w1x t +w2H f-1 -1)

[0011] Among them, H f represents the forward hidden state at time f, F represents the activation function, w1 represents the first weight, x trepresents the input at time t, w2 represents the second weight, H f-1 The present invention combines the input at the current moment and the forward hidden state at the previous moment to capture and transmit the temporal information in the gait data, introduces relevant weights so that the model adjusts the weights according to the characteristics of the data, thereby better fitting the distribution and changes of the gait data.

[0012] Optionally, the backward hidden state result satisfies the following relationship:

[0013] H b =F(w3x t +w5H b+1 -1)

[0014] Among them, H b represents the backward hidden state at time b, F represents the activation function, w3 represents the third weight, x t represents the input at time t, w5 represents the fifth weight, H b+1 The backward hidden state of the present invention can capture and utilize information from the future moment to the current moment, fully utilize the bidirectional time sequence information of the sequence, and thus more accurately understand the dynamic changes of gait data and improve the accuracy of recognition.

[0015] Optionally, performing attention analysis on the second residual output result based on the TimeDistributed attention layer to obtain a final attention output result, and recognizing the gait of multiple people based on the final attention output result includes: combining the forward hidden state result and the backward hidden state result of the second residual output result to obtain the final attention output result of the second residual output result; and recognizing the gait of multiple people based on the final attention output result. The present invention performs attention analysis on the second residual output result based on the TimeDistributed attention layer to improve recognition accuracy, enhance model robustness, and optimize computing resource allocation.

[0016] Optionally, the final attention output result satisfies the following relationship:

[0017] Y t =g(w4H f +w6H b )

[0018] Among them, Y t represents the output of the multi-person gait recognition model at time t, g represents the activation function, w4 represents the fourth weight, H f represents the forward hidden state at time f, w6 represents the sixth weight, H bRepresents the backward hidden state at time b. The present invention provides strong support for multi-person gait recognition models by fusing bidirectional information, providing flexibility in weight learning, extracting nonlinear features, and optimizing model performance, which helps the model to more accurately identify the gait characteristics of different individuals.

[0019] In a second aspect, the present invention further provides a multi-person gait recognition system, which can efficiently execute a multi-person gait recognition model provided by the present invention, the system includes an input device, a processor, an output device and a memory, wherein the input device, the processor, the output device and the memory are interconnected, the memory includes a computer-readable storage medium as described in the first aspect of the present invention, the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions. The system provided by the present invention has a compact structure, strong applicability, and greatly improves the operating efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flow chart of the multi-person gait recognition model of the present invention;

[0021] Figure 2 It is a schematic diagram of comparison between initial data and first step gait recognition data in the multi-person gait recognition model of the present invention;

[0022] Figure 3 It is a schematic diagram of the second gait recognition data in the multi-person gait recognition model of the present invention;

[0023] Figure 4 This is a schematic diagram of the LSTM network model of the multi-person gait recognition model of the present invention;

[0024] Figure 5 This is a schematic diagram of the BiLSTM structure of the multi-person gait recognition model of the present invention;

[0025] Figure 6 This is a schematic diagram showing the comparison of the accuracy of the training data set and the verification data set of the multi-person gait recognition model of the present invention;

[0026] Figure 7 A schematic diagram showing the comparison of the loss rates of the training data set and the verification data set of the multi-person gait recognition model of the present invention;

[0027] Figure 8 A schematic diagram of a confusion matrix of a multi-person gait recognition model of the present invention;

[0028] Fig. 9 A schematic structural diagram of a multi-person gait recognition system of the present invention. DETAILED DESCRIPTION

[0029] The specific embodiments of the present invention will be described in detail below. It should be noted that the embodiments described herein are only for illustration and are not intended to limit the present invention. In the following description, a large number of specific details are set forth in order to provide a thorough understanding of the present invention. However, it is obvious to those of ordinary skill in the art that these specific details do not need to be adopted to implement the present invention. In other examples, in order to avoid confusing the present invention, known circuits, software or methods are not specifically described.

[0030] Throughout the specification, references to "one embodiment," "an embodiment," "an example," or "an example" mean that a particular feature, structure, or characteristic described in conjunction with the embodiment or example is included in at least one embodiment of the present invention. Therefore, the phrases "in one embodiment," "in an embodiment," "an example," or "an example" appearing in various places throughout the specification do not necessarily all refer to the same embodiment or example. In addition, particular features, structures, or characteristics may be combined in one or more embodiments or examples in any suitable combination and / or subcombination. In addition, it should be understood by those of ordinary skill in the art that the figures provided herein are for illustrative purposes and that the figures are not necessarily drawn to scale.

[0031] See also Figure 1 In order to solve the problems of poor data quality, insufficient recognition accuracy and poor environmental adaptability in multi-person gait recognition models in urban environments, a bidirectional long short-term memory network is introduced to optimize the attention mechanism, avoiding problems such as network degradation caused by an increase in the number of network layers. The present invention provides a multi-person gait recognition model, which includes the following steps:

[0032] S1. Collecting plantar pressure sensor data and human body acceleration data, obtaining initial gait recognition data based on the plantar pressure sensor data and human body acceleration data, further processing the initial gait recognition data, and obtaining gait recognition target data. The specific implementation steps and related contents are as follows:

[0033] In this embodiment, 16 plantar sensors are set in the multi-person gait recognition model, and plantar pressure sensor data is collected based on the above 16 plantar sensors. At the same time, human body three-axis acceleration parts are preset in the multi-person gait recognition model, and human body part acceleration data is collected based on the human body three-axis acceleration parts. The specific implementation content is as follows:

[0034] In this embodiment, based on the urban environment multimodal gait database built by Honda European Research, gait recognition technology is deeply experimentally studied. The above database mainly includes 20 participants in good health, including 5 women and 15 men, with an age range of 18 to 69 years old. The average height of the participants is 178.5 cm, with a standard deviation of 7.64 cm, and the average weight is 72.9 kg, corresponding to a standard deviation of 8.7 kg. In this embodiment, the participants all wore full-body suits equipped with 17 IMU sensors, insoles with 8 pressure sensing units on each foot, and carried a mobile eye tracker, and each participant completed three different walking tasks.

[0035] Furthermore, in this embodiment, the data with the most uniform sample distribution is selected as the initial data for gait recognition, and then the plantar pressure data of each participant, as well as the acceleration data of the foot, calf, thigh, waist and hands are analyzed in detail to ensure the consistency of the initial data for gait recognition. The data of all sensors have been downsampled to 60Hz, containing a total of 846715 data points.

[0036] In order to ensure the practicality and feasibility of the initial data of gait recognition, a label mapping operation is performed on the initial data of gait recognition, and the original action type label is converted into a digital label that can be recognized by the multi-person gait recognition model. In an optional embodiment, the walking action type is mapped to the number 0, the stair climbing action is mapped to the number 1, the stair descending action is mapped to the number 2, the uphill action is mapped to the number 3, and the downhill action is mapped to the number 4, which provides a clear and easy-to-process information basis for the subsequent gait recognition model.

[0037] Furthermore, in the present embodiment, the method for acquiring the initial data for gait recognition is merely an optional condition of the present invention. In one or some other embodiments, the method for acquiring the initial data can be flexibly adjusted according to the data collection requirements and the circumstances of the participants, so as to ensure that the initial data can accurately reflect the gait characteristics of the participants and improve the accuracy and reliability of the initial data.

[0038] In the embodiment, the initial gait recognition data is processed and the gait recognition target data is obtained, and the specific implementation content is as follows:

[0039] The multi-person gait recognition model uses a filter to smooth the initial gait recognition data to obtain the first step gait recognition data.

[0040] Since the initial data collected by the acceleration sensor is often mixed with various irrelevant signals, including but not limited to factors such as magnetic field disturbance and instability or jitter of the device itself, and the collection of plantar pressure data is affected by factors such as sole material, cushioning performance and friction, etc. In order to minimize the adverse effects of external factors on the initial data of gait recognition, the initial data of gait recognition is optimized in the embodiment, wherein the purpose of the filter is to remove noise.

[0041] The above-mentioned smoothing filtering methods include but are not limited to Gaussian filtering, Savitzky-Golay filtering and mean filtering. Among them, the Savitzky-Golay filter, as a low-pass filter with excellent performance, has a significant effect in smoothing noise data, based on which the first step state recognition data is obtained.

[0042] In order to more intuitively demonstrate the data smoothing effect of the filter, a curve comparison diagram of the initial data before and after filtering and the first step recognition data is drawn in the embodiment. For details, please refer to Figure 2 ,Comparing the data curves before and after filtering helps to deeply understand the ,performance characteristics of the filter and provide strong data support for the subsequent ,gait recognition work.

[0043] Based on the plantar pressure and waist acceleration data curves before and after filtering, it can be clearly seen that the Savitzky-Golay filter can not only effectively smooth the noise in the original signal, but also better preserve the edge features of the data, showing a strong edge retention ability.

[0044] In the embodiment, filtering preprocessing is performed on the initial gait recognition data, which can effectively eliminate the interference of noise, making the gait recognition data purer and more accurate, thereby improving the accuracy and stability of the gait recognition results, and laying a solid data foundation for subsequent gait analysis and recognition.

[0045] The multi-person gait recognition model uses a sliding time window method to divide the first gait recognition data and obtain the second gait recognition data.

[0046] In this embodiment, the data collected by the plantar pressure sensor and IMU are data streams that are continuously distributed with time as the axis. It is arduous and difficult to directly extract features from the entire sequence, so it is necessary to segment the first step recognition data to efficiently perform subsequent data analysis and processing. Before building the neural network model, a sliding window technique is used to extract a subsequence of fixed length from the time series, and the above subsequence will be used as the input for model training.

[0047] The sliding time window method refers to the window slicing process of the first step recognition data, setting a window of time length L in advance, and moving the window backward on the time axis according to a fixed step length d. In order to allow a certain overlap between adjacent windows, the step length d is often set to half of the window size L. In the existing recurrent neural network model, the label corresponding to the data point with the largest proportion in the window is often used as the label of the entire window, but it is easy to cause the model to learn wrong training data, thereby affecting the accuracy of the model.

[0048] In order to overcome the above problems, an improved sliding time window method is proposed in this embodiment to obtain the second gait recognition data. When dividing the window, first check whether the data point in the window corresponds to only one label. If the above conditions are met, the window and its corresponding label are retained; otherwise, the above window is discarded. Based on the characteristics of the sliding window, it can be seen that even if some windows are discarded, as long as the step size is set small enough, the discarded data can be included in the previous window or in the window of the next action, so there is no need to worry about the loss or missing of the first gait recognition data.

[0049] In this embodiment, the improved method of the sliding time window method can not only prevent the model from learning wrong training data, but also improve the accuracy of the model. In practical applications, the time window size can be set to 1s and the step size can be set to 0.33s to better balance the data coverage and model performance. For details, please refer to Figure 3 , which respectively plots the x-axis acceleration of the left thigh, the x-axis acceleration of the left calf, the x-axis acceleration of the left hand, the x-axis acceleration of the right thigh, the x-axis acceleration of the right calf, and the x-axis acceleration of the right hand, which can more intuitively display the segmentation content of the first step recognition data sliding window.

[0050] The multi-person gait recognition model uses PCA technology to optimize the second gait recognition data to obtain gait recognition target data.

[0051] On the other hand, in gait analysis scenarios that require rapid response, including but not limited to sports training and health monitoring, data processing efficiency and model training speed are crucial. The embodiment uses PCA dimensionality reduction technology to convert the original high-dimensional second gait recognition data into a low-dimensional representation, which not only greatly reduces the computational burden and number of parameters of the model, but also improves the training efficiency of the model. In the above process, PCA dimensionality reduction technology ensures that key variance information in the data is retained. In an optional embodiment, the original 37-dimensional second gait recognition data is reduced to 30 feature dimensions through PCA technology, which not only meets the timeliness requirements, but also ensures the effectiveness of the second gait recognition data, thereby improving the application effect in fields such as training and monitoring. The second gait recognition data is optimized using PCA technology to obtain gait recognition target data, which not only optimizes the model performance, but also provides strong technical support for real-time gait analysis.

[0052] Furthermore, in this embodiment, the method for processing the initial data of gait recognition is only an optional condition of the present invention. In one or some other embodiments, the data processing method can be changed according to actual application requirements and data requirements, which can better meet data analysis requirements, adapt to different data situations, and improve data processing efficiency and performance.

[0053] In addition, in this embodiment, a model training environment is built based on the Python 3.10 platform and the Tensor Flow framework. In order to prevent the model from overfitting, an early stopping mechanism is used in the embodiment. The above mechanism monitors the loss value of the validation set. If the loss value does not continue to decrease in 10 consecutive training cycles, the training is stopped immediately. During the entire implementation process, ensure that the input of all data remains consistent, and use a ratio of 8:1:1 to construct the training set, validation set, and test set respectively when dividing the data set. Based on this, not only can the training efficiency and accuracy of the model be improved, but also the stability and reliability of the model in practical applications can be enhanced. Through the early stopping mechanism and scientific data set division, an efficient and robust gait recognition model is constructed, which provides strong technical support for application scenarios with high real-time requirements such as training and monitoring.

[0054] S2. Based on the above gait recognition target data, the output results of the fully connected layer, the output results of the first BiLSTM layer, and the output results of the second BiLSTM layer are obtained. The specific implementation steps and related contents are as follows:

[0055] The multi-person gait recognition model obtains the fully connected layer output result based on the gait recognition target data and the fully connected layer. The specific implementation content is as follows:

[0056] The gait recognition target data is fed into the fully connected layer of the model. The fully connected layer is a layer in the neural network, in which each node is connected to all nodes in the previous layer. In the multi-person gait recognition model, the fully connected layer performs weighted and biased operations on the input features and performs nonlinear transformations through activation functions to extract higher-level feature representations.

[0057] During the processing of the fully connected layer, the model will learn a set of weights and bias parameters, which are continuously optimized through training. In this embodiment, the training process includes but is not limited to the steps of forward propagation, loss calculation, back propagation and parameter update, so that the model can more accurately identify the gait of different individuals.

[0058] Finally, the fully connected layer will output a result, which can be a vector or probability distribution, indicating the possibility that the input data belongs to different individuals. In the scenario of multi-person gait recognition, the model needs to distinguish and recognize the gaits of multiple different individuals, so the output result of the fully connected layer has sufficient discrimination.

[0059] In this embodiment, the multi-person gait recognition model based on the fully connected layer can analyze the feature information in the gait recognition target data and perform effective individual identification and classification. In practical applications, it helps the multi-person gait recognition technology to better understand and analyze individual behavior patterns and achieve more accurate and personalized applications.

[0060] The multi-person gait recognition model obtains the output result of the first BiLSTM layer based on the gait recognition target data and the first BiLSTM layer. The specific implementation content is as follows:

[0061] First, the multi-person gait recognition model receives the gait recognition target data as input. The above data is fed into the first BiLSTM layer of the model. BiLSTM is a special recurrent neural network (RNN) that combines forward LSTM and backward LSTM to simultaneously capture the forward and backward dependencies in sequence data. In multi-person gait recognition, since gait data is temporal, the BiLSTM layer can make full use of temporal information to extract dynamic features and contextual dependencies in gait data.

[0062] Recurrent neural networks (RNNs) have certain advantages in dealing with time series regression problems, but there are problems such as gradient vanishing or gradient exploding. Since the hidden state of a recurrent neural network (RNN) is affected by both the current input and the hidden state of the previous moment, when faced with a long sentence, it is difficult for the model to maintain memory of the beginning information. In this embodiment, in order to solve the above problem, LSTM introduces a gating mechanism to achieve selective retention of information. Compared with the original recurrent neural network (RNN) model, it no longer retains all information indiscriminately, thereby significantly improving the operating efficiency of the model. For a complete LSTM network model diagram, please refer to Figure 4 .

[0063] Since the LSTM network module is the basic building block of BiLSTM, but the LSTM network module only relies on the forward information input neural network for prediction, in this embodiment, BiLSTM can use both forward and backward information at the same time, and obtain more comprehensive context information through the bidirectional information flow neural network, so as to make more accurate predictions. For a schematic diagram of the BiLSTM structure, please refer to Figure 5 .

[0064] During the processing of the first BiLSTM layer, the model will learn another set of weights and bias parameters, which are continuously optimized through training. During the training process, the model will try to minimize the difference between the predicted value and the true value, and update the weights and biases of the BiLSTM layer through the back-propagation algorithm, so that it can better capture the key information in the gait data.

[0065] Finally, the first BiLSTM layer outputs the first BiLSTM layer output result. The above result mainly contains the feature vector or sequence of time series information. The feature vector or sequence not only retains the time series of gait data, but also extracts the key features in the data, providing strong support for subsequent classification or recognition tasks.

[0066] In this embodiment, through the processing of the first BiLSTM layer, the multi-person gait recognition model can more effectively utilize the temporal information and context dependency in the gait recognition target data, thereby improving the accuracy and robustness of the multi-person gait recognition model.

[0067] The multi-person gait recognition model obtains the output result of the second BiLSTM layer based on the gait recognition target data and the second BiLSTM layer. The specific implementation content is as follows:

[0068] The gait recognition target data enters the second BiLSTM layer of the model. The second BiLSTM layer uses the characteristics of BiLSTM to consider both the forward and reverse dependencies of the input data, thereby capturing the deep context information in the gait sequence. This two-way information processing method enables the model to understand the gait data more comprehensively and improve the accuracy of recognition.

[0069] During the processing of the second BiLSTM layer, the model will learn and optimize another set of weights and bias parameters so that the model can better fit and recognize different gait data. Through algorithms such as forward propagation and back propagation, the model will gradually adjust the above parameters to make the output results of the second BiLSTM layer more accurate and reliable.

[0070] Finally, the second BiLSTM layer outputs the second BiLSTM layer output result. The above result mainly includes the key gait information processed by the second BiLSTM layer, which helps the multi-person gait recognition model to realize accurate recognition of multi-person gaits.

[0071] Furthermore, the method of obtaining different output results in the present embodiment is only an optional condition of the present invention. In one or some other embodiments, the output method can be adjusted according to the actual data situation and the hierarchical decomposition requirements, so as to effectively extract and process the context information in the gait sequence, thereby improving the accuracy of multi-person gait recognition.

[0072] S3. Add the residual of the output result of the fully connected layer and the output result of the first BiLSTM layer to obtain the first residual output result. The specific implementation steps and related contents are as follows:

[0073] Residual addition can improve the practical performance of the multi-person gait recognition model, and the original input or the output of a layer is directly added to the output of the subsequent layer through a skip-layer connection. In this embodiment, the output result of the fully connected layer and the output result of the first BiLSTM layer are added to the corresponding elements. The above addition operation is performed on the feature dimension, requiring the feature dimensions of the two to be the same. On the other hand, the residual network introduces skip-layer connections, making it easier for the network to learn the identity mapping, thereby alleviating the gradient vanishing and representation bottleneck problems in the training of deep neural networks. At the same time, residual addition allows the gradient to flow directly back to the previous layer, which helps to alleviate problems such as gradient vanishing, thereby improving the training speed and performance of the multi-person gait recognition model.

[0074] After the first residual addition, the result obtained is the first residual output result. In the embodiment, the first residual output result mainly includes the global features extracted by the fully connected layer and the temporal features extracted by the first BiLSTM layer. By adding the residuals of the output results of the fully connected layer and the first BiLSTM layer, the fusion of global features and temporal features is achieved, which provides richer feature information for subsequent multi-person gait recognition tasks, helps the model to understand the input data more comprehensively, and improves the accuracy of recognition.

[0075] Furthermore, in this embodiment, the method of adding the residuals of the output results of the fully connected layer and the output results of the first BiLSTM layer is only an optional condition of the present invention. In one or some other embodiments, the analysis method can be optimized according to the target requirements and actual application conditions to provide richer feature information for the multi-person gait recognition model.

[0076] S4. Obtain a second residual output result according to the first residual output result and the second BiLSTM layer output result. The specific implementation steps and related contents are as follows:

[0077] First, the first residual output result is used as the input of the second BiLSTM layer.

[0078] The second BiLSTM layer continues to use the characteristics of the bidirectional long short-term memory network to capture the deep temporal dependencies and contextual information in the input sequence. After being processed by the second BiLSTM layer, it will output a feature sequence containing richer temporal features. After obtaining the output result of the second BiLSTM layer, it is residually connected with the first residual output result.

[0079] In this embodiment, the first residual output result is directly added to the output of the current layer, i.e., the second BiLSTM layer, through a skip layer connection, and then the corresponding elements are added. The above addition is performed on the feature dimension, and the feature dimensions of the first residual output result and the second BiLSTM layer output result must match.

[0080] The result obtained after residual addition is the second residual output result. The above second residual output result combines the global and temporal features in the first residual output result, as well as the temporal features further extracted by the second BiLSTM layer, to achieve further enhancement and fusion of features. Through continuous BiLSTM layers and residual connections, the multi-person gait recognition model can gradually extract and fuse feature information at different levels, which helps to improve the multi-person gait recognition model's ability to represent complex gait data.

[0081] In the embodiment, residual learning not only helps to alleviate the gradient vanishing problem in deep neural network training, but also improves the training efficiency and performance of the model. This model using the skip-layer connection method can more easily learn the identity mapping or approximate identity mapping, thereby accelerating the training process and improving the model accuracy.

[0082] S5. Perform attention analysis on the second residual output result based on the TimeDistributed attention layer to obtain a final attention output result, and identify the gaits of multiple people according to the final attention output result. The specific implementation steps and related contents are as follows:

[0083] In an optional embodiment, attention is calculated in various ways, among which additive attention and multiplicative attention are two common methods. In this embodiment, the additive attention mechanism is adopted, which has been widely used in the field of machine translation, and its core lies in using a fully connected network to calculate the attention weight.

[0084] In order to better adapt to the sequence data characteristics of gait recognition, this embodiment improves the existing additive attention mechanism and introduces the TimeDistributed layer to help the model be more flexible and efficient when processing time series data. Based on this, it can better capture the key information in the gait sequence, thereby improving the accuracy of gait recognition. In this embodiment, the TimeDistributed layer is introduced to the additive attention mechanism, so that the multi-person gait recognition model can adapt to the various needs of gait recognition.

[0085] In an optional embodiment, let the word sequence received by the input encoder be X=(x1, x2, ..., x T ), in order to make full use of the context information in the sequence, a bidirectional recurrent neural network (Bi-RNN) is used as an encoder in the embodiment. The forward recurrent neural network encodes x in the order of the word sequence to generate a forward hidden state sequence that satisfies the following relationship:

[0086]

[0087] Among them, H forward represents the forward hidden state sequence, Represents the forward hidden state. Each of the above forward hidden states represents the information from the beginning of the sequence to the current word.

[0088] At the same time, the reverse recurrent neural network starts from the end of the sequence and encodes x in reverse order to produce a reverse hidden state sequence that satisfies the following relationship:

[0089]

[0090] Among them, Hbackward represents the backward hidden state sequence, represents the backward hidden state, and each backward hidden state in the sequence captures the information from the current word to the end of the sequence.

[0091] In order to integrate the information captured by the forward and reverse recurrent neural networks, the two hidden states corresponding to each word are concatenated to obtain a new state. The above state contains both the information before and after the word and satisfies the following relationship:

[0092]

[0093] Among them, h i Indicates the encoding result of the text. represents the forward hidden state, Represents the backward hidden state. Therefore, the encoding result of the input text sequence integrates the bidirectional information state and constitutes a sequence. The above encoding process not only considers the order of words, but also fully considers the interdependence between words, which helps the model understand the meaning of the text more accurately.

[0094] After encoding the sentence, it is further decoded using a unidirectional LSTM combined with an attention mechanism. Based on the hidden state sequence after encoding, a decoding sequence corresponding to the input sentence is gradually generated. The unidirectional LSTM is responsible for capturing the temporal dependencies in the decoding process, while the attention mechanism helps the model dynamically focus on the key parts of the encoding sequence during decoding, thereby enabling a more accurate and flexible decoding process.

[0095] The above decoding process satisfies the following relationship:

[0096] P(y i |y1,…,y i-1 , x) = g(y i-1 ,s i , c i )

[0097] Among them, P represents the conditional probability, y i represents the target word, y1 represents the first target word, y i-1 represents the previous target word, x represents the input text, g represents the activation function, s i represents the latent vector of the LSTM decoder at the i-th time step, c i It means that Additive Attention depends on the context encoding result. In this embodiment, it is based on Got it.

[0098] Among them, encoding through Additive Attention depends on the context and needs to satisfy the following relationship:

[0099]

[0100] Among them, c i Indicates the encoding result that depends on the context through Additive Attention, a ij represents the hidden vector of the i-th word in the LSTM decoder s i and the weight between the Bi-RNN encoder for the jth word encoding vector, h j Indicates hidden state.

[0101] In the process of decoding using unidirectional LSTM combined with attention mechanism, weights act as a bridge, connecting each hidden state in the encoded sequence with the current decoding step. The above weights not only reflect the importance of each hidden state in the encoded sequence for the current decoding step, but also allow the decoder to dynamically adjust its focus during the decoding process. By comparing or calculating the similarity between the current state in the decoder and the encoder hidden state sequence, a corresponding weight can be assigned to each hidden state. The above weights are then used to perform a weighted summation of the encoder's hidden states to generate a context vector. The context vector is used as the input of the decoder's next step, helping the decoder to focus more accurately on the key parts of the input sequence, thereby achieving a more accurate and flexible decoding process. Dynamically adjusting weights based on the attention mechanism allows the decoder to pay different attention to different parts of the input sequence during decoding, thereby optimizing the decoding effect.

[0102] The above weights satisfy the following relationship:

[0103]

[0104] Among them, a ij represents the hidden vector s of the i-th word of the LSTM decoder i The weight calculated between the Bi-RNN encoder and the j-th word encoding vector, e ij represents the attention model, which scores the matching degree between the input around position j and the output at position i, e ik It indicates the score of the matching degree between the input around position k and the output at position i.

[0105] e ij =a(s i-1 ,h j )

[0106] Among them, e ijIt represents the score of the matching degree between the input around position j and the output of position i, a represents a feedforward neural network, s i-1 represents the latent vector of the i-1th time step of the LSTM decoder, h j Indicates hidden state.

[0107] Since the decoding process is equivalent to a sequence generation task, the decoding result of the current position not only depends on the decoding output of the previous time step, but also needs to comprehensively consider the encoding information of each position in the input sequence. Therefore, in this embodiment, the decoding result of the previous position and the encoding results of each position are input into the feedforward neural network for calculation, based on which the correlation between the information at different positions is learned, and the corresponding weights are calculated. After the weights are normalized by the softmax function, their sum is 1, thereby ensuring reasonable attention to the information at different positions during the decoding process.

[0108] In this embodiment, the attention mechanism is introduced to realize dynamic attention and effective utilization of important features of the input sequence, improve the accuracy and efficiency of the decoding results, and not only improve the performance of the multi-person gait recognition model, but also provide new ideas and methods for related language processing tasks.

[0109] In this embodiment, the forward hidden state result of the second residual output result is obtained based on the TimeDistributed attention layer, and its specific implementation content is as follows:

[0110] In the forward propagation process, for any time t i (i=1, 2, ..., n) performs forward calculation and saves the output of the forward hidden layer at the current moment. The above outputs together constitute the forward hidden layer output sequence H f , the hidden layer output at the current moment is calculated based on the current input and the hidden layer state at the previous moment.

[0111] By saving the forward hidden layer output at each moment, the model can use the above information to perform more complex calculations in subsequent steps, including but not limited to the calculation and decoding of attention weights. The method of saving intermediate results in the embodiment enables the model to flexibly utilize historical information, thereby improving the expressiveness and practical performance of the model.

[0112] The above forward hidden state results satisfy the following relationship:

[0113] H f =F(w1x t +w2H f-1 -1)

[0114] Among them, H f represents the forward hidden state at time f, F represents the activation function, w1 represents the first weight, xt represents the input at time t, w2 represents the second weight, H f-1 Represents the forward hidden state at time f-1. Through the forward calculation method in the forward propagation process, the model can gradually build a hidden layer representation of the input sequence, providing strong support for subsequent task processing.

[0115] Then, based on the TimeDistributed attention layer, the backward hidden state result of the second residual output result is obtained, and its specific implementation content is as follows:

[0116] During the back propagation process, for any time t i (i=n,…,2,1) performs reverse calculation and saves the output of the backward hidden layer at the current moment. By starting the backward calculation from the end of the sequence, the model can capture the information from the end of the sequence to the current moment, which helps the model to more comprehensively understand the context of the entire sequence and save the backward hidden layer output at each moment, so that the model can combine the forward and backward information for more accurate calculations and decisions in subsequent steps.

[0117] The above backward hidden state results satisfy the following relationship:

[0118] H b =F(w3x t +w5H b+1 -1)

[0119] Among them, H b represents the backward hidden state at time b, F represents the activation function, w3 represents the third weight, x t represents the input at time t, w5 represents the fifth weight, H b+1 represents the backward hidden state at time b+1.

[0120] By combining the input information at the current moment and the backward hidden state at the next moment, the model can better capture the contextual information in the sequence, which helps the model understand the overall structure of the sequence and consider a wider context when decoding, thereby improving the accuracy of the decoding results.

[0121] Since the above weights are learnable parameters, the model can automatically adjust the weights according to the training data to adapt to different tasks and data distributions, making the model more flexible and adaptable, and able to perform well in different scenarios.

[0122] In the decoding process combined with the attention mechanism, the calculation method of the backward hidden state helps the model to allocate attention weights more accurately. By comparing the current state in the decoder with the hidden state sequence of the encoder, the model can dynamically focus on different parts of the input sequence, making the decoding process more efficient and accurate.

[0123] Finally, the forward hidden state result and the backward hidden state result of the second residual output result are combined to obtain the final attention output result of the second residual output result, and the specific implementation content is as follows:

[0124] In the embodiment, the forward hidden state result and the backward hidden state result capture the information of the sequence from different directions respectively. In order to fully utilize the above information, the corresponding outputs of the forward and backward directions at any time are combined to obtain the final output result. The above combination method can integrate the forward and backward information of the sequence, provide more comprehensive and accurate information support for the decoding process, and based on this model, the overall structure and contextual relationship of the sequence can be better understood, thereby improving the accuracy and efficiency of decoding.

[0125] The above final attention output results satisfy the following relationship:

[0126] Y t =g(w4H f +w6H b )

[0127] Among them, Y t represents the output of the multi-person gait recognition model at time t, g represents the activation function, w4 represents the fourth weight, H f represents the forward hidden state at time f, w6 represents the sixth weight, H b Represents the backward hidden state at time b.

[0128] In an embodiment, the output of the multi-person gait recognition model at time t is the final attention output result of the second residual output result, and the forward and backward hidden states are fused to obtain more comprehensive and in-depth sequence information, which can further optimize the decoding result.

[0129] The multi-person gait recognition model also includes result analysis. Based on the model analysis results, the performance of the multi-person gait recognition model can be accurately evaluated, the performance bottleneck of the model can be understood, and then the improvement and optimization of the model can be guided to improve the accuracy of the recognition results.

[0130] In an optional embodiment, the multi-person gait recognition model is a classifier designed based on multivariate time series, and its performance evaluation depends on multiple key indicators, including but not limited to accuracy (A), error value (loss), and confusion matrix, etc. The accuracy reflects the proportion of correct predictions of the model in all samples, and its calculation formula is as follows:

[0131] A=(TP+TN) / (TP+TN+FP+FN)×100%

[0132] Among them, A represents the accuracy, TP represents the true positive example, TN represents the true negative example, FP represents the false positive example, and FN represents the false negative example.

[0133] A true positive example refers to a situation where the model correctly predicts a sample that is actually a positive class as a positive class, that is, the model correctly identifies the positive class sample; a true negative example refers to a situation where the model correctly predicts a sample that is actually a negative class as a negative class, that is, the model correctly excludes the negative class sample. Both of the above situations reflect the correct prediction ability of the model in the classification task, where a true positive example reflects the model's recognition accuracy for positive class samples, and a true negative example reflects the model's exclusion accuracy for negative class samples. Statistics and analysis of the two situations can give us a comprehensive understanding of the performance of the model.

[0134] A false positive example refers to a situation where the model mistakenly predicts a negative sample as a positive sample, that is, a false alarm occurs in the model, which means that the model mistakenly identifies some samples that should not be identified as positive as positive, resulting in an incorrect positive prediction; a false negative example refers to a situation where the model mistakenly predicts a positive sample as a negative sample, that is, a false negative occurs in the model, indicating that the model fails to correctly identify positive samples and mistakenly classifies samples that should belong to the positive class as negative, resulting in the omission of positive samples. Based on the number of false positives and false negatives, we can further understand the type and degree of errors of the model in the classification task, so as to optimize the model performance in a targeted manner, reduce the number of false positives and false negatives, and help improve the accuracy of the model.

[0135] The confusion matrix provides an intuitive way to show the comparison between the model prediction results and the actual labels. The confusion matrix is ​​usually a two-dimensional table, where the rows represent the actual categories and the columns represent the predicted categories. The confusion matrix can clearly show the prediction accuracy of the model in each category, including the true positive rate of the positive class, the true negative rate of the negative class, the false alarm rate and the missed alarm rate. The above evaluation indicators together constitute a complete framework for the performance evaluation of the gait recognition model, which helps to fully and accurately understand the performance of the model and provide an important basis for the optimization and improvement of the model.

[0136] To further demonstrate the performance changes of the gait recognition model during training and validation, see Figure 6 and Figure 7 , compare and analyze the accuracy and loss rate of the training data set and the validation data set. By comparing the accuracy and loss rate of the training data set and the validation data set, it is possible to promptly discover whether the model is overfitting or underfitting, which is of great significance for adjusting the model structure, optimizing the algorithm or adjusting the hyperparameters, and helps to improve the generalization ability of the model.

[0137] Based on the image comparison results, the performance changes of the model during the training and verification process can be intuitively displayed, which helps to evaluate the quality of the model. At the same time, based on the performance of the model in different cycles, the training strategy can be adjusted, including but not limited to increasing the number of training rounds, adjusting the learning rate, introducing regularization, etc., to optimize the performance of the model.

[0138] In addition, based on the image, it can be seen that with the increase in the number of iterations, both the training set and the validation set have a steady upward trend in accuracy, while the loss value gradually decreases. The above information shows that the model has a good fitting effect on the data, and its performance is constantly improving. The model in the embodiment has an accuracy of up to 99.20% on the training set and 98.20% on the test set.

[0139] To further verify the classification effect of the model on different categories and facilitate a comprehensive evaluation of the model performance, please refer to Figure 8 The confusion matrix diagram shows the predicted labels and the true labels. Based on the confusion matrix diagram, we can comprehensively evaluate the classification effect of the model on different categories, find potential problems and formulate corresponding improvement strategies, which is helpful to improve the performance of the model, optimize feature selection and guide practical applications.

[0140] In the process of training the multi-person gait recognition model, the loss values ​​of the training set and the validation set continued to decrease while the accuracy rate steadily increased. In this embodiment, the early stopping function is used to effectively stop the model training at the 20th model training, avoiding the occurrence of overfitting. According to the test results, the model performs well in the multi-person action classification task with an accuracy rate of up to 98.20%. In addition, the model's precision, recall rate and F1 score are 98.32%, 98.61% and 98.46% respectively, further proving the excellence of the model performance.

[0141] In order to comprehensively evaluate the performance of the multi-person gait recognition model, the accuracy, precision, and recall rates of the model are compared and analyzed with those of three existing motion classification algorithms. For the specific comparative analysis, please see Table 1.

[0142] Table 1

[0143]

[0144] Table comparing indicators of different action classification algorithms

[0145] Among them, the CNN-LSTM model combines convolutional neural networks and long short-term memory networks. Although its overall accuracy is slightly lower than that of the multi-person gait recognition model of this embodiment, the C4M4BL model adopts a block-based multi-scale convolutional neural network. Although it has a certain gap with the previous two in terms of overall accuracy and recall rate, it still shows competitiveness in some indicators. The CNN model is slightly inferior in overall performance, especially in terms of average single-person accuracy.

[0146] Based on this, it can be seen that the multi-person gait recognition model of this embodiment shows significant advantages in multi-person action recognition tasks. It not only has a high overall accuracy and precision, but also maintains a stable average single-person accuracy rate, which fully demonstrates the superior performance and generalization ability of the multi-person gait recognition model of this embodiment in multi-person action recognition tasks in urban environments.

[0147] See also Fig. 9 In an optional embodiment, in order to efficiently execute the multi-person gait recognition model provided by the present invention, the present invention also provides a multi-person gait recognition system, the multi-person gait recognition system comprising a processor, an input device, an output device and a memory, the processor, the input device, the output device and the memory are interconnected, wherein the memory is used to store a computer program, the computer program comprises program instructions, the processor is configured to call the program instructions, and execute the specific steps of the multi-person gait recognition model and related embodiments provided by the present invention. The multi-person gait recognition system of the present invention has a complete structure and is objective and stable.

[0148] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. A multi-person gait recognition model, characterized in that: The steps include: Collecting plantar pressure sensor data and human body part acceleration data, obtaining gait recognition initial data based on the plantar pressure sensor data and the human body part acceleration data, processing the gait recognition initial data, and obtaining gait recognition target data; Obtaining a fully connected layer output result, a first BiLSTM layer output result, and a second BiLSTM layer output result based on the gait recognition target data; Perform residual addition on the output result of the fully connected layer and the output result of the first BiLSTM layer to obtain a first residual output result; Obtaining a second residual output result according to the first residual output result and the second BiLSTM layer output result; Performing attention analysis on the second residual output result based on the TimeDistributed attention layer to obtain a final attention output result, and recognizing the gaits of multiple people according to the final attention output result; The performing attention analysis on the second residual output result based on the TimeDistributed attention layer to obtain the final attention output result includes: Obtain a forward hidden state result of a second residual output result based on the TimeDistributed attention layer; Obtaining a backward hidden state result of a second residual output result based on the TimeDistributed attention layer; The forward hidden state result satisfies the following relationship: , in, express The forward hidden state at time t, represents the activation function, represents the first weight, express Input at the moment, represents the second weight, express The forward hidden state at time t; The backward hidden state result satisfies the following relationship: , in, express The backward hidden state at time t, represents the activation function, represents the third weight, represents the input at time t, represents the fifth weight, Indicates The backward hidden state at the moment; The performing attention analysis on the second residual output result based on the TimeDistributed attention layer to obtain a final attention output result, and recognizing the gaits of multiple people according to the final attention output result includes: Combining the forward hidden state result and the backward hidden state result of the second residual output result to obtain a final attention output result of the second residual output result; Recognize the gaits of multiple people according to the final attention output result; The final attention output result satisfies the following relationship: , in, Indicates that the multi-person gait recognition model is The output of the moment, represents the activation function, represents the fourth weight, express The forward hidden state at time t, represents the sixth weight, express The backward hidden state at time t.

2. The multi-person gait recognition model according to claim 1, characterized in that: The collecting of plantar pressure sensor data and human body acceleration data includes: 16 plantar sensors are arranged in the multi-person gait recognition model, and plantar pressure sensor data are collected based on the 16 plantar sensors; A human body three-axis acceleration part is preset in the multi-person gait recognition model; and human body part acceleration data is collected according to the human body three-axis acceleration part.

3. The multi-person gait recognition model according to claim 1, characterized in that: The processing of the initial gait recognition data and obtaining the gait recognition target data includes: The multi-person gait recognition model uses a filter to perform data smoothing processing on the initial gait recognition data to obtain the first gait recognition data; The multi-person gait recognition model divides the first gait recognition data using a sliding time window method to obtain second gait recognition data; The multi-person gait recognition model optimizes the second gait recognition data using the PCA technology to obtain gait recognition target data.

4. The multi-person gait recognition model according to claim 1, characterized in that: The obtaining of the fully connected layer output result, the first BiLSTM layer output result and the second BiLSTM layer output result based on the gait recognition target data comprises: The multi-person gait recognition model obtains a fully connected layer output result based on the gait recognition target data and the fully connected layer; The multi-person gait recognition model obtains a first BiLSTM layer output result based on the gait recognition target data and the first BiLSTM layer; The multi-person gait recognition model obtains a second BiLSTM layer output result based on the gait recognition target data and the second BiLSTM layer.

5. A multi-person gait recognition system, characterized in that: The system includes a processor, an input device, an output device and a memory, wherein the processor, the input device, the output device and the memory are interconnected, wherein the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the multi-person gait recognition model as described in any one of claims 1-4.

Citation Information

Patent Citations

  • Artificial limb gait recognition method, device and storage medium

    CN113780223A

  • Dynamic gesture recognition method based on hand key point and double-layer bidirectional LSTM network

    CN117576783A

  • Exoskeleton movement mode transfer identification method under complex terrain

    CN117911707A