A gait recognition method based on spatio-temporal correlation degree determination

Abstract

The application belongs to the technical field of biological recognition, and discloses a gait recognition method based on space-time correlation degree determination, which comprises the following steps: extracting a gait sequence of a human target from monitoring video data; constructing a city monitoring point map, establishing a vector representation of each node and an interaction relationship between elements of the vector based on the city monitoring point map, obtaining a space feature, obtaining a time feature according to the time of day at the current time, and generating an adaptive space-time representation map based on the space feature and the time feature; deeply fusing the gait feature vector and the adaptive space-time representation map to obtain a gait feature vector with fusion space-time correlation; and inputting the gait feature vector with fusion space-time correlation into a gait recognition network for gait recognition to obtain a gait feature. The application can more accurately perform gait recognition by capturing the spatial relationship between city monitoring points and the dynamic change of pedestrian gait.

Description

Technical Field

[0001] This invention relates to the field of biometrics, and in particular to a gait recognition method based on spatiotemporal correlation determination. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Gait recognition technology is an emerging biometric identification technology that primarily identifies individuals by analyzing their body shape (such as height, muscle mass, joint structure, and other physiological characteristics) and walking posture. Integrating gait database construction, gait recognition, gait retrieval, and large-scale tracking, gait recognition technology supports real-time intelligent interconnection of gait recognition across massive numbers of cameras, enabling instantaneous retrieval and location of massive historical and real-time video data. It has broad application prospects in areas such as smart cities and intelligent security.

[0004] In the field of intelligent video surveillance, gait recognition technology has advantages over general image recognition. Because pedestrians differ in their center of gravity, tendon and muscle strength, bone length, and bone density, gait recognition technology can identify individuals by extracting and analyzing these unique gait features. In practical applications of gait recognition technology, applying temporal and spatial correlations for gait retrieval is a complex but efficient process. During gait retrieval, the system uses this temporal and spatial correlation information to narrow the search scope and improve accuracy. For example, the system might first filter gait data that is temporally similar to the target gait pattern based on temporal correlation information, and then further filter gait data that is geographically close to the target based on spatial correlation information. Finally, the system performs a comprehensive analysis of the filtered gait data to identify the individual whose gait best matches the target gait. This process may involve weighing and comparing multiple gait features to ensure the accuracy and reliability of the retrieval results.

[0005] By applying temporal and spatial correlations for gait retrieval, gait recognition technology can quickly and accurately identify individuals matching specific gait patterns from large amounts of surveillance video or image data, significantly improving its usability. This has important application value for fields such as public security criminal investigation and personnel management, helping relevant departments improve work efficiency and accuracy, and maintain social security and stability.

[0006] Gait recognition technology is highly dependent on time and space. For example, lighting conditions at different times can affect the gait characteristics of pedestrian gait sequences, and the angle and orientation of monitoring points at different locations can also affect the extracted gait features, making the gait recognition process more difficult. However, existing methods have certain limitations in extracting the spatiotemporal relationships in gait features. They only impose time and space constraints during gait retrieval and do not extract deeper relationships between individual gait features, thus failing to fully utilize the deep spatiotemporal representations inherent within them. Summary of the Invention

[0007] To address the aforementioned issues, this invention proposes a gait recognition method based on spatiotemporal correlation determination. By leveraging spatiotemporal information to establish a graph neural network based on urban monitoring points for gait recognition, and by capturing the spatial relationships between urban monitoring points and the dynamic changes in pedestrian gait, gait recognition can be performed more accurately.

[0008] To achieve the above objectives, the present invention adopts the following technical solution:

[0009] This invention provides a gait recognition method based on spatiotemporal correlation determination, comprising the following steps:

[0010] Acquire surveillance video data and extract the gait sequence of human targets from the surveillance video data;

[0011] Each monitoring point is treated as a node, and a city monitoring point map is constructed. Based on the city monitoring point map, a vector representation of each node and the interaction relationship between each element of the vector are established to obtain the spatial features of the gait sequence. According to the current time of day, the temporal features of the gait sequence are obtained. Based on the spatial and temporal features, an adaptive spatiotemporal representation map is generated.

[0012] Gait feature vectors are deeply fused with adaptive spatiotemporal representation maps to obtain gait feature vectors that incorporate spatiotemporal correlations;

[0013] Gait feature vectors that incorporate spatiotemporal correlations are input into a gait recognition network for gait recognition to obtain gait features.

[0014] As an alternative implementation, a spatiotemporal representation map convolutional layer is used to deeply fuse gait feature vectors with an adaptive spatiotemporal representation map. The spatiotemporal representation map convolutional layer is constructed based on a spatiotemporal representation map convolution operator, and multiple layers of the spatiotemporal representation map convolutional layer are stacked.

[0015] As an alternative implementation, skip connections are used in stacked multi-layer spatiotemporal representation graph convolutional layers.

[0016] As an alternative implementation method, based on the urban monitoring point location map, the vector representation of each node is established as follows:

[0017] Acquire relevant data for each monitoring point, extract multiple features that represent the characteristics of each monitoring point, and combine the multiple feature values ​​of each monitoring point into a multi-dimensional vector.

[0018] As an alternative implementation method, based on the urban monitoring point location map, the interaction relationships between the elements of the vector of each node are established, specifically as follows:

[0019] Establish hypotheses about possible interaction relationships between the features of monitoring points, and quantify the defined interaction relationships into a numerical value.

[0020] As an alternative implementation, the time features are mapped to one-hot codes based on which time of day the current time is, and then mapped to D-dimensional space using a fully connected network.

[0021] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0022] This disclosure proposes a gait recognition method based on spatiotemporal correlation. It utilizes spatiotemporal information to establish a graph neural network (GNN) based on urban monitoring points for gait recognition, which is of great significance for improving the efficiency of urban safety management and intelligent monitoring systems. The GNN can handle complex graph-structured data, extracting pedestrian gait features in real time while fusing input temporal and spatial features. By capturing the spatial relationships between urban monitoring points and the dynamic changes in pedestrian gait, it can more accurately recognize gait. Furthermore, the established GNN can cope with various complex scenarios, such as changes in lighting, occlusion, and pedestrian posture changes, thereby improving the robustness of the gait recognition system. Through efficient algorithm and model design, the GNN can achieve real-time gait recognition, providing timely and effective support for urban safety management.

[0023] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0024] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0025] Figure 1 This is a flowchart illustrating the framework of the gait recognition method based on spatiotemporal correlation determination in Embodiment 1 of the present invention. Detailed Implementation

[0026] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0027] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0028] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0029] Example 1

[0030] like Figure 1 As shown, this embodiment provides a gait recognition method based on spatiotemporal correlation determination, including the following steps:

[0031] S1 acquires surveillance video data and extracts the gait sequence of the human target from the surveillance video data;

[0032] S2 treats each monitoring point as a node, constructs a city monitoring point map, establishes the vector representation of each node and the interaction relationship between each element of the vector based on the city monitoring point map, obtains the spatial features of the gait sequence, obtains the temporal features of the gait sequence according to the current time of day, and generates an adaptive spatiotemporal representation map based on the spatial and temporal features.

[0033] S3 performs deep fusion of gait feature vectors with adaptive spatiotemporal representation maps to obtain gait feature vectors that incorporate spatiotemporal correlations;

[0034] S4 inputs the gait feature vectors that integrate spatiotemporal correlations into the gait recognition network to perform gait recognition and obtain gait features.

[0035] The purpose of this disclosure is to capture the spatial relationships between monitoring points and the dynamic changes in pedestrian gait using graph neural network algorithms. Specifically, this disclosure proposes a spatiotemporal representation map construction module, which, while extracting pedestrian gait features in real time, fuses input temporal and spatial features, performs unsupervised adjustments to embed the spatiotemporal representation, and generates an adaptive spatiotemporal representation map. The real-time gait sequence extracted under monitoring is processed by a fully connected layer and input together with the adaptive spatiotemporal representation map into the spatiotemporal representation map convolution module. The spatiotemporal representation map convolution module is constructed based on the spatiotemporal representation map convolution operator and consists of multiple stacked spatiotemporal representation map convolution layers. The spatiotemporal representation map convolution layers fuse gait feature vectors with spatiotemporal representations. This module also uses skip connections to fully utilize the depth of the neural network, enabling the neural network to learn deeper feature representations and improve the representational ability of the fused features. The gait feature vector with fused spatiotemporal features is input into a conventional gait recognition network for gait feature extraction, and the extracted gait features have high generalization ability.

[0036] The spatiotemporal representation map construction module of this application aims to simultaneously extract the dynamic temporal correlation, spatial correlation, and cross-spatiotemporal correlation among pedestrian gait sequences. Specifically, each monitoring point is treated as a node, and a city monitoring point map is constructed. Based on this map, a spatial node embedding vector is first randomly initialized for each node, and then a fully connected layer is used to map the spatial node embedding vector into a D-dimensional space. Since pedestrian gait sequences are affected by illumination at different times, this disclosure introduces temporal features to construct the spatiotemporal representation embedding vector. This disclosure considers one temporal feature: the position of the current time t within the day. This temporal feature is mapped to a one-hot encoding, and a fully connected network is used to map it into a D-dimensional space. At each node at each time point, the spatial node embedding vector and the temporal feature vector are added together to obtain the spatiotemporal embedding vector for all nodes at each time point, and this vector, along with the gait feature vector, is used as input to the model training process.

[0037] A graph is constructed based on urban surveillance camera locations, where each camera location is considered a node. All nodes in the graph are distributed in the same space. The vector representation of each node in this space and the interaction relationships between the elements of the vectors jointly influence the relationships between the nodes. Two matrices are constructed based on the urban surveillance camera location map: E∈R N×D Let represent the D-dimensional vectors of the N nodes in the graph in this space, where B∈R. D×D EBE represents the interaction relationships between the elements of a D-dimensional vector. T It can represent the connections between nodes. It is continuously adjusted during model training.

[0038] Based on the spatiotemporal embedding method, the adaptive spatiotemporal combined graph adjacency matrix at time t can be defined as:

[0039]

[0040]

[0041] The main purpose of the g(·) function is to discard weak connections (less than a specified threshold δ) in the adjacency matrix, while the softmax function is used for normalization. t This represents the connection relationship between N nodes at time t.

[0042] The purpose of establishing matrix E (node ​​vector matrix): Matrix E is used to represent the position or characteristics of each node (i.e., monitoring point) in the D-dimensional space. Its establishment process is as follows:

[0043] Data Collection: First, it is necessary to collect relevant data about each monitoring point. This data includes geographical location (latitude and longitude), monitoring range, monitoring angle, and historical monitoring records (such as pedestrian flow, vehicle flow, etc.).

[0044] Feature extraction: Based on the collected data, extract D features that represent the characteristics of each monitoring point. These features can be direct (such as latitude and longitude) or calculated or transformed (such as an activity index calculated based on historical data).

[0045] Vectorization: The D feature values ​​of each monitoring point are combined into a D-dimensional vector, and these vectors form the rows of matrix E. Therefore, E is an N×D matrix, where N is the number of monitoring points and D is the number of features.

[0046] The purpose of establishing matrix B (interaction matrix): Matrix B is used to represent the interaction relationships between the elements (i.e., features) in a D-dimensional vector. Its establishment process is as follows:

[0047] Relationship Definition: First, hypothesize the possible interactive relationships between features. These relationships may be based on physical meaning (such as the interaction between geographical location and monitoring range) or on data-driven factors (such as correlations between features discovered through statistical or machine learning methods).

[0048] Quantization of Relationships: The defined interaction relationships are quantified into a numerical value and filled into matrix B. B is a D×D matrix where the element bij represents the interaction strength between the i-th feature and the j-th feature. This value is automatically adjusted by an optimization algorithm during model training.

[0049] Initialization: Before model training begins, B needs to be randomly initialized.

[0050] Matrix EBE TThe significance is: by performing matrix multiplication of E and B, and then multiplying by the transpose of E (i.e., EBE), T This yields a matrix representing the connection strength between nodes. Each element of this matrix (EBE) T ) ij Both reflect the connection strength between node i and node j, which is calculated based on their positions in D-dimensional space (denoted by E) and the interaction relationships between the features at those positions (denoted by B).

[0051] During model training, the values ​​of E and B are continuously adjusted through optimization algorithms to minimize the loss function (a function that reflects the difference between the actual and predicted connections between nodes). As training progresses, E and B will gradually approach the optimal values ​​that accurately reflect the relationships between urban monitoring points.

[0052] Based on the spatiotemporal representation map construction module, this method also proposes a spatiotemporal representation map convolution module. By defining an adaptive spatiotemporal representation map convolution operator, an adaptive spatiotemporal representation map convolution layer is constructed to deeply fuse the input pedestrian gait sequence with spatiotemporal features.

[0053] The spatiotemporal representation map convolution operator is defined as:

[0054] P = f(OXW + b)

[0055] In the formula: X and P are the input and output signals of the graph convolution operator, respectively; W is the node weight; b is the bias; f(·) represents the activation function; and O is the normalized adjacency matrix.

[0056] Based on the spatiotemporal representation map convolution operator, spatiotemporal representation map convolutional layers can be constructed. First, a fully connected network is used to map the input gait sequence data into a D-dimensional space, and then the data is fed into multiple layers of spatiotemporal representation map convolutional layers. The spatiotemporal representation map convolution operator, based on the deep spatiotemporal correlation at a certain time point, can better fuse spatiotemporal features and gait feature vectors by stacking multiple layers of spatiotemporal representation map convolutional layers. Using skip connections in the spatiotemporal representation map convolution module not only mitigates the gradient vanishing problem but also enables the neural network to learn a deeper, more joint feature representation. Inputting this feature into the gait recognition network yields gait features that fuse spatiotemporal correlation. Retrieval and comparison in a gait data base completes the process of identifying the pedestrian target.

[0057] Those skilled in the art will recognize that the units and algorithm steps described in conjunction with the embodiments herein can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0058] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A gait recognition method based on spatiotemporal correlation determination, characterized in that, Includes the following steps: Acquire surveillance video data and extract the gait sequence of human targets from the surveillance video data; Each monitoring point is treated as a node, and a city monitoring point map is constructed. Based on the city monitoring point map, a vector representation of each node and the interaction relationship between each element of the vector are established to obtain the spatial features of the gait sequence. According to the current time of day, the temporal features of the gait sequence are obtained. Based on the spatial and temporal features, an adaptive spatiotemporal representation map is generated. The gait feature vector is deeply fused with the adaptive spatiotemporal representation map to obtain a gait feature vector that incorporates spatiotemporal correlation. Specifically, the gait feature vector is deeply fused with the adaptive spatiotemporal representation map using a spatiotemporal representation map convolutional layer. The spatiotemporal representation map convolutional layer is constructed based on the spatiotemporal representation map convolution operator and has multiple layers stacked. Gait feature vectors that incorporate spatiotemporal correlations are input into a gait recognition network for gait recognition to obtain gait features.

2. The gait recognition method based on spatiotemporal correlation determination as described in claim 1, characterized in that, Use skip connections in stacked multi-layered spatiotemporal representation graph convolutional layers.

3. The gait recognition method based on spatiotemporal correlation determination as described in claim 1, characterized in that, Based on the city's surveillance point location map, the vector representation of each node is established as follows: Acquire relevant data for each monitoring point, extract multiple features that represent the characteristics of each monitoring point, and combine the multiple feature values ​​of each monitoring point into a multi-dimensional vector.

4. The gait recognition method based on spatiotemporal correlation determination as described in claim 3, characterized in that, Based on the urban surveillance point location map, the interaction relationships between the elements of the vector of each node are established, specifically as follows: Establish hypotheses about possible interaction relationships between the features of monitoring points, and quantify the defined interaction relationships into a numerical value.

5. The gait recognition method based on spatiotemporal correlation determination as described in claim 1, characterized in that, Based on the current time of day, the time features are mapped to one-hot encodings, and then mapped to D-dimensional space using a fully connected network.

Images