A clothes-changing pedestrian re-identification method based on graph attention human body part dynamic correlation modeling

Abstract

The application discloses a clothes-changing pedestrian re-identification method based on graph attention human body part dynamic correlation modeling. The method first acquires a video sequence of a to-be-identified pedestrian, performs part segmentation on the video sequence through a human body analysis model; then extracts part features under different receptive fields and suppresses the background by using a part perception multi-scale feature extraction module; then combines each frame and the part to construct a graph node, and constructs a heterogeneous space-time part graph containing space edges, time edges and non-local edges; the graph attention part correlation modeling module performs message passing on the three types of edges, and fuses to obtain enhanced node features; then the enhanced node features are mapped back to the original space feature dimension; adaptive weighted integration is performed on different granularity features; finally, an identity representation vector is generated, similarity is calculated with a graph library feature, and a matching result is output. The method captures the time sequence dynamic correlation between human body parts, so that the identity representation generated by the model is more focused on stable human body structure rather than changeable clothing appearance.

Description

Technical Field

[0001] This invention relates to a method for re-identifying pedestrians changing clothes based on graph attention-based dynamic association modeling of human body parts, belonging to the field of computer vision technology. Background Technology

[0002] Pedestrian re-identification is an important research direction in the fields of computer vision and intelligent video analysis. Its goal is to achieve cross-time and cross-viewpoint identity matching and retrieval of the same pedestrian target under conditions of multiple cameras and non-overlapping fields of view, which has significant value in applications such as public safety, intelligent transportation, and urban management. With the development of deep learning technology, pedestrian re-identification methods based on appearance feature learning have made some progress under controlled conditions.

[0003] However, in practical applications, pedestrians often change their clothing at different times, resulting in significant changes in the appearance, color, texture, and local details of the same pedestrian. This makes it difficult to maintain consistency in feature representations that rely on clothing appearance, leading to a significant decline in pedestrian re-identification performance in clothing-changing scenarios. How to extract stable and discriminative identity features under clothing change conditions has become one of the important challenges facing current pedestrian re-identification research.

[0004] To mitigate the impact of clothing changes, some existing technologies have begun to incorporate methods such as human body analysis, keypoint detection, or body part segmentation to perform feature modeling at the human body structure level, thereby enhancing the ability to describe identity-related and relatively stable regions. However, these methods mostly focus on single-frame images or static spatial structure modeling, and do not adequately utilize the temporal evolution and cooperative motion relationships of various structural parts of the human body in video sequences.

[0005] In video-based pedestrian re-identification scenarios involving clothing changes, human body structures not only possess relatively stable spatial anatomical relationships but also exhibit individualized temporal collaborative patterns during movements such as walking. Different body parts differ in terms of movement amplitude, rhythm, and correlation strength, and dynamic correlations may also arise between parts across time and body parts. This dynamic correlation information between human body structures is more stable than clothing appearance and is of significant value for identity verification, but it has not yet been fully explored and effectively modeled in current technologies. Summary of the Invention

[0006] To address the aforementioned problems, this invention proposes a method for re-identifying pedestrians changing clothes based on graph attention-based dynamic association modeling of human body parts, comprising the following steps:

[0007] S1: Obtain the video sequence of the pedestrian to be identified, and obtain the part mask by segmentation through the human body analysis model;

[0008] S2: Based on the part mask, the part-aware multi-scale feature extraction module is used to perform part-aware multi-scale modeling of the input video features. The part-aware multi-scale feature extraction module constructs multiple receptive fields through spatiotemporal convolution kernels with different dilation rates to extract multi-scale features, and generates part-specific channel weights by combining part-specific channel attention branches, weighting and enhancing the feature channels of the corresponding parts and suppressing the response of the background region, and outputting part-aware multi-scale features.

[0009] S3: Construct each frame and corresponding part in the sequence as a graph node, establish spatial edges based on human anatomical topology, establish temporal edges based on cross-frame motion information, and establish non-local edges based on cross-frame and cross-part motion pattern similarity to form a heterogeneous spatiotemporal part graph containing spatial, temporal and non-local relationships.

[0010] S4: Incorporate part category encoding, temporal position encoding, and cross-frame motion information into node features, and perform multi-head graph attention message passing on spatial edges, temporal edges, and non-local edges through the graph attention part association modeling module;

[0011] S5: Generate the fusion weight of each node feature in the three types of relationships through the node gating mechanism, and perform weighted combination of the updated node features under the three types of relationships; then multiply the node-by-node importance weights output by the part importance prediction module to obtain the enhanced node features.

[0012] S6: Map the enhanced node features back to the original spatial dimension, and after normalization, inject them into the aforementioned part-aware multi-scale features in the form of residuals to obtain spatiotemporal structure-aware part features.

[0013] S7: Integrate the fine-grained part features and coarse-grained global features in the spatiotemporal structure perception part features with the graph-level spatiotemporal relationship features in the heterogeneous spatiotemporal part map through the multi-granularity feature fusion module to generate a clothing change robust identity representation vector; calculate the cosine similarity between this vector and the map library features, and generate a pedestrian matching list by sorting the similarity.

[0014] Furthermore, the part-aware multi-scale feature extraction module in S2 includes:

[0015] S21: Multiple sets of 3D convolutional branches are used, each set of convolutional branches uses spatiotemporal convolutional kernels with different dilation rates to perform convolution operations on the input video feature map to construct multiple receptive fields and extract multi-scale spatiotemporal features; each set of convolutional branches is connected in series with group convolution, batch normalization and activation function; after parallel processing by multiple sets of convolutional branches, multiple global feature maps with different receptive fields are obtained.

[0016] S22: For each part, multiply the corresponding part mask with the global feature map of each different receptive field in the spatial dimension to extract the local features of the part under different receptive fields; and fuse all the local features of the same part under different receptive fields to obtain the multi-scale fused features of each part.

[0017] S23: Configure a part-specific channel attention branch for each part. Each part-specific channel attention branch is processed by adaptive global spatiotemporal pooling and then passes through a channel mapping layer of dimensionality reduction and dimensionality increase and nonlinear activation in sequence to output the exclusive channel attention weight for that part.

[0018] S24: Multiply the multi-scale fusion features of each part with their corresponding dedicated channel attention weights to achieve weighted enhancement of the feature channels of the corresponding parts and suppress the response of the background region, thereby obtaining the local feature representation of each part;

[0019] S25: Represent the local features of all parts on the channel dimension and then stitch and fuse them to obtain part-aware multi-scale features.

[0020] Furthermore, the construction of the heterogeneous spatiotemporal region map in S3 includes:

[0021] S31: Spatial edges are initialized with a static adjacency matrix based on human anatomical topology, connecting only nodes in the same frame that have anatomical connections.

[0022] S32: The time edge only connects nodes of the same part in different frames. The maximum number of time jumps is adaptively determined based on the cross-frame motion information of the part. Bidirectional time connections are established within the range of not exceeding the maximum number of jumps.

[0023] S33: Non-local edges determine the connection relationship by combining the similarity of motion patterns of each part, and non-local edges exclude connections that are repeated with spatial and temporal edges, focusing on establishing cross-frame and cross-part associations, and retaining highly correlated connections through adaptive sparsification.

[0024] Furthermore, S4 includes:

[0025] S41: The part category code is generated through the embedding layer. The input of the embedding layer is the part index, and the output dimension matches the node feature dimension. The time code is generated through the embedding layer and the linear layer. The input of the embedding layer is the frame index, and the linear layer projects the output of the embedding layer onto the node feature dimension. The part category code and the time code are superimposed on the node features to achieve explicit encoding of the node identity features and time position.

[0026] S42: Velocity information is calculated from the feature difference of the same part in adjacent frames, and acceleration information is calculated from the velocity difference in adjacent frames. The velocity information and acceleration information are projected onto the node features through a linear layer respectively.

[0027] S43: The graph attention network employs a multi-head attention mechanism, where the number of heads is set according to preset hyperparameters. Node features are projected as query, key, and value vector representations. When calculating the attention score, a multi-head bias obtained from relation embedding mapping is introduced. The relation embedding is used to distinguish different relation types such as spatial edges, temporal edges, and non-local edges. An adjacency matrix mask is applied to the attention score, and after Softmax normalization and Dropout processing, the value vector features of adjacent nodes are weighted and summed. The weighted aggregation result is projected onto the output, and the updated node features are obtained through a non-linear activation function.

[0028] Furthermore, S5 includes:

[0029] S51: A node gating mechanism is used to generate fusion weights for each node on three types of relationships: spatial edges, temporal edges, and nonlocal edges. The node gating mechanism is implemented through a multilayer perceptron, which sequentially connects a linear layer, an activation function, a linear layer, and a normalized activation function. The fusion weights are multiplied by the updated node features under the three types of relationships and summed to achieve adaptive weighted fusion of node features under the three types of relationships, resulting in fused node features.

[0030] S52: An importance weight is generated for each node through the location importance prediction module; the location importance prediction module is implemented through a multilayer perceptron, which is a multilayer perceptron that sequentially connects a linear layer, an activation function, and another linear layer and an activation function; the importance weight is multiplied node by node by node with the fused node features to obtain enhanced node features.

[0031] Furthermore, S6 includes:

[0032] S61: Reshape the enhanced node features into a format of batch, frame number, number of parts, and feature dimension; exclude background nodes and retain only human body part nodes to obtain non-background node features and non-background part masks; use Einstein summation to weight and accumulate the non-background node features according to the corresponding non-background part masks in the spatial dimension to obtain a part structure enhancement feature map with the same shape as the part-aware multi-scale features output by S2.

[0033] S62: Calculate the spatial area of ​​the mask for each part and superimpose a small constant as a normalization factor to normalize the part structure enhancement feature map, prevent division by zero and cancel the difference in mask area of ​​different parts, and obtain the normalized part structure enhancement feature map.

[0034] S63: Perform a learnable interactive projection transformation on the normalized part structure enhancement feature map; introduce a learnable interactive scaling parameter, and add the scaled projected features to the part-aware multi-scale features output by S2 in the form of residuals to obtain the spatiotemporal structure-aware part features.

[0035] Furthermore, the multi-granularity feature fusion module in S7 includes:

[0036] S71: Fine-grained part features are six types of human body part features after excluding the background, obtained by global spatiotemporal pooling of spatiotemporal structure-aware part features within the masked region of each part; coarse-grained global features are the global spatiotemporal pooling results of spatiotemporal structure-aware part features; graph-level spatiotemporal relationship features are the mean pooling results of graph node features.

[0037] S72: Input the three types of features described in S71 into the multilayer perceptron to generate adaptive fusion weights; at the same time, introduce learnable global fusion weights and calculate the final fusion weights proportionally.

[0038] S73: The three types of features are weighted and summed using the final fusion weights to obtain the fusion features; the fusion features are then passed through a linear layer, layer normalization, and nonlinear activation, and then feature normalization is performed to output the identity representation vector.

[0039] Furthermore, the method for re-identifying pedestrians changing clothes based on graph attention-based dynamic association modeling of human body parts also includes a multi-layer stacking step of the multi-head graph attention message passing process of the graph attention part association modeling module in S4 and the node feature weighted fusion and enhancement process in S5: the multi-head graph attention message passing and node feature weighted fusion and enhancement processes are stacked in multiple layers, and the node features output by each layer together with the initial node features constitute a feature set; the feature set is weighted and summed by learnable layer weights, and the layer weights are normalized to improve the depth and robustness of the part dynamic association modeling.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] First, this invention uses human body analysis results to semantically segment pedestrian video sequences and performs feature modeling at the part level. It explicitly utilizes human body structure information to effectively reduce the interference of clothing appearance changes on identity representation and improves the robustness of pedestrian re-identification in clothing changing scenarios.

[0042] Second, this invention constructs a heterogeneous spatiotemporal human body part diagram that includes spatial relationships, temporal relationships, and non-local relationships. It performs unified modeling of the motion association and cooperative relationship between human body structural parts in different frames, which can fully explore the dynamic association information between human body structural parts that changes over time, thereby enhancing the ability to express stable features related to identity.

[0043] Third, this invention uses graph attention part association modeling and node feature adaptive fusion mechanism to dynamically model and weight the relationship between different types of parts and the importance of different parts, so that the model can highlight key parts and key relationships according to the specific spatiotemporal context, suppress redundant or unstable information, and further improve the discriminativeness and stability of identity representation.

[0044] Fourth, experimental results on the publicly available video person re-identification dataset VCCR show that the present invention has achieved significant improvements in key evaluation metrics such as mAP and Rank-1, fully verifying the effectiveness and advancement of the proposed method in the person re-identification task. Attached Figure Description

[0045] To make the technical solution and beneficial effects of the present invention clearer, the specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the accompanying drawings are only for illustrating the principle structure or method flow examples of the present invention, and are not intended to limit the scope of protection of the present invention. Those skilled in the art can make various modifications or substitutions to the shown structures or steps without departing from the spirit and substance of the present invention, and all such modifications or substitutions should fall within the scope of protection of the present invention.

[0046] Figure 1 A flowchart illustrating the steps of a method for re-identifying pedestrians changing clothes based on graph attention-based dynamic association modeling of human body parts, provided by this invention;

[0047] Figure 2 The overall framework diagram of the method for re-identifying pedestrians changing clothes based on graph attention dynamic association modeling of human body parts provided by the present invention;

[0048] Figure 3 This is an example image of a pedestrian matching list generated by a pedestrian re-identification method for changing clothes based on graph attention-based dynamic association modeling of human body parts, provided by the present invention. Detailed Implementation

[0049] To enable those skilled in the art to more intuitively and comprehensively understand the technical solution and implementation process of the present invention, the present invention will be further described below in conjunction with specific embodiments. It should be understood that the embodiments described below are only used to illustrate the principles and technical details of the present invention and do not constitute a limitation on the scope of protection of the present invention. Various modifications or substitutions can be made by those skilled in the art without departing from the spirit and substance of the present invention, and all such modifications or substitutions should fall within the scope of protection of the present invention.

[0050] like Figure 1 The diagram shows a flowchart of the steps in a method for re-identifying pedestrians changing clothes based on graph attention-based dynamic association modeling of human body parts, provided by the present invention. The steps include:

[0051] S1: Obtain the video sequence of the pedestrian to be identified, and obtain the part mask by segmentation through the human body analysis model;

[0052] S2: Based on the part mask, the part-aware multi-scale feature extraction module is used to perform part-aware multi-scale modeling of the input video features. The part-aware multi-scale feature extraction module constructs multiple receptive fields through spatiotemporal convolution kernels with different dilation rates to extract multi-scale features, and generates part-specific channel weights by combining part-specific channel attention branches, weighting and enhancing the feature channels of the corresponding parts and suppressing the response of the background region, and outputting part-aware multi-scale features.

[0053] S3: Construct each frame and corresponding part in the sequence as a graph node, establish spatial edges based on human anatomical topology, establish temporal edges based on cross-frame motion information, and establish non-local edges based on cross-frame and cross-part motion pattern similarity to form a heterogeneous spatiotemporal part graph containing spatial, temporal and non-local relationships.

[0054] S4: Incorporate part category encoding, temporal position encoding, and cross-frame motion information into node features, and perform multi-head graph attention message passing on spatial edges, temporal edges, and non-local edges through the graph attention part association modeling module;

[0055] S5: Generate the fusion weight of each node feature in the three types of relationships through the node gating mechanism, and perform weighted combination of the updated node features under the three types of relationships; then multiply the node-by-node importance weights output by the part importance prediction module to obtain the enhanced node features.

[0056] S6: Map the enhanced node features back to the original spatial dimension, and after normalization, inject them into the aforementioned part-aware multi-scale features in the form of residuals to obtain spatiotemporal structure-aware part features.

[0057] S7: Integrate the fine-grained part features and coarse-grained global features in the spatiotemporal structure perception part features with the graph-level spatiotemporal relationship features in the heterogeneous spatiotemporal part map through the multi-granularity feature fusion module to generate a clothing change robust identity representation vector; calculate the cosine similarity between this vector and the map library features, and generate a pedestrian matching list by sorting the similarity.

[0058] like Figure 2 The diagram shown illustrates the overall framework of a method for re-identifying pedestrians changing clothes based on graph attention-based dynamic association modeling of human body parts, as provided by this invention. It mainly includes a part-aware multi-scale feature extraction module, a graph attention part association modeling module, and a multi-granularity feature fusion module.

[0059] This invention provides a preferred embodiment for executing S2. This embodiment aims to extract multi-scale features by constructing multiple receptive fields based on part masks and using spatiotemporal convolution kernels with different dilation rates, and to enhance human structural part features and suppress background responses by combining part-specific channel attention mechanisms, ultimately outputting high-quality part-aware multi-scale features.

[0060] S21: Using pedestrian video features X as input, multiple parallel 3D convolutional branches are employed to extract multi-scale global spatiotemporal features. Each convolutional branch is configured with a different dilation rate. In this embodiment, the dilation rate is set to 1, 2, and 4, and the convolutional kernel size is (1, 3, 3). Each group of convolutional branches is concatenated with grouped convolution, batch normalization (BatchNorm3d), and the ReLU activation function. The last group of branches adds an additional 1×1×1 convolutional layer to further refine the features. After processing by multiple groups of convolutional branches, multiple global feature maps with different receptive fields are obtained. The calculation formula is as follows:

[0061] ,

[0062] in, The inflation rate of group k is indicated by the following expression: 3D convolution operation; This is the global feature map output by the k-th convolutional branch. The three branches correspond to expansion rates of 1, 2, and 4.

[0063] S22: For each location p ( N represents the total number of body parts. In this embodiment, N=7, including the background, head, torso, upper arm, lower arm, thigh, and calf. First, the mask for each body part is obtained. The mask is multiplied element-wise with the global feature maps in each receptive field in the spatial dimension to extract the local features of the region in different receptive fields. Then, the mean of all local features of the same region is taken and fused to obtain the multi-scale fused features of that region. The calculation formula is as follows:

[0064] ,

[0065] in, This represents the local features of region p within the k-th group of convolutional receptive fields. This represents element-wise multiplication; This represents the multi-scale fusion feature of region p.

[0066] S23: Configure an independent part-specific channel attention branch for each part p. This branch first performs adaptive global spatiotemporal pooling (AdaptiveAvgPool3d) on the pedestrian video feature X; then it sequentially passes through dimensionality reduction convolution, ReLU activation, dimensionality increase convolution, and Sigmoid activation, outputting the part-specific channel attention weight A.p The calculation formula is as follows:

[0067] ,

[0068] in, This represents the global spatiotemporal pooling result for the attention branch at location p; This represents a dimension reduction convolution operation; This represents a higher-dimensional convolution operation; The dedicated channel attention weight for part p.

[0069] S24: Multi-scale fusion features of part p Its dedicated channel attention weight By performing element-wise multiplication along the channel dimension, the feature channels of the target region are weighted and enhanced, while the response of the background region is suppressed, thus obtaining the local features of region p. The calculation formula is as follows:

[0070] ,

[0071] in, The local features of region p after channel attention weighting are represented.

[0072] S25: Local features of all parts The features are stitched together along the channel dimension; then, a 3D convolution operation is used to fuse the stitched features across channels, ultimately outputting multi-scale features for part perception. The calculation formula is as follows:

[0073] ,

[0074] in, This indicates a channel-level concatenation operation; The result of stitching together local features from all parts; This indicates that fusion is performed using 3D convolution operations; Multi-scale features are perceived for the final output location.

[0075] This invention provides a preferred embodiment for executing S3. This embodiment aims to model the parts of each frame in a video sequence as graph nodes, and to form a heterogeneous spatiotemporal part graph that can characterize the multi-dimensional relationships of human body parts by constructing three types of heterogeneous edges: spatial edges, temporal edges, and non-local edges, thus providing a structured graph topology for subsequent graph attention feature interactions.

[0076] S31: Construct a static adjacency matrix corresponding to spatial edges based on human anatomical topology, connecting only site nodes with anatomical connections within the same frame. First, define the anatomical connections between human body parts (e.g., head-torso, torso-upper arm, torso-thigh, etc.), and initialize a static anatomical adjacency matrix of size N×N. (N is the total number of parts, N=7 in this embodiment), assigning a value of 1 to part nodes with anatomical connections and a value of 0 to nodes without connections; then introducing a learnable spatial residual matrix. After activation and scaling by Sigmoid, the dynamic spatial adjacency matrix is ​​obtained by fusing it with the static anatomical adjacency matrix. Finally, this is extended to the entire sequence node dimension to construct a spatial edge adjacency matrix containing nodes from all frame locations. The calculation formula is as follows:

[0077] ,

[0078] in, This is a static adjacency matrix; The residual matrix is ​​the learnable space. The residual matrix after activation; The adjacency matrix is ​​used for dynamic dissection; t is the time frame index (value range 0 to T-1, T is the time frame number); i and j are the part indices (value range 1 to N-1, 0 is the background index, which does not participate in the connection); It is the spatial adjacency matrix of the entire sequence, with dimensions Y×Y (Y=T×N). express The transpose of the matrix; express The transpose of .

[0079] S32: Construct a temporal edge adjacency matrix based on cross-frame motion information, connecting only nodes of the same part in different frames, and adaptively determining the maximum number of time jumps. First, calculate the cross-frame motion velocity norm of each part node. (t is the time frame, ranging from 0 to T-2; n is the location index, ranging from 1 to N-1, 0 is the background index, which is not included in the connection), based on the average velocity norm of the most recent frame. Adaptively determine the maximum number of time jumps for this part in frame t. Subsequently, within a range not exceeding the maximum number of time hops, bidirectional time connections are established for nodes at the same location to obtain the time edge adjacency matrix. The calculation formula is as follows:

[0080] ,

[0081] ,

[0082] in, The node features of the nth part in the t-th frame; The velocity norm for a single frame; Represents the L2 norm; The average velocity norm of the most recent frames; To adapt to the maximum number of time hops; Indicates the number of time jumps.

[0083] S33: Construct a nonlocal edge adjacency matrix based on cross-frame and cross-part motion pattern similarity, excluding connections that are duplicated with spatial and temporal edges, establishing only cross-frame and cross-part associations, and adaptively sparsifying the matrix. First, for all node features... L2 normalization is performed, and the cosine similarity matrix S between nodes is calculated. Then, connections to background nodes are excluded, and highly relevant connections are adaptively determined based on node feature entropy. For each node, the top k highly similar connections are retained, and a sparse similarity matrix is ​​constructed. Finally, Softmax normalization is performed to obtain the nonlocal edge adjacency matrix. The calculation formula is as follows:

[0084] ,

[0085] in, Here, S represents the normalized node features, and S is the cosine similarity matrix. It is a sparse similarity matrix. It is a nonlocal edge adjacency matrix.

[0086] Finally, the spatial edge adjacency matrix Temporal adjacency matrix Nonlocal edge adjacency matrix Together, they form a heterogeneous spatiotemporal part graph containing spatial, temporal, and nonlocal relationships, whose topological structure can be represented as:

[0087] ,

[0088] in, For a heterogeneous spatiotemporal region graph, , , These are the sets of spatial edges, temporal edges, and nonlocal edges, respectively.

[0089] This invention provides a preferred embodiment for executing S4. This embodiment aims to integrate node features into part category encoding, temporal position encoding, and cross-frame motion information based on the constructed heterogeneous spatiotemporal part graph. Through the graph attention part association modeling module, multi-head graph attention message passing is performed on spatial edges, temporal edges, and non-local edges to achieve deep spatiotemporal association enhancement of node features.

[0090] S41: A set of nodes in a heterogeneous spatiotemporal partial graph As input, a unique part category code and time location code are generated for each node. These two codes are then layered element-wise onto the original node features to explicitly encode the node's part identity and time location. First, a part index is generated. With time index The part index Time index corresponding to the body part to which the node belongs The corresponding time frame of the node; the part category code is generated through the embedding layer; the time code is generated jointly through the embedding layer and the linear layer. Its calculation formula is:

[0091] ,

[0092] in, Indicates the embedded layer for part categories; Encode the generated part category; Represents the time embedding layer; Indicates a linear projection layer; Encode the generated time location; These are the node features after superposition and encoding; This indicates an element-wise addition operation.

[0093] S42: Based on node features after superposition coding First, reshape it into a spacetime structure. Subsequently, velocity and acceleration information at the feature level are calculated separately. These two types of motion information are then projected onto the node feature dimension via a linear layer and weighted and superimposed. Velocity information is calculated from the difference in node features at the same location in adjacent frames, and acceleration information is calculated from the difference in velocity information between adjacent frames. The calculation formula is as follows:

[0094] ,

[0095] in, The original velocity information is given by time frame index t and human body part index n; The original acceleration information is given by time frame index t and human body part index n. and Linear projection layers representing velocity and acceleration information respectively; and These are the projected velocity and acceleration characteristics, respectively. and These are the weighting coefficients for velocity information and acceleration information, respectively. These are the node features after fusing motion information.

[0096] S43: Employs a multi-head graph attention mechanism, setting the number of attention heads h and the spatial edge adjacency matrix. Temporal adjacency matrix Nonlocal edge adjacency matrix Attention weights and adjacent node features are calculated for the node connections corresponding to the three types of edges, respectively, to obtain independently updated node features for the three types of relationships. The calculation process is as follows:

[0097] ,

[0098] ,

[0099] in, These are the projection weight matrices for vectors Q, K, and V, respectively. This indicates a feature dimension reshaping operation; These are the reshaped multi-head Q, K, and V vectors, respectively. For multi-head bias parameters; The original attention score; It is a single-head feature dimension; The adjacency matrix mask corresponds to the following: and ; The attention score after masking; The processed attention weights; After aggregation feature; This represents the reverse reshaping operation of merging multiple dimensions; To output the projection matrix; The updated node features are for a single-sided type, where type can be spatial, temporal, or nonlocal, and the corresponding output is... The three types of relationships update node characteristics.

[0100] This invention provides a preferred embodiment for executing S5. This embodiment aims to achieve feature weighting by adaptively generating fusion weights based on the updated node features according to spatial, temporal, and nonlocal relationships through a node gating mechanism, and further enhance the expression of key part features by combining the node-by-node weights output by the part importance prediction module.

[0101] S51: The node features updated for the three types of relationships are used to construct a node gating mechanism through a multilayer perceptron. This generates fusion weights for spatial edges, temporal edges, and non-local edges for each node. These weights are then used to adaptively weight and sum the updated node features under the three types of relationships, achieving dynamic fusion of features from different relationships. The node-gated multilayer perceptron uses node features that fuse encoded and motion information. As input, the data passes through a dimensionality reduction linear layer, ReLU activation, a dimensionality increase linear layer, and Softmax normalization, outputting fusion weights that match the number of relation types. The weights are then multiplied element-wise by the updated features of their corresponding relations and summed to obtain the fused node features. The calculation formula is as follows:

[0102] ,

[0103] in, Indicates a linear layer of dimensionality reduction; This represents the linear layer of increased dimension; G is the fusion weight matrix generated by node gating. These are the weight components corresponding to spatial, temporal, and nonlocal relations, respectively. These are the node features updated for the three types of relationships; The fused node features are the fused features of the three types of relationship features.

[0104] S52: Based on fused node features, a part importance prediction module is constructed using a multilayer perceptron. A unique importance weight is generated for each node, and this weight is multiplied element-wise with the fused node features to enhance the features of key parts. The part importance multilayer perceptron uses fused node features... As input, the data sequentially passes through a dimensionality-reducing linear layer, ReLU activation, a dimensionality-increasing linear layer, and a Sigmoid activation, outputting node-by-node importance weights. These weights are then multiplied node-by-node by the fused features, combined with residual connections and normalization operations to obtain the final enhanced node features. The calculation formula is as follows:

[0105] ,

[0106] in, Indicates a linear layer of dimensionality reduction; Indicates a linear layer of increased dimensionality; This is a weight matrix representing the importance of different locations. These are the enhanced node features after importance weighting; This is the enhanced node feature for the final output.

[0107] This invention provides a preferred embodiment for executing S6. This embodiment aims to map the enhanced node features from the node dimension back to the original spatial dimension, eliminate the difference in part mask area through normalization, and fuse them with part-aware multi-scale features in the form of residuals, ultimately obtaining spatiotemporal structure-aware part features that combine node association enhancement and spatial structure perception.

[0108] S61: The enhanced node features are reshaped into a four-dimensional structure of batch, frame number, number of parts, and feature dimension. Background nodes are removed, and human body part node features are retained. The part mask is then adjusted to a matching spatiotemporal part format. A spatial weighted accumulation of node features and mask is achieved through Einstein summation, mapping back to the original spatial dimension to output the enhanced part structure features. The calculation formula is as follows:

[0109] ,

[0110] in, To exclude background nodes and define the features of human body parts; Masking of human body parts after removing the background; Enhanced feature map of the part structure.

[0111] S62: Calculate the number of pixels in the mask for each part in the spatial dimension to obtain the mask area, and then sum the minimum values. (This embodiment takes) Using as the denominator, the spatial location of the enhanced structural features is normalized. The calculation formula is:

[0112] ,

[0113] ,

[0114] ,

[0115] in, The spatial area of ​​a single part of the mask; This represents the sum of the mask areas at each location; This is an enhanced feature of the normalized part structure.

[0116] S63: The normalized region-based structural enhancement feature map is subjected to interactive projection and nonlinear activation of feature channels through 3D convolution. Then, the feature weights are adjusted using learnable parameters, and finally fused with the region-aware multi-scale feature residuals. The calculation formula is as follows:

[0117] ,

[0118] ,

[0119] ,

[0120] in, This is the enhanced feature map after projection; Scaling parameters for learnable interactions; This is the scaled-up enhanced feature map; The spatiotemporal structure sensing features are the final output.

[0121] This invention provides a preferred embodiment for executing S7. This embodiment aims to integrate fine-grained part features of spatiotemporal structure perception part features, coarse-grained global features, and graph-level spatiotemporal relationship features of heterogeneous spatiotemporal part maps through a multi-granularity feature fusion module to generate a clothing change-robust identity representation vector, and generate a pedestrian matching list through cosine similarity calculation and sorting.

[0122] S71: Fine-grained part features are six types of human body part features after excluding the background, obtained by global spatiotemporal pooling of spatiotemporal structure-aware part features within the masked region of each part; coarse-grained global features are the global spatiotemporal pooling results of spatiotemporal structure-aware part features; graph-level spatiotemporal relationship features are the mean pooling results of graph node features. The calculation formula is:

[0123] ,

[0124] in, This represents the fine-grained feature of the nth region; It is the set of features for all fine-grained parts; It is a coarse-grained global feature; This is a mean pooling operation along the node dimension; This represents the spatiotemporal relationship characteristics at the graph level.

[0125] S72: Input the three types of features into a multilayer perceptron to generate adaptive fusion weights; simultaneously, introduce learnable global fusion weights and fuse them proportionally to obtain the final fusion weights. The calculation formula is as follows:

[0126] ,

[0127] in, It is a multilayer perceptron, which includes a dimension-reduced linear layer, ReLU activation, and a dimension-upgrading linear layer; For adaptive fusion weights; These are learnable global weight parameters; The normalized global fusion weights; 0.7 represents the final fusion weight; 0.7 and 0.3 represent the fusion ratio between the adaptive weight and the global weight.

[0128] S73: The final fusion weights are used to weight and sum the three types of features to obtain the fused feature. After linear layer, layer normalization, nonlinear activation, and feature normalization, a clothing-variable-robust identity representation vector is output. Finally, the cosine similarity between this vector and the features in the image library is calculated, and a pedestrian matching list is generated by sorting the similarity scores. The calculation formula is as follows:

[0129] ,

[0130] in, This is a multi-granularity fusion feature; This is an identity representation vector; To query the sample identity vector; This is the identity vector of the image library samples; This is the cosine similarity matrix; This is an index operation that retrieves the top r-largest similarity values; Rank is the list of matched pedestrians.

[0131] like Figure 3 The image shown is an example of a pedestrian matching list generated by a pedestrian re-identification method based on graph attention and dynamic association modeling of human body parts, provided by this invention. Column 1 represents the original query image; columns 2 through 11 display the search results, sorted by similarity from highest to lowest, showing the top 10 most relevant candidate matches. This indicates a successful retrieval, meaning the candidate image and the query image belong to the same pedestrian. This indicates a retrieval error, meaning that images of different pedestrians were mistakenly identified as having the same identity.

[0132] This embodiment conducts experimental validation on the publicly available and challenging VCCR video person re-identification dataset. This dataset supports model testing for both clothing-changing scenarios (CC) and standard scenarios, providing a comprehensive and rigorous benchmark for the overall performance of the method. Model performance is evaluated using two common core metrics in the field of person re-identification: mean average accuracy (mAP) and Rank 1 accuracy in the cumulative matching feature curve.

[0133] Table 1 compares the experimental results of the proposed method and the comparison method on the VCCR dataset.

[0134]

[0135] As shown in Table 1, the proposed method for re-identifying pedestrians in changing clothes based on graph attention-based dynamic association modeling of human body parts in this embodiment significantly outperforms existing comparative methods in both the mAP and Rank-1 metrics in the changing clothes scene (CC) and standard scene (Standard) of the VCCR dataset.

[0136] Specifically, in the more challenging clothing-changing scenario, the proposed method achieves an mAP of 45.7% and a Rank-1 score of 53.5%, representing improvements of 2.5 and 0.6 percentage points respectively compared to the second-best method, ASGL (mAP 43.2%, Rank-1 52.9%). In the standard scenario, the proposed method achieves mAP and Rank-1 scores of 67.3% and 88.6% respectively, also surpassing the best performance among the compared methods (ASGL: mAP 65.8%, Rank-1 88.1%). These results fully validate the effectiveness and superiority of the proposed method in handling the task of re-identifying pedestrians changing clothes.

Claims

1. A method for re-identifying pedestrians changing clothes based on graph attention-based dynamic association modeling of human body parts, characterized in that, Includes the following steps: S1: Obtain the video sequence of the pedestrian to be identified, and obtain the part mask by segmentation through the human body analysis model; S2: Based on the part mask, the part-aware multi-scale feature extraction module is used to perform part-aware multi-scale modeling of the input video features. The part-aware multi-scale feature extraction module constructs multiple receptive fields through spatiotemporal convolution kernels with different dilation rates to extract multi-scale features, and combines the part channel attention branch to generate part-specific channel weights, weighted and enhanced the feature channels of the corresponding parts and suppressed the response of the background region, and outputs part-aware multi-scale features. S3: Construct each frame and corresponding part in the sequence as a graph node, establish spatial edges based on human anatomical topology, establish temporal edges based on cross-frame motion information, and establish non-local edges based on cross-frame and cross-part motion pattern similarity to form a heterogeneous spatiotemporal part graph containing spatial, temporal and non-local relationships. S4: Incorporate part category encoding, temporal position encoding, and cross-frame motion information into node features, and perform multi-head graph attention message passing on spatial edges, temporal edges, and non-local edges through the graph attention part association modeling module; S4 includes: S41: The part category code is generated through the embedding layer. The input of the embedding layer is the part index, and the output dimension matches the node feature dimension. The time code is generated through the embedding layer and the linear layer. The input of the embedding layer is the frame index, and the linear layer projects the output of the embedding layer onto the node feature dimension. The part category code and the time code are superimposed on the node features to achieve explicit encoding of the node identity features and time position. S42: Velocity information is calculated from the feature difference of the same part in adjacent frames, and acceleration information is calculated from the velocity difference in adjacent frames. The velocity information and acceleration information are projected onto the node features through a linear layer respectively. S43: The graph attention network employs a multi-head attention mechanism, where the number of heads is set according to preset hyperparameters. Node features are projected as query, key, and value vectors. When calculating the attention score, a multi-head bias obtained from relation embedding is introduced, where relation embedding is used to distinguish different relation types such as spatial edges, temporal edges, and non-local edges. An adjacency matrix mask is applied to the attention score, and after Softmax normalization and Dropout processing, the value vector features of adjacent nodes are weighted and summed. The weighted aggregation result is projected onto the output, and the updated node features are obtained through a non-linear activation function. S5: Generate the fusion weight of each node feature in the three types of relationships through the node gating mechanism, and perform weighted combination of the updated node features under the three types of relationships; then multiply the node-by-node importance weights output by the part importance prediction module to obtain the enhanced node features. S6: Map the enhanced node features back to the original spatial dimension, and after normalization, inject them into the aforementioned part-aware multi-scale features in the form of residuals to obtain spatiotemporal structure-aware part features. S7: Integrate the fine-grained part features and coarse-grained global features in the spatiotemporal structure perception part features with the graph-level spatiotemporal relationship features in the heterogeneous spatiotemporal part map through the multi-granularity feature fusion module to generate a clothing change robust identity representation vector; calculate the cosine similarity between this vector and the map library features, and generate a pedestrian matching list by sorting the similarity.

2. The method for re-identifying pedestrians changing clothes based on graph attention-based dynamic association modeling of human body parts as described in claim 1, characterized in that, The location-aware multi-scale feature extraction module in S2 includes: S21: Multiple sets of 3D convolutional branches are used, each set of convolutional branches uses spatiotemporal convolutional kernels with different dilation rates to perform convolution operations on the input video feature map to construct multiple receptive fields and extract multi-scale spatiotemporal features; each set of convolutional branches is connected in series with group convolution, batch normalization and activation function; after parallel processing by multiple sets of convolutional branches, multiple global feature maps with different receptive fields are obtained. S22: For each part, multiply the corresponding part mask with the global feature map of each different receptive field in the spatial dimension to extract the local features of the part under different receptive fields; and fuse all the local features of the same part under different receptive fields to obtain the multi-scale fused features of each part. S23: Configure a local channel attention branch for each part. Each local channel attention branch is processed by adaptive global spatiotemporal pooling and then passes through a channel mapping layer of dimensionality reduction and dimensionality increase and nonlinear activation in sequence to output the local channel attention weight. S24: Multiply the multi-scale fusion features of each part with their corresponding dedicated channel attention weights to achieve weighted enhancement of the feature channels of the corresponding parts and suppress the response of the background region, thereby obtaining the local feature representation of each part; S25: Represent the local features of all parts on the channel dimension and then stitch and fuse them to obtain part-aware multi-scale features.

3. The method for re-identifying pedestrians changing clothes based on graph attention-based dynamic association modeling of human body parts as described in claim 1, characterized in that, The construction of the heterogeneous spatiotemporal region map in S3 includes: S31: Spatial edges are initialized with a static adjacency matrix based on human anatomical topology, connecting only nodes in the same frame that have anatomical connections. S32: The time edge only connects nodes of the same part in different frames. The maximum number of time jumps is adaptively determined based on the cross-frame motion information of the part. Bidirectional time connections are established within the range of not exceeding the maximum number of jumps. S33: Non-local edges determine the connection relationship by combining the similarity of motion patterns of each part, and non-local edges exclude connections that are repeated with spatial and temporal edges, focusing on establishing cross-frame and cross-part associations, and retaining highly correlated connections through adaptive sparsification.

4. The method for re-identifying pedestrians changing clothes based on graph attention-based dynamic association modeling of human body parts as described in claim 1, characterized in that, S5 includes: S51: A node gating mechanism is used to generate fusion weights for each node on three types of relationships: spatial edges, temporal edges, and nonlocal edges. The node gating mechanism is implemented through a multilayer perceptron, which sequentially connects a linear layer, an activation function, a linear layer, and a normalized activation function. The fusion weights are multiplied by the updated node features under the three types of relationships and summed to achieve adaptive weighted fusion of node features under the three types of relationships, resulting in fused node features. S52: An importance weight is generated for each node through the location importance prediction module; the location importance prediction module is implemented through a multilayer perceptron, which is a multilayer perceptron that sequentially connects a linear layer, an activation function, and another linear layer and an activation function; the importance weight is multiplied node by node by node with the fused node features to obtain enhanced node features.

5. The method for re-identifying pedestrians changing clothes based on graph attention-based dynamic association modeling of human body parts as described in claim 1, characterized in that, S6 includes: S61: Reshape the enhanced node features into a format of batch, frame number, number of parts, and feature dimension; exclude background nodes and retain only human body part nodes to obtain non-background node features and non-background part masks; use Einstein summation to weight and accumulate the non-background node features according to the corresponding non-background part masks in the spatial dimension to obtain a part structure enhancement feature map with the same shape as the part-aware multi-scale features output by S2. S62: Calculate the spatial area of ​​the mask for each part and superimpose a small constant as a normalization factor to normalize the part structure enhancement feature map, prevent division by zero and cancel the difference in mask area of ​​different parts, and obtain the normalized part structure enhancement feature map. S63: Perform a learnable interactive projection transformation on the normalized part structure enhancement feature map; introduce a learnable interactive scaling parameter, and add the scaled projected features to the part-aware multi-scale features output by S2 in the form of residuals to obtain the spatiotemporal structure-aware part features.

6. The method for re-identifying pedestrians changing clothes based on graph attention-based dynamic association modeling of human body parts as described in claim 1, characterized in that, The multi-granularity feature fusion module in S7 includes: S71: Fine-grained part features are six types of human body part features after excluding the background, obtained by global spatiotemporal pooling of spatiotemporal structure-aware part features within the masked region of each part; coarse-grained global features are the global spatiotemporal pooling results of spatiotemporal structure-aware part features; graph-level spatiotemporal relationship features are the mean pooling results of graph node features. S72: Input the three types of features described in S71 into the multilayer perceptron to generate adaptive fusion weights; at the same time, introduce learnable global fusion weights and calculate the final fusion weights proportionally. S73: The three types of features are weighted and summed using the final fusion weights to obtain the fusion features; the fusion features are then passed through a linear layer, layer normalization, and nonlinear activation, and then feature normalization is performed to output the identity representation vector.

7. The method for re-identifying pedestrians changing clothes based on graph attention-based dynamic association modeling of human body parts as described in claim 1, characterized in that, It also includes a multi-layer stacking step for the multi-head graph attention message passing process of the graph attention part association modeling module described in S4 and the node feature weighted fusion and enhancement process described in S5: the multi-head graph attention message passing and node feature weighted fusion and enhancement processes are stacked in multiple layers, and the node features output by each layer together with the initial node features constitute a feature set; The feature set is weighted and summed by learnable layer weights, which are then normalized to improve the depth and robustness of the dynamic correlation modeling of parts.

Images