Dressing pedestrian re-identification method based on graph attention human body part dynamic correlation modeling

Abstract

The invention discloses a clothes changing pedestrian re-identification method based on graph attention human body part dynamic correlation modeling. The method comprises the following steps: acquiring a video sequence of a pedestrian to be identified, and performing part segmentation through a human body analysis model; a part perception multi-scale feature extraction module is used to extract the features of each part under different receptive fields and suppress the background; then, the combination of each frame and the part is constructed into a graph node, and a heterogeneous space-time part graph containing a space edge, a time edge and a non-local edge is constructed; performing message passing on the three types of edges through a graph attention part association modeling module, and fusing to obtain enhanced node features; then, the enhanced node features are mapped back to original spatial feature dimensions; carrying out adaptive weighted integration on different granularity features; and finally, an identity representation vector is generated, similarity between the identity representation vector and image library features is calculated, and a matching result is output. According to the method, by capturing time sequence dynamic association between human body parts, identity representation generated by the model is more focused on a stable human body structure instead of a variable clothing appearance.

Description

A method for re-identifying pedestrians changing clothes based on graph attention-based dynamic association modeling of human body parts Technical Field

[0001] This invention relates to a method for re-identifying pedestrians changing clothes based on graph attention-based dynamic association modeling of human body parts, belonging to the field of computer vision technology. Background Technology

[0002] Pedestrian re-identification is an important research direction in the fields of computer vision and intelligent video analysis. Its goal is to achieve cross-time and cross-viewpoint identity matching and retrieval of the same pedestrian target under conditions of multiple cameras and non-overlapping fields of view, which has significant value in applications such as public safety, intelligent transportation, and urban management. With the development of deep learning technology, pedestrian re-identification methods based on appearance feature learning have made some progress under controlled conditions.

[0003] However, in practical applications, pedestrians often change their clothing at different times, resulting in significant changes in the appearance, color, texture, and local details of the same pedestrian. This makes it difficult to maintain consistency in feature representations that rely on clothing appearance, leading to a significant decline in pedestrian re-identification performance in clothing-changing scenarios. How to extract stable and discriminative identity features under clothing change conditions has become one of the important challenges facing current pedestrian re-identification research.

[0004] To mitigate the impact of clothing changes, some existing technologies have begun to incorporate methods such as human body analysis, keypoint detection, or body part segmentation to perform feature modeling at the human body structure level, thereby enhancing the ability to describe identity-related and relatively stable regions. However, these methods mostly focus on single-frame images or static spatial structure modeling, and do not adequately utilize the temporal evolution and cooperative motion relationships of various structural parts of the human body in video sequences.

[0005] In video-based pedestrian re-identification scenarios involving clothing changes, human body structures not only possess relatively stable spatial anatomical relationships but also exhibit individualized temporal collaborative patterns during movements such as walking. Different body parts differ in terms of movement amplitude, rhythm, and correlation strength, and dynamic correlations may also arise between parts across time and body parts. This dynamic correlation information between human body structures is more stable than clothing appearance and is of significant value for identity verification, but it has not yet been fully explored and effectively modeled in current technologies. Summary of the Invention

[0006] To address the aforementioned issues, this invention proposes a method for re-identifying pedestrians changing clothes based on graph attention-based dynamic association modeling of human body parts. The method includes the following steps: S1: Acquire a video sequence of the pedestrian to be identified, and obtain a part mask through human body analysis model segmentation; S2: Based on the part mask, use a part-aware multi-scale feature extraction module to perform part-aware multi-scale modeling of the input video features. This module constructs multiple receptive fields using spatiotemporal convolution kernels with different dilation rates to extract multi-scale features, and combines part-specific channel attention branches to generate part-specific channel weights. It then weights and enhances the feature channels of the corresponding parts while suppressing background region responses, outputting part-aware multi-scale features; S3: Construct graph nodes for each frame and corresponding part in the sequence, establishing spatial edges based on human anatomical topology, temporal edges based on cross-frame motion information, and non-local edges based on cross-frame and cross-part motion pattern similarity, thus forming a heterogeneous spatiotemporal part graph containing spatial, temporal, and non-local relationships; S4: In The node features incorporate part category encoding, temporal location encoding, and cross-frame motion information. A graph attention part association modeling module performs multi-head graph attention message passing on spatial edges, temporal edges, and non-local edges. S5: A node gating mechanism generates fusion weights for each node feature across three types of relationships, and the updated node features under these relationships are weighted and combined. These weights are then multiplied node-by-node by the node importance weights output by the part importance prediction module to obtain enhanced node features. S6: The enhanced node features are mapped back to the original spatial dimension, normalized, and then injected as residuals into the aforementioned part-aware multi-scale features to obtain spatiotemporal structure-aware part features. S7: A multi-granularity feature fusion module integrates fine-grained part features, coarse-grained global features, and graph-level spatiotemporal relationship features from the spatiotemporal structure-aware part features with those from the heterogeneous spatiotemporal part graph to generate a robust identity representation vector for clothing changes. The cosine similarity between this vector and the library features is calculated, and a pedestrian matching list is generated by sorting the similarity.

[0007] Furthermore, the part-aware multi-scale feature extraction module in S2 includes: S21: employing multiple sets of three-dimensional convolutional branches, each set using spatiotemporal convolutional kernels with different dilation rates, to perform convolution operations on the input video feature map to construct multiple receptive fields and extract multi-scale spatiotemporal features; each set of convolutional branches is concatenated with group convolution, batch normalization, and activation functions; after parallel processing by multiple sets of convolutional branches, multiple global feature maps of different receptive fields are obtained; S22: for each part, the corresponding part mask is multiplied with the global feature maps of each different receptive field in the spatial dimension to extract the local features of the part under different receptive fields; and all local features of the same part under different receptive fields are extracted. The features are fused to obtain multi-scale fused features for each part; S23: Configure part-specific channel attention branches for each part. Each part-specific channel attention branch is processed by adaptive global spatiotemporal pooling and then sequentially passed through channel mapping layers of dimensionality reduction and dimensionality increase and nonlinear activation to output the exclusive channel attention weight for that part; S24: Multiply the multi-scale fused features of each part with their corresponding exclusive channel attention weights to achieve weighted enhancement of the feature channels of the corresponding part and suppress the response of the background region, thereby obtaining the local feature representation of each part; S25: Concatenate and fuse the local feature representations of all parts in the channel dimension to obtain part-aware multi-scale features.

[0008] Furthermore, the construction of the heterogeneous spatiotemporal part graph in S3 includes: S31: Spatial edges initialize a static adjacency matrix based on human anatomical topology, connecting only part nodes with anatomical connections within the same frame; S32: Temporal edges connect only the same part nodes in different frames, adaptively determining the maximum number of time jumps based on the cross-frame motion information of the part, and establishing bidirectional temporal connections within the range not exceeding the maximum number of jumps; S33: Non-local edges determine the connection relationship by combining the similarity of motion patterns of each part, and non-local edges exclude connections that are repeated with spatial and temporal edges, focusing on establishing cross-frame and cross-part associations, and retaining highly correlated connections after adaptive sparsification.

[0009] Furthermore, S4 includes: S41: The part category code is generated through an embedding layer, where the input to the embedding layer is the part index, and the output dimension matches the node feature dimension; the time code is generated through an embedding layer and a linear layer, where the input to the embedding layer is the frame index, and the linear layer projects the output of the embedding layer onto the node feature dimension; the part category code and the time code are superimposed on the node features to achieve explicit encoding of the node identity features and time position; S42: Velocity information is calculated from the difference in features of the same part in adjacent frames, and acceleration information is calculated from the velocity difference in adjacent frames; the velocity information and acceleration information are projected through a linear layer and then superimposed on the node features. S43: The graph attention network adopts a multi-head attention mechanism, the number of heads of which is set according to a preset hyperparameter; the node features are projected into query, key, and value vector representations, and a multi-head bias obtained by relation embedding is introduced when calculating the attention score. The relation embedding is used to distinguish different relation types such as spatial edges, temporal edges, and non-local edges; an adjacency matrix mask is applied to the attention score, and after Softmax normalization and Dropout processing, the value vector features of adjacent nodes are weighted and summed; the weighted aggregation result is projected onto the output, and the updated node features are obtained through a nonlinear activation function.

[0010] Furthermore, S5 includes: S51: Generating fusion weights for each node on three types of relationships—spatial edge, temporal edge, and nonlocal edge—through a node gating mechanism; the node gating mechanism is implemented through a multilayer perceptron, which is sequentially connected with a linear layer, an activation function, a linear layer, and a normalized activation function; the fusion weights are multiplied by the updated node features under the three types of relationships and summed to achieve adaptive weighted fusion of node features under the three types of relationships, resulting in fused node features; S52: Generating importance weights for each node through a location importance prediction module; the location importance prediction module is implemented through a multilayer perceptron, which is sequentially connected with a linear layer, an activation function, a linear layer, and an activation function; the importance weights are multiplied by the fused node features node by node to obtain enhanced node features.

[0011] Further, S6 includes: S61: reshaping the enhanced node features into a format of batch, frame number, part number, and feature dimension; excluding background nodes and retaining only human body part nodes to obtain non-background node features and non-background part masks; using Einstein summation to weight and accumulate the non-background node features according to the corresponding non-background part masks in the spatial dimension to obtain a part structure enhancement feature map with the same shape as the part-aware multi-scale features output by S2; S62: calculating the spatial area of ​​each part mask and superimposing a small constant as a normalization factor to normalize the part structure enhancement feature map, preventing division by zero and canceling the differences in the area of ​​different part masks, to obtain a normalized part structure enhancement feature map; S63: performing a learnable interactive projection transformation on the normalized part structure enhancement feature map; introducing a learnable interactive scaling parameter, scaling the projected features and adding them to the part-aware multi-scale features output by S2 in the form of residuals to obtain spatiotemporal structure-aware part features.

[0012] Furthermore, the multi-granularity feature fusion module in S7 includes: S71: fine-grained part features are six types of human body part features after excluding the background, obtained by global spatiotemporal pooling of spatiotemporal structure-aware part features within the masked regions of each part; coarse-grained global features are the global spatiotemporal pooling results of spatiotemporal structure-aware part features; graph-level spatiotemporal relationship features are the mean pooling results of graph node features; S72: the three types of features mentioned in S71 are input into a multilayer perceptron to generate adaptive fusion weights; at the same time, learnable global fusion weights are introduced, and the final fusion weights are calculated proportionally; S73: the three types of features are weighted and summed using the final fusion weights to obtain fused features; the fused features are sequentially passed through a linear layer, layer normalization, and nonlinear activation, and then feature normalization is performed to output an identity representation vector.

[0013] Furthermore, the method for re-identifying pedestrians changing clothes based on graph attention-based dynamic association modeling of human body parts also includes a multi-layer stacking step of the multi-head graph attention message passing process of the graph attention part association modeling module in S4 and the node feature weighted fusion and enhancement process in S5: the multi-head graph attention message passing and node feature weighted fusion and enhancement processes are stacked in multiple layers, and the node features output by each layer together with the initial node features constitute a feature set; the feature set is weighted and summed by learnable layer weights, and the layer weights are normalized to improve the depth and robustness of the part dynamic association modeling.

[0014] Compared with existing technologies, this invention has the following beneficial effects: First, by introducing human body analysis results to semantically segment pedestrian video sequences and performing feature modeling at the part level, this invention explicitly utilizes human body structural information, effectively reducing the interference of clothing appearance changes on identity representation and improving the robustness of pedestrian re-identification in clothing-changing scenarios; Second, this invention constructs a heterogeneous spatiotemporal human body part map that includes spatial, temporal, and non-local relationships, and uniformly models the motion associations and collaborative relationships of human body structural parts between different frames. This fully explores the dynamic association information between human body structural parts that changes over time, thereby enhancing the robustness of identity-related stable... Third, this invention utilizes graph attention-based part association modeling and node feature adaptive fusion mechanism to dynamically model and weight the relationships between different types of parts and the importance of different parts. This enables the model to highlight key parts and key relationships based on specific spatiotemporal contexts, suppress redundant or unstable information, and further improve the discriminativeness and stability of identity representation. Fourth, experimental results on the publicly available video person re-identification dataset VCCR show that this invention has achieved significant improvements in mAP and Rank-1 key evaluation metrics, fully verifying the effectiveness and advancement of the proposed method in the person re-identification task. Attached Figure Description

[0015] To make the technical solution and beneficial effects of the present invention clearer, the specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the accompanying drawings are only for illustrating the principle structure or method flow examples of the present invention, and are not intended to limit the scope of protection of the present invention. Those skilled in the art can make various modifications or substitutions to the shown structures or steps without departing from the spirit and substance of the present invention, and all such modifications or substitutions should fall within the scope of protection of the present invention.

[0016] Figure 1 is a flowchart of the steps of a pedestrian re-identification method based on graph attention and dynamic association modeling of human body parts provided by the present invention; Figure 2 is an overall framework diagram of a pedestrian re-identification method based on graph attention and dynamic association modeling of human body parts provided by the present invention; Figure 3 is an example diagram of the pedestrian matching list generated by a pedestrian re-identification method based on graph attention and dynamic association modeling of human body parts provided by the present invention. Detailed Implementation

[0017] To enable those skilled in the art to more intuitively and comprehensively understand the technical solution and implementation process of the present invention, the present invention will be further described below in conjunction with specific embodiments. It should be understood that the embodiments described below are only used to illustrate the principles and technical details of the present invention and do not constitute a limitation on the scope of protection of the present invention. Various modifications or substitutions can be made by those skilled in the art without departing from the spirit and substance of the present invention, and all such modifications or substitutions should fall within the scope of protection of the present invention.

[0018] Figure 1 shows a flowchart of the steps of a method for re-identifying pedestrians changing clothes based on graph attention-based dynamic association modeling of human body parts provided by the present invention, including: S1: acquiring a video sequence of the pedestrian to be identified, and obtaining a part mask by segmentation using a human body analysis model; S2: based on the part mask, using a part-aware multi-scale feature extraction module to perform part-aware multi-scale modeling of the input video features. The part-aware multi-scale feature extraction module constructs multiple receptive fields through spatiotemporal convolution kernels with different dilation rates to extract multi-scale features, and generates part-specific channel weights by combining part-specific channel attention branches, weighting and enhancing the feature channels of the corresponding parts and suppressing background region responses, and outputting part-aware multi-scale features; S3: constructing each frame in the sequence and the corresponding part as graph nodes, establishing spatial edges based on human anatomical topology, establishing temporal edges based on cross-frame motion information, and establishing non-local edges based on cross-frame and cross-part motion pattern similarity, forming a heterogeneous spatiotemporal part graph containing spatial, temporal, and non-local relationships; S4: The node features are integrated with part category encoding, temporal location encoding, and cross-frame motion information. A graph attention part association modeling module performs multi-head graph attention message passing on spatial edges, temporal edges, and non-local edges. S5: A node gating mechanism generates fusion weights for each node feature across three types of relationships. The updated node features under these three relationships are then weighted and combined. The node importance weights output by the part importance prediction module are then multiplied node-by-node to obtain enhanced node features. S6: The enhanced node features are mapped back to the original spatial dimension, normalized, and then injected as residuals into the aforementioned multi-scale part-aware features to obtain spatiotemporal structure-aware part features. S7: A multi-granularity feature fusion module integrates fine-grained part features, coarse-grained global features, and graph-level spatiotemporal relationship features from the spatiotemporal structure-aware part features with those from the heterogeneous spatiotemporal part graph to generate a robust identity representation vector for clothing changes. The cosine similarity between this vector and the library features is calculated, and a pedestrian matching list is generated by sorting the similarities.

[0019] Figure 2 shows the overall framework of a method for re-identifying pedestrians changing clothes based on graph attention-based dynamic association modeling of human body parts, provided by this invention. It mainly includes a part-aware multi-scale feature extraction module, a graph attention part association modeling module, and a multi-granularity feature fusion module.

[0020] This invention provides a preferred embodiment for executing S2. This embodiment aims to extract multi-scale features by constructing multiple receptive fields based on part masks and using spatiotemporal convolution kernels with different dilation rates, and to enhance human structural part features and suppress background responses by combining part-specific channel attention mechanisms, ultimately outputting high-quality part-aware multi-scale features.

[0021] S21: Using pedestrian video features X as input, multiple parallel 3D convolutional branches are employed to extract multi-scale global spatiotemporal features. Each convolutional branch is configured with a different dilation rate. In this embodiment, the dilation rate is set to 1, 2, and 4, and the convolutional kernel size is (1, 3, 3). Each group of convolutional branches is concatenated with grouped convolution, batch normalization (BatchNorm3d), and the ReLU activation function. The last group of branches adds an additional 1×1×1 convolutional layer to further refine the features. After processing by multiple groups of convolutional branches, multiple global feature maps with different receptive fields are obtained. The calculation formula is as follows: ,in, The inflation rate of group k is indicated by the following expression: 3D convolution operation; This is the global feature map output by the k-th convolutional branch. The three branches correspond to expansion rates of 1, 2, and 4.

[0022] S22: For each location p ( N represents the total number of body parts. In this embodiment, N=7, including the background, head, torso, upper arm, lower arm, thigh, and calf. First, the mask for each body part is obtained. The mask is multiplied element-wise with the global feature maps in each receptive field in the spatial dimension to extract the local features of the region in different receptive fields. Then, the mean of all local features of the same region is taken and fused to obtain the multi-scale fused features of that region. The calculation formula is as follows: ,in, This represents the local features of region p within the k-th group of convolutional receptive fields. This represents element-wise multiplication; This represents the multi-scale fusion feature of region p.

[0023] S23: Configure an independent part-specific channel attention branch for each part p. This branch first performs adaptive global spatiotemporal pooling (AdaptiveAvgPool3d) on the pedestrian video feature X; then it sequentially passes through dimensionality reduction convolution, ReLU activation, dimensionality increase convolution, and Sigmoid activation, outputting the part-specific channel attention weight A. p The calculation formula is as follows: ,in, This represents the global spatiotemporal pooling result for the attention branch at location p; This represents a dimension reduction convolution operation; This represents a higher-dimensional convolution operation; The dedicated channel attention weight for part p.

[0024] S24: Multi-scale fusion features of part p Its dedicated channel attention weight By performing element-wise multiplication along the channel dimension, the feature channels of the target region are weighted and enhanced, while the response of the background region is suppressed, thus obtaining the local features of region p. The calculation formula is as follows: ,in, The local features of region p after channel attention weighting are represented.

[0025] S25: Local features of all parts The features are stitched together along the channel dimension; then, a 3D convolution operation is used to fuse the stitched features across channels, ultimately outputting multi-scale features for part perception. The calculation formula is as follows: ,in, This indicates a channel-level concatenation operation; The result of stitching together local features from all parts; This indicates that fusion is performed using 3D convolution operations; To perceive multi-scale features for the final output location.

[0026] This invention provides a preferred embodiment for executing S3. This embodiment aims to model the parts of each frame in a video sequence as graph nodes, and to form a heterogeneous spatiotemporal part graph that can characterize the multi-dimensional relationships of human body parts by constructing three types of heterogeneous edges: spatial edges, temporal edges, and non-local edges, thus providing a structured graph topology for subsequent graph attention feature interactions.

[0027] S31: Construct a static adjacency matrix corresponding to spatial edges based on human anatomical topology, connecting only site nodes with anatomical connections within the same frame. First, define the anatomical connections between human body parts (e.g., head-torso, torso-upper arm, torso-thigh, etc.), and initialize a static anatomical adjacency matrix of size N×N. (N is the total number of parts, N=7 in this embodiment), assigning a value of 1 to part nodes with anatomical connections and a value of 0 to nodes without connections; then introducing a learnable spatial residual matrix. After activation and scaling by Sigmoid, the dynamic spatial adjacency matrix is ​​obtained by fusing it with the static anatomical adjacency matrix. Finally, this is extended to the entire sequence node dimension to construct a spatial edge adjacency matrix containing nodes from all frame locations. The calculation formula is as follows: ,in, This is a static adjacency matrix; The residual matrix is ​​the learnable space. The residual matrix after activation; The adjacency matrix is ​​used for dynamic dissection; t is the time frame index (value range 0 to T-1, T is the time frame number); i and j are the part indices (value range 1 to N-1, 0 is the background index, which does not participate in the connection); It is the spatial adjacency matrix of the entire sequence, with dimensions Y×Y (Y=T×N). express The transpose of the matrix; express The transpose of .

[0028] S32: Construct a temporal edge adjacency matrix based on cross-frame motion information, connecting only nodes of the same part in different frames, and adaptively determining the maximum number of time jumps. First, calculate the cross-frame motion velocity norm of each part node. (t is the time frame, ranging from 0 to T-2; n is the location index, ranging from 1 to N-1, 0 is the background index, which is not included in the connection), based on the average velocity norm of the most recent frame. Adaptively determine the maximum number of time jumps for this part in frame t. Subsequently, within a range not exceeding the maximum number of time hops, bidirectional time connections are established for nodes at the same location to obtain the time edge adjacency matrix. The calculation formula is as follows: , ,in, The node features of the nth part in the t-th frame; The velocity norm for a single frame; Represents the L2 norm; The average velocity norm of the most recent frames; To adapt to the maximum number of time hops; Indicates the number of time jumps.

[0029] S33: Construct a nonlocal edge adjacency matrix based on cross-frame and cross-part motion pattern similarity, excluding connections that are duplicated with spatial and temporal edges, establishing only cross-frame and cross-part associations, and adaptively sparsifying the matrix. First, for all node features... L2 normalization is performed, and the cosine similarity matrix S between nodes is calculated. Then, connections to background nodes are excluded, and highly relevant connections are adaptively determined based on node feature entropy. For each node, the top k highly similar connections are retained, and a sparse similarity matrix is ​​constructed. Finally, Softmax normalization is performed to obtain the nonlocal edge adjacency matrix. The calculation formula is as follows: ,in, Here, S represents the normalized node features, and S is the cosine similarity matrix. It is a sparse similarity matrix. It is a nonlocal edge adjacency matrix.

[0030] Finally, the spatial edge adjacency matrix Temporal adjacency matrix Nonlocal edge adjacency matrix Together, they form a heterogeneous spatiotemporal part graph containing spatial, temporal, and nonlocal relationships, whose topological structure can be represented as: ,in, For a heterogeneous spatiotemporal region graph, , , These are the sets of spatial edges, temporal edges, and nonlocal edges, respectively.

[0031] This invention provides a preferred embodiment for executing S4. This embodiment aims to integrate node features into part category encoding, temporal position encoding, and cross-frame motion information based on the constructed heterogeneous spatiotemporal part graph. Through the graph attention part association modeling module, multi-head graph attention message passing is performed on spatial edges, temporal edges, and non-local edges to achieve deep spatiotemporal association enhancement of node features.

[0032] S41: A set of nodes in a heterogeneous spatiotemporal partial graph As input, a unique part category code and time location code are generated for each node. These two codes are then layered element-wise onto the original node features to explicitly encode the node's part identity and time location. First, a part index is generated. With time index The part index Time index corresponding to the body part to which the node belongs The corresponding time frame of the node; the part category code is generated through the embedding layer; the time code is generated jointly through the embedding layer and the linear layer. Its calculation formula is: ,in, Indicates the embedded layer for part categories; Encode the generated part category; Represents the time embedding layer; Indicates a linear projection layer; Encode the generated time location; These are the node features after superposition and encoding; This indicates an element-wise addition operation.

[0033] S42: Based on node features after superposition coding First, reshape it into a spacetime structure. Subsequently, velocity and acceleration information at the feature level are calculated separately. These two types of motion information are then projected onto the node feature dimension via a linear layer and weighted and superimposed. Velocity information is calculated from the difference in node features at the same location in adjacent frames, and acceleration information is calculated from the difference in velocity information between adjacent frames. The calculation formula is as follows: ,in, The original velocity information is given by time frame index t and human body part index n; The original acceleration information is given by time frame index t and human body part index n. and Linear projection layers representing velocity and acceleration information respectively; and These are the projected velocity and acceleration characteristics, respectively. and These are the weighting coefficients for velocity information and acceleration information, respectively. These are the node features after fusing motion information.

[0034] S43: Employs a multi-head graph attention mechanism, setting the number of attention heads h and the spatial edge adjacency matrix. Temporal adjacency matrix Nonlocal edge adjacency matrix Attention weights and adjacent node features are calculated for the node connections corresponding to the three types of edges, respectively, to obtain independently updated node features for the three types of relationships. The calculation process is as follows: , ,in, These are the projection weight matrices for vectors Q, K, and V, respectively. This indicates a feature dimension reshaping operation; These are the reshaped multi-head Q, K, and V vectors, respectively. For multi-head bias parameters; The original attention score; It is a single-head feature dimension; The adjacency matrix mask corresponds to the following respectively. and ; The attention score after masking; The processed attention weights; After aggregation feature; This represents the reverse reshaping operation of merging multiple dimensions; To output the projection matrix; The updated node features are for a single-sided type, where type can be spatial, temporal, or nonlocal, and the corresponding output is... The three types of relationships update node characteristics.

[0035] This invention provides a preferred embodiment for executing S5. This embodiment aims to achieve feature weighting by adaptively generating fusion weights based on the updated node features according to spatial, temporal, and nonlocal relationships through a node gating mechanism, and further enhance the expression of key part features by combining the node-by-node weights output by the part importance prediction module.

[0036] S51: The node features updated for the three types of relationships are used to construct a node gating mechanism through a multilayer perceptron. This generates fusion weights for spatial edges, temporal edges, and non-local edges for each node. These weights are then used to adaptively weight and sum the updated node features under the three types of relationships, achieving dynamic fusion of features from different relationships. The node-gated multilayer perceptron uses node features that fuse encoded and motion information. As input, the data passes through a dimensionality reduction linear layer, ReLU activation, a dimensionality increase linear layer, and Softmax normalization, outputting fusion weights that match the number of relation types. The weights are then multiplied element-wise by the updated features of their corresponding relations and summed to obtain the fused node features. The calculation formula is as follows: ,in, Indicates a linear layer of dimensionality reduction; This represents the linear layer of increased dimension; G is the fusion weight matrix generated by node gating. These are the weight components corresponding to spatial, temporal, and nonlocal relations, respectively. These are the node features updated for the three types of relationships; The fused node features are the fused features of the three types of relationship features.

[0037] S52: Based on fused node features, a part importance prediction module is constructed using a multilayer perceptron. A unique importance weight is generated for each node, and this weight is multiplied element-wise with the fused node features to enhance the features of key parts. The part importance multilayer perceptron uses fused node features... As input, the data sequentially passes through a dimensionality-reducing linear layer, ReLU activation, a dimensionality-increasing linear layer, and a Sigmoid activation, outputting node-by-node importance weights. These weights are then multiplied node-by-node by the fused features, combined with residual connections and normalization operations to obtain the final enhanced node features. The calculation formula is as follows: ,in, Indicates a linear layer of dimensionality reduction; Indicates a linear layer of increased dimensionality; This is a weight matrix representing the importance of different locations. These are the enhanced node features after importance weighting; This is the enhanced node feature for the final output.

[0038] This invention provides a preferred embodiment for executing S6. This embodiment aims to map the enhanced node features from the node dimension back to the original spatial dimension, eliminate the difference in part mask area through normalization, and fuse them with part-aware multi-scale features in the form of residuals, ultimately obtaining spatiotemporal structure-aware part features that combine node association enhancement and spatial structure perception.

[0039] S61: The enhanced node features are reshaped into a four-dimensional structure of batch, frame number, number of parts, and feature dimension. Background nodes are removed, and human body part node features are retained. The part mask is then adjusted to a matching spatiotemporal part format. A spatial weighted accumulation of node features and mask is achieved through Einstein summation, mapping back to the original spatial dimension to output the enhanced part structure features. The calculation formula is as follows: ,in, To exclude background nodes and define the features of human body parts; Masking of human body parts after removing the background; Enhanced feature map of the part structure.

[0040] S62: Calculate the number of pixels in the mask for each part in the spatial dimension, obtain the mask area, and sum the minimum values. (This embodiment takes) Using as the denominator, the spatial location of the enhanced structural features is normalized. The calculation formula is: , , ,in, The spatial area of ​​a single part of the mask; This represents the sum of the mask areas at each location; This is an enhanced feature of the normalized part structure.

[0041] S63: The normalized region-based structural enhancement feature map is subjected to interactive projection and nonlinear activation of feature channels through 3D convolution. Then, the feature weights are adjusted using learnable parameters, and finally fused with the region-aware multi-scale feature residuals. The calculation formula is as follows: , , ,in, This is the enhanced feature map after projection; Scaling parameters for learnable interactions; This is the scaled-up enhanced feature map; The spatiotemporal structure sensing features are the final output.

[0042] This invention provides a preferred embodiment for executing S7. This embodiment aims to integrate fine-grained part features of spatiotemporal structure perception part features, coarse-grained global features, and graph-level spatiotemporal relationship features of heterogeneous spatiotemporal part maps through a multi-granularity feature fusion module to generate a clothing change-robust identity representation vector, and generate a pedestrian matching list through cosine similarity calculation and sorting.

[0043] S71: Fine-grained part features are six types of human body part features after excluding the background, obtained by global spatiotemporal pooling of spatiotemporal structure-aware part features within the masked region of each part; coarse-grained global features are the global spatiotemporal pooling results of spatiotemporal structure-aware part features; graph-level spatiotemporal relationship features are the mean pooling results of graph node features. The calculation formula is: ,in, This represents the fine-grained feature of the nth region; It is the set of features for all fine-grained parts; It is a coarse-grained global feature; This is a mean pooling operation along the node dimension; This represents the spatiotemporal relationship characteristics at the graph level.

[0044] S72: Input the three types of features into a multilayer perceptron to generate adaptive fusion weights; simultaneously, introduce learnable global fusion weights and fuse them proportionally to obtain the final fusion weights. The calculation formula is as follows: ,in, It is a multilayer perceptron, which includes a dimension-reduced linear layer, ReLU activation, and a dimension-upgrading linear layer; For adaptive fusion weights; These are learnable global weight parameters; The normalized global fusion weights; 0.7 represents the final fusion weight; 0.7 and 0.3 represent the fusion ratio between the adaptive weight and the global weight.

[0045] S73: The final fusion weights are used to weight and sum the three types of features to obtain the fused feature. After linear layer, layer normalization, nonlinear activation, and feature normalization, a clothing-variable-robust identity representation vector is output. Finally, the cosine similarity between this vector and the features in the image library is calculated, and a pedestrian matching list is generated by sorting the similarity scores. The calculation formula is as follows: ,in, This is a multi-granularity fusion feature; This is an identity representation vector; To query the sample identity vector; This is the identity vector of the image library samples; This is the cosine similarity matrix; This is an index operation that retrieves the top r-largest similarity values; Rank is the list of matched pedestrians.

[0046] Figure 3 shows an example of a pedestrian matching list generated by a pedestrian re-identification method based on graph attention and dynamic association modeling of human body parts provided by this invention. Column 1 represents the original query image; columns 2 through 11 display the search results, sorted by similarity from highest to lowest, showing the top 10 most relevant candidate matches. This indicates a successful retrieval, meaning the candidate image and the query image belong to the same pedestrian. This indicates a retrieval error, meaning that images of different pedestrians were mistakenly identified as having the same identity.

[0047] This embodiment conducts experimental validation on the publicly available and challenging VCCR video person re-identification dataset. This dataset supports model testing for both clothing-changing scenarios (CC) and standard scenarios, providing a comprehensive and rigorous benchmark for the overall performance of the method. Model performance is evaluated using two common core metrics in the field of person re-identification: mean average accuracy (mAP) and Rank 1 accuracy in the cumulative matching feature curve.

[0048] Table 1 compares the experimental results of the proposed method and the comparison method on the VCCR dataset.

[0049] As shown in Table 1, the proposed method for re-identifying pedestrians in changing clothes based on graph attention-based dynamic association modeling of human body parts in this embodiment significantly outperforms existing comparative methods in both the mAP and Rank-1 metrics in the changing clothes scene (CC) and standard scene (Standard) of the VCCR dataset.

[0050] Specifically, in the more challenging clothing-changing scenario, the proposed method achieves an mAP of 45.7% and a Rank-1 score of 53.5%, representing improvements of 2.5 and 0.6 percentage points respectively compared to the second-best method, ASGL (mAP 43.2%, Rank-1 52.9%). In the standard scenario, the proposed method achieves mAP and Rank-1 scores of 67.3% and 88.6% respectively, also surpassing the best performance among the compared methods (ASGL: mAP 65.8%, Rank-1 88.1%). These results fully validate the effectiveness and superiority of the proposed method in handling the task of re-identifying pedestrians changing clothes.

Claims

1. A method for re-identifying pedestrians changing clothes based on graph attention-based dynamic association modeling of human body parts, characterized in that, Includes the following steps: S1: Obtain the video sequence of the pedestrian to be identified, and obtain the part mask by segmentation through the human body analysis model; S2: Based on the aforementioned part mask, a part-aware multi-scale feature extraction module is used to perform part-aware multi-scale modeling of the input video features. This module constructs multiple receptive fields using spatiotemporal convolution kernels with different dilation rates to extract multi-scale features. It also combines part-specific channel attention branches to generate part-specific channel weights, weighting and enhancing the feature channels of the corresponding parts while suppressing background region responses, outputting part-aware multi-scale features. S3: Each frame in the sequence is constructed as a graph node with its corresponding part. Spatial edges are established based on human anatomical topology, temporal edges are established based on cross-frame motion information, and... Cross-frame and cross-part motion pattern similarity establishes non-local edges, forming a heterogeneous spatiotemporal part graph containing spatial, temporal, and non-local relationships; S4: Part category encoding, temporal position encoding, and cross-frame motion information are integrated into node features, and multi-head graph attention message passing is performed on spatial edges, temporal edges, and non-local edges through the graph attention part association modeling module; S5: The fusion weight of each node feature on the three types of relationships is generated through a node gating mechanism, and the updated node features under the three types of relationships are weighted and combined; then, the node importance weights output by the part importance prediction module are multiplied node by node to obtain enhanced node features; S6: Map the enhanced node features back to the original spatial dimension, and after normalization, inject them into the aforementioned part-aware multi-scale features in the form of residuals to obtain spatiotemporal structure-aware part features. S7: Integrate the fine-grained part features and coarse-grained global features in the spatiotemporal structure perception part features with the graph-level spatiotemporal relationship features in the heterogeneous spatiotemporal part map through the multi-granularity feature fusion module to generate a clothing change robust identity representation vector; calculate the cosine similarity between this vector and the map library features, and generate a pedestrian matching list by sorting the similarity.

2. The method for re-identifying pedestrians changing clothes based on graph attention-based dynamic association modeling of human body parts as described in claim 1, characterized in that, The part-aware multi-scale feature extraction module in S2 includes: S21: using multiple sets of three-dimensional convolutional branches, each set of convolutional branches employing spatiotemporal convolutional kernels with different dilation rates, to perform convolution operations on the input video feature map to construct multiple receptive fields and extract multi-scale spatiotemporal features; each set of convolutional branches is concatenated with group convolution, batch normalization, and activation functions; after parallel processing by multiple sets of convolutional branches, multiple global feature maps of different receptive fields are obtained; S22: for each part, the corresponding part mask is multiplied with the global feature maps of each different receptive field in the spatial dimension to extract the local features of that part under different receptive fields; and all local features of the same part under different receptive fields are processed... S23: Configure a region-specific channel attention branch for each region. Each region-specific channel attention branch is processed by adaptive global spatiotemporal pooling and then sequentially passes through a channel mapping layer for dimensionality reduction and dimensionality increase, as well as nonlinear activation, to output the region-specific channel attention weight. S24: Multiply the multi-scale fusion features of each region with their corresponding region-specific channel attention weights to achieve weighted enhancement of the feature channels of the corresponding region and suppress the response of the background region, thereby obtaining the local feature representation of each region. S25: Concatenate and fuse the local feature representations of all regions along the channel dimension to obtain region-aware multi-scale features.

3. The method for re-identifying pedestrians changing clothes based on graph attention-based dynamic association modeling of human body parts as described in claim 1, characterized in that, The construction of the heterogeneous spatiotemporal part graph in S3 includes: S31: Spatial edges initialize a static adjacency matrix based on human anatomical topology, connecting only part nodes with anatomical connections within the same frame; S32: Temporal edges connect only the same part nodes in different frames, adaptively determining the maximum number of time jumps based on the cross-frame motion information of the part, and establishing bidirectional temporal connections within the range not exceeding the maximum number of jumps; S33: Non-local edges determine the connection relationship by combining the similarity of motion patterns of each part, and non-local edges exclude connections that are repeated with spatial and temporal edges, focusing on establishing cross-frame and cross-part associations, and retaining highly correlated connections after adaptive sparsification.

4. The method for re-identifying pedestrians changing clothes based on graph attention-based dynamic association modeling of human body parts as described in claim 1, characterized in that, S4 includes: S41: Part category encoding is generated through an embedding layer, where the input to the embedding layer is the part index, and the output dimension matches the node feature dimension; temporal encoding is generated through an embedding layer and a linear layer, where the input to the embedding layer is the frame index, and the linear layer projects the embedding layer output onto the node feature dimension; the part category encoding and temporal encoding are superimposed on the node features to achieve explicit encoding of node identity features and temporal location; S42: Velocity information is calculated from the difference in features of the same part in adjacent frames, and acceleration information is calculated from the velocity difference in adjacent frames; the velocity information and acceleration information are projected through a linear layer and then superimposed on the node features; S4 3: The graph attention network employs a multi-head attention mechanism, where the number of heads is set according to preset hyperparameters. Node features are projected as query, key, and value vectors. When calculating the attention score, a multi-head bias obtained from relation embedding is introduced, where relation embedding is used to distinguish different relation types such as spatial edges, temporal edges, and non-local edges. An adjacency matrix mask is applied to the attention score, and after Softmax normalization and Dropout processing, the value vector features of adjacent nodes are weighted and summed. The weighted aggregation result is then projected onto the output, and the updated node features are obtained through a nonlinear activation function.

5. The method for re-identifying pedestrians changing clothes based on graph attention-based dynamic association modeling of human body parts as described in claim 1, characterized in that, S5 includes: S51: Generating fusion weights for each node on three types of relationships—spatial edge, temporal edge, and nonlocal edge—through a node gating mechanism; the node gating mechanism is implemented through a multilayer perceptron, which is sequentially connected with a linear layer, an activation function, a linear layer, and a normalized activation function; the fusion weights are multiplied by the updated node features under the three types of relationships and summed to achieve adaptive weighted fusion of node features under the three types of relationships, resulting in fused node features; S52: Generating importance weights for each node through a location importance prediction module; the location importance prediction module is implemented through a multilayer perceptron, which is sequentially connected with a linear layer, an activation function, a linear layer, and an activation function; the importance weights are multiplied by the fused node features node by node to obtain enhanced node features.

6. The method for re-identifying pedestrians changing clothes based on graph attention-based dynamic association modeling of human body parts as described in claim 1, characterized in that, S6 includes: S61: reshaping the enhanced node features into a format of batch, frame number, part number, and feature dimension; excluding background nodes and retaining only human part nodes to obtain non-background node features and non-background part masks; using Einstein summation to weight and accumulate the non-background node features according to the corresponding non-background part masks in the spatial dimension to obtain a part structure enhancement feature map with the same shape as the part-aware multi-scale features output by S2; S62: calculating the spatial area of ​​each part mask and superimposing a small constant as a normalization factor to normalize the part structure enhancement feature map, preventing division by zero and canceling the differences in the area of ​​different part masks, to obtain a normalized part structure enhancement feature map; S63: performing a learnable interactive projection transformation on the normalized part structure enhancement feature map; introducing a learnable interactive scaling parameter, scaling the projected features and adding them to the part-aware multi-scale features output by S2 in the form of residuals to obtain spatiotemporal structure-aware part features.

7. The method for re-identifying pedestrians changing clothes based on graph attention-based dynamic association modeling of human body parts as described in claim 1, characterized in that, The multi-granularity feature fusion module in S7 includes: S71: fine-grained part features are six types of human body part features after excluding the background, obtained by global spatiotemporal pooling of spatiotemporal structure-aware part features within the masked regions of each part; coarse-grained global features are the global spatiotemporal pooling results of spatiotemporal structure-aware part features; graph-level spatiotemporal relationship features are the mean pooling results of graph node features; S72: the three types of features mentioned in S71 are input into a multilayer perceptron to generate adaptive fusion weights; at the same time, learnable global fusion weights are introduced, and the final fusion weights are calculated proportionally; S73: the three types of features are weighted and summed using the final fusion weights to obtain fused features; the fused features are sequentially passed through a linear layer, layer normalization, and nonlinear activation, and then feature normalization is performed to output an identity representation vector.

8. The method for re-identifying pedestrians changing clothes based on graph attention-based dynamic association modeling of human body parts as described in claim 1, characterized in that, It also includes a multi-layer stacking step for the multi-head graph attention message passing process of the graph attention part association modeling module described in S4 and the node feature weighted fusion and enhancement process described in S5: the multi-head graph attention message passing and node feature weighted fusion and enhancement processes are stacked in multiple layers, and the node features output by each layer together with the initial node features constitute a feature set; The feature set is weighted and summed by learnable layer weights, which are then normalized to improve the depth and robustness of the dynamic correlation modeling of parts.

Images