A method, apparatus, equipment, medium, and product for re-identifying pedestrians changing clothes.

Abstract

This application discloses a method, apparatus, device, medium, and product for re-identifying pedestrians changing clothes, relating to the field of target detection technology. The method includes: decomposing the appearance features of each pedestrian in target video data into body structure features and clothing variable features; based on the target video data, using a trained spatiotemporal graph convolutional network to generate spatiotemporal dynamic features of joint movements within the gait cycle, and combining this with a gait energy map to generate a multimodal gait representation for each pedestrian; the gait energy map is generated based on the target video data; inputting the body structure features, clothing variable features, and multimodal gait representation of each pedestrian into a trained scene-aware gating network to obtain fused features for each pedestrian; and determining whether each pedestrian is the same person based on the fused features. This application can achieve cross-modal feature complementarity, improving the accuracy of pedestrian re-identification when changing clothes.

Description

Technical Field

[0001] This application relates to the field of target detection technology, and in particular to a method, apparatus, equipment, medium and product for re-identifying pedestrians changing clothes. Background Technology

[0002] Person re-identification (Re-ID) aims to match and identify pedestrians across camera scenarios, and has broad application prospects in fields such as intelligent security, smart transportation, and video surveillance. However, when pedestrians change their clothing, traditional appearance-based person re-identification methods often experience a significant drop in accuracy due to the substantial changes in clothing appearance. Clothing is an important component of a pedestrian's appearance, and changing clothes alters key visual features such as color, texture, and style, making it difficult for models based on single appearance features to cope with such changes.

[0003] Gait, as a biological characteristic, possesses a certain degree of stability and uniqueness, and is not easily affected by clothing changes. Gait features can reflect information such as a pedestrian's posture, rhythm, and movement patterns, complementing physical appearance. Although gait features are stable, the dynamic complementary mechanism between them and physical appearance has not yet been fully explored. Summary of the Invention

[0004] The purpose of this application is to provide a method, apparatus, device, medium, and product for re-identifying pedestrians changing clothes, which can achieve cross-modal feature complementarity and improve the accuracy of pedestrian re-identification.

[0005] To achieve the above objectives, this application provides the following solution:

[0006] Firstly, this application provides a method for re-identifying pedestrians changing clothes, including:

[0007] The appearance features of each pedestrian in the target video data are decomposed into body structure features and clothing variable features;

[0008] Based on the target video data, a trained spatiotemporal graph convolutional network is used to generate spatiotemporal dynamic features of joint motion within the gait cycle, and combined with the gait energy map to generate multimodal gait representations for each pedestrian; the gait energy map is generated based on the target video data.

[0009] The body structure features, clothing variable features, and multimodal gait representations of each pedestrian are input into a trained scene-aware gating network to obtain the fused features of each pedestrian.

[0010] Based on the merging characteristics of each pedestrian, determine whether each pedestrian is the same person.

[0011] Optionally, the appearance features of each pedestrian in the target video data are decomposed into body structure features and clothing variable features, specifically including:

[0012] The target video data is processed to obtain the sequence of key point coordinates of pedestrians, body structure region mask, and clothing region mask in each single frame image of the target video data.

[0013] Extract the global features of each single frame image, and combine them with the corresponding body structure region mask and clothing region mask to obtain the body structure region mask and clothing region mask of the pedestrian in each single frame image;

[0014] A trained dual-stream network is used to extract body structure features and clothing variable features from the body structure region mask and clothing region mask of each single frame image.

[0015] Optionally, the target video data is processed to obtain the sequence of keypoint coordinates of pedestrians, body structure region masks, and clothing region masks in each single frame image of the target video data, specifically including:

[0016] A trained human pose estimation model is used to extract each preset joint point from each single frame image, and each preset joint point constitutes a joint point coordinate sequence.

[0017] The trained semantic segmentation model is used to extract body structure region masks and clothing region masks from each single frame image.

[0018] Optionally, global image features of each single frame are extracted and combined with the corresponding body structure region mask and clothing region mask to obtain the body structure region mask and clothing region mask of the pedestrian in each single frame image, specifically including:

[0019] Global image features of each single frame were extracted using ResNet50 as the backbone network.

[0020] The body structure region mask and clothing region mask are optimized through thresholding and morphological operations to obtain the body structure region mask and clothing region mask of the pedestrian in each single frame image.

[0021] Optionally, the loss function used during the training of the trained two-stream network is a contrastive loss function;

[0022]

[0023] Among them, L sep L represents the comparative loss. contrast () denotes the contrastive loss function, f1 and f2 represent the two input variables, and the superscript T indicates transpose. and Represents the body structure feature vectors of the same pedestrian at different times; and This represents the variable feature vector of clothing of the same pedestrian at different times, where N is the total number of negative samples, i represents the index of the currently processed positive sample, j represents the index of the negative sample, and f 2,j Let τ represent the body structure feature vector or clothing variable feature vector of the j-th negative sample, and τ represent the temperature parameter.

[0024] Optionally, based on the target video data, a trained spatiotemporal graph convolutional network is used to generate spatiotemporal dynamic features of joint movements within the gait cycle, and combined with the gait energy map to generate multimodal gait representations for each pedestrian, specifically including:

[0025] A spatiotemporal graph is constructed based on the key point coordinate sequence. In the time dimension, the spatiotemporal graph establishes connections between the same key points of each adjacent frame image. In the spatial dimension, for each single frame image, connections are established between adjacent key points based on the human skeletal structure.

[0026] A trained spatiotemporal dynamic feature extraction spatiotemporal graph convolutional network is used to extract spatiotemporal dynamic features from the spatiotemporal graph; the trained spatiotemporal dynamic feature extraction spatiotemporal graph convolutional network is a structure with two layers of graph convolution and two layers of temporal convolution stacked alternately.

[0027] Convert the target video data into a gait energy map;

[0028] The spatiotemporal dynamic features and gait energy map are concatenated along the channel dimension to obtain the multimodal gait representation.

[0029] Secondly, this application provides a pedestrian re-identification device for changing clothes, comprising:

[0030] The appearance feature decoupling module is used to decompose the appearance features of each pedestrian in the target video data into body structure features and clothing variable features;

[0031] The gait feature enhancement module is used to generate spatiotemporal dynamic features of joint motion within the gait cycle based on target video data using a trained spatiotemporal graph convolutional network, and to generate multimodal gait representations of each pedestrian by combining the gait energy map; the gait energy map is generated based on the target video data.

[0032] The feature fusion module is used to input the body structure features, clothing variable features, and multimodal gait representations of each pedestrian into the trained scene perception gating network to obtain the fused features of each pedestrian;

[0033] The recognition module is used to determine whether each pedestrian is the same person based on the fusion characteristics of each pedestrian.

[0034] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the pedestrian re-identification method for changing clothes as described above.

[0035] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the pedestrian re-identification method for changing clothes described above.

[0036] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the pedestrian re-identification method for changing clothes described above.

[0037] According to the specific embodiments provided in this application, the following technical effects are disclosed:

[0038] This application provides a method, apparatus, device, medium, and product for re-identifying pedestrians changing clothes. It decomposes the appearance features of each pedestrian in the target video data into body structure features and clothing-variable features, effectively decoupling clothing-independent body structure features and clothing-variable features, reducing the interference of clothing changes on appearance features. Based on the target video data, a trained spatiotemporal graph convolutional network is used to generate spatiotemporal dynamic features of joint movements within the gait cycle. Combined with the gait energy map, multimodal gait representations of each pedestrian are generated, enhancing the expressive power of gait features. An adaptive fusion of appearance and gait features is achieved through a scene-aware gating network, dynamically adjusting feature weights according to the degree of clothing change, realizing cross-modal feature complementarity, and improving the accuracy of pedestrian re-identification changing clothes. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 A flowchart illustrating a method for re-identifying pedestrians changing clothes, provided as an embodiment of this application;

[0041] Figure 2 A detailed flowchart illustrating a method for re-identifying pedestrians changing clothes, provided in an embodiment of this application.

[0042] Figure 3 This is an overall framework diagram of a pedestrian re-identification method for changing clothes provided in an embodiment of this application;

[0043] Figure 4 This is a schematic diagram of the appearance feature decoupling module structure provided in an embodiment of this application;

[0044] Figure 5 This is a schematic diagram of the gait feature enhancement module structure provided in an embodiment of this application;

[0045] Figure 6 A functional module diagram of a pedestrian re-identification device for changing clothes is provided in one embodiment of this application;

[0046] Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0047] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0048] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0049] Existing methods for re-identifying pedestrians changing clothes have the following limitations: Feature coupling problem: Clothing and body features are not effectively decoupled in appearance features, resulting in feature drift after changing clothes; Rigid modality fusion: Fixed weight fusion cannot adapt to the dynamic changes in feature importance under different changing scenarios; Insufficient utilization of spatiotemporal information: Gait feature extraction relies only on traditional spatiotemporal networks and does not fully model the joint motion trajectory within the gait cycle.

[0050] This application proposes a three-level framework for dynamic feature decoupling, enhancement, and fusion:

[0051] Appearance feature decoupling: Through a feature separation module guided by semantic segmentation, appearance features are decomposed into clothing-independent body structure features (such as body shape and joint position) and clothing-variable features, thus suppressing interference from changing clothes.

[0052] Gait feature enhancement: A spatiotemporal graph convolutional network (ST-GCN) is introduced to model the motion trajectory of joints within the gait cycle, and combined with the gait energy map (GEI) to generate a multimodal gait representation.

[0053] Adaptive fusion mechanism: Design a scene-aware gating network (SANet) to dynamically adjust the fusion weights of appearance and gait features according to the degree of clothing change, so as to achieve cross-modal feature complementarity.

[0054] In one exemplary embodiment, this application provides a method for re-identifying pedestrians changing clothes, such as... Figures 1-3 As shown, the pedestrian re-identification method for changing clothes includes steps 101-102.

[0055] Step 101: Decompose the appearance features of each pedestrian in the target video data into body structure features and clothing variable features.

[0056] Step 102: Based on the target video data, a trained spatiotemporal graph convolutional network is used to generate spatiotemporal dynamic features of joint motion within the gait cycle, and combined with the gait energy map to generate multimodal gait representations of each pedestrian; the gait energy map is generated based on the target video data.

[0057] Step 103: Input the body structure features, clothing variable features, and multimodal gait representations of each pedestrian into the trained scene-aware gating network to obtain the fused features of each pedestrian;

[0058] Step 104: Based on the merging characteristics of each pedestrian, determine whether each pedestrian is the same person.

[0059] In one exemplary embodiment, such as Figure 4 As shown, step 101 specifically includes steps 201-203.

[0060] Step 201: Process the target video data to obtain the key point coordinate sequence of pedestrians, body structure region mask, and clothing region mask in each single frame image of the target video data.

[0061] Step 202: Extract the global features of each single frame image, and combine them with the corresponding body structure region mask and clothing region mask to obtain the body structure region mask and clothing region mask of the pedestrian in each single frame image.

[0062] Step 203: Use a trained dual-stream network to extract body structure features and clothing variable features from the body structure region mask and clothing region mask of each single frame image.

[0063] In an exemplary embodiment, the target video data is processed to obtain the key point coordinate sequence, body structure region mask, and clothing region mask of the pedestrian in each single frame image of the target video data. Specifically, this includes: extracting each preset key point from each single frame image using a trained human pose estimation model, and the preset key points forming a key point coordinate sequence; and extracting the body structure region mask and clothing region mask from each single frame image using a trained semantic segmentation model.

[0064] The target video data is video data containing a changing scene. The video data containing the changing scene is then processed as follows:

[0065] Single-frame images are sampled at a fixed frame rate and the resolution is uniformly set to 256×128.

[0066] Human pose estimation models (such as OpenPose) are used to extract the key point coordinate sequence of pedestrians in each single frame image, and semantic region masks such as clothing and body are generated through semantic segmentation models (DeepLabv3+).

[0067] Data augmentation (random flipping, cropping) and normalization are performed on the images to provide more sample images and improve the model training effect.

[0068] In an exemplary embodiment, global image features of each single frame are extracted, and combined with the corresponding body structure region mask and clothing region mask to obtain the body structure region mask and clothing region mask of the pedestrian in each single frame image, specifically including:

[0069] Global image features of each single frame were extracted using ResNet50 as the backbone network.

[0070] The body structure region mask and clothing region mask are optimized through thresholding and morphological operations to obtain the body structure region mask and clothing region mask of the pedestrian in each single frame image.

[0071] In step 203, body structural features include body shape and joint position, while clothing variable features include color, texture, and style.

[0072] In one exemplary embodiment, the trained two-stream network uses a contrastive loss function during training.

[0073] The contrastive loss function is expressed as follows:

[0074] Among them, L sep L represents the comparative loss. contrast () denotes the contrastive loss function, f1 and f2 represent the two input variables, and the superscript T indicates transpose. and Represents the body structure feature vectors of the same pedestrian at different times; and Let f represent the variable feature vector of the clothing of the same pedestrian at different times, N be the total number of negative samples in the current batch, i be the index of the positive sample being processed, corresponding to the i-th sample in the batch, and the value range is 1≤i≤B (B is the batch size); j be the index of the negative sample, satisfying j≠i, that is, excluding other pedestrian indices of the current positive sample pair; 2,jτ represents the body structure feature vector or clothing variable feature vector of the j-th negative sample; τ represents the temperature parameter used to scale the probability distribution of feature vector similarity, with a value of 0.1.

[0075] In one exemplary embodiment, such as Figure 5 As shown, step 102 specifically includes steps 301-304.

[0076] Step 301: Construct a spatiotemporal graph based on the key point coordinate sequence. In the time dimension, the spatiotemporal graph establishes connections between the same key points of each adjacent frame image. In the spatial dimension, for each single frame image, connections are established between adjacent key points based on the human skeletal structure.

[0077] Step 302: Extract spatiotemporal dynamic features from the spatiotemporal graph using a trained spatiotemporal dynamic feature extraction spatiotemporal graph convolutional network; the trained spatiotemporal dynamic feature extraction spatiotemporal graph convolutional network is a structure with two layers of graph convolution and two layers of temporal convolution stacked alternately.

[0078] Step 303: Convert the target video data into a gait energy map.

[0079] Step 304: Concatenate the spatiotemporal dynamic features and the gait energy map along the channel dimension to obtain the multimodal gait representation.

[0080] This application utilizes joint coordinates to construct a spatiotemporal graph, and models the joint motion trajectory within the gait cycle through a spatiotemporal graph convolutional network to extract spatiotemporal dynamic features.

[0081] The target video data, i.e., the video sequence, is converted into a gait energy map (GEI). The spatiotemporal dynamic features and GEI static features are fused to generate a multimodal gait representation that includes temporal dynamics and global energy distribution.

[0082] Step 103: Concatenate body structure features, clothing variable features, and multimodal gait features, and input them into a fully connected network to extract scene features related to the degree of clothing change.

[0083] The gating unit generates fusion weights for appearance and gait features based on scene characteristics. When the degree of clothing change is high, the weight of gait features is increased, and vice versa, the weight of appearance features is increased, so as to achieve feature complementarity.

[0084] During the training phase, the cross-entropy loss function is used to optimize the pedestrian re-identification model for changing clothes. During the testing phase, cosine similarity is used to calculate the pedestrian feature similarity to complete the re-identification task of pedestrians changing clothes across cameras.

[0085] The pedestrian re-identification model for changing clothes includes a decoupling module for appearance features, a gait feature enhancement module, a scene-aware gating network, and a classifier connected in sequence. The appearance feature decoupling module implements the function of step 101, the gait feature enhancement module implements the function of step 102, and the scene-aware gating network and classifier implement the functions of steps 103 and 104, respectively.

[0086] In the testing phase, each pedestrian is an independent detection target in a single frame. The feature of each pedestrian is the fusion result of body and clothing features. In the testing phase, the fused features (including body structure and clothing features) of each pedestrian detected in a single frame image are extracted. The consistency of the vector direction of pedestrian features under different cameras is calculated by using cosine similarity to determine whether they are the same person. This mainly takes into account both feature discriminativeness (optimized in the training phase) and matching efficiency (lightweight computation in the testing phase). The pre-trained model is used to complete the feature matching forward propagation to extract features without updating the parameters.

[0087] In one exemplary embodiment, a method for re-identifying pedestrians changing clothes is implemented based on the deep learning framework PyTorch.

[0088] Step S1: Sample the video from the publicly available pedestrian re-identification dataset at a frame rate of 15 frames per second to obtain a set of single-frame images. I i Let represent the i-th frame image, and N be the number of images in a single frame. The resolution of all images is uniformly adjusted to 256×128 pixels for easier subsequent network processing.

[0089] We used the open-source human pose estimation model OpenPose, whose network structure is based on a convolutional neural network (CNN), initialized with parameters pre-trained on the COCO dataset. A single frame image I... i By inputting the OpenPose model, 18 key joints (preset joints) can be detected, yielding a sequence of joint coordinates. J i Let j be the coordinate sequence of the j-th joint. i,k Let j be the coordinates of the k-th joint in the j-th joint coordinate sequence, where each joint coordinate j i,k =(x i,k ,y i,k ), x i,k and y i,k These represent the horizontal and vertical coordinates of the joint in the image. Preset joints include the shoulder, elbow, and hip joints.

[0090] The semantic segmentation model DeepLabv3+ is used, which uses ResNet50 as the backbone network and is pre-trained on the Cityscapes dataset. Image I... iInput DeepLabv3+, output the mask S for different semantic regions such as clothing, skin, and hair. i Mask S i Is with image I i Multi-channel images of the same size, with each channel corresponding to a semantic category.

[0091] For image I i Data augmentation operations such as random flipping and cropping are performed, and the image is normalized to the [0,1] interval to facilitate subsequent network processing.

[0092] Step S2: Input the preprocessed image into the appearance feature decoupling module, which consists of a semantic segmentation guidance branch and a feature separation branch. Specifically, it includes:

[0093] Step S21: Extract image I using ResNet50 as the backbone network i global features F global The ResNet50 consists of 5 residual blocks, and the network's output feature map size is 8×4×2048. During training, weights are randomly initialized, and batch normalization is used to reduce internal covariate bias and improve the stability of network training.

[0094] Step S22: Apply the mask S obtained from semantic segmentation i Through thresholding and morphological operations, a more accurate mask M for the body structure region is obtained. body and clothing area mask M cloth Among them, M body and M cloth Is it related to global feature F global The corresponding binary mask has a value of 1 indicating that the corresponding feature is valid, and 0 indicating that it is invalid.

[0095] Step S23: The two-stream network structure for feature separation branches, each consisting of 3 convolutional layers and 2 fully connected layers. The body structure feature extraction branch will use F... global With M body Perform element-wise multiplication to obtain the body structural region features F. body Then, convolutional layers are used to further extract body structural features unrelated to clothing, such as body shape and joint position. The kernel sizes of the convolutional layers are 3×3, 3×3, and 3×3, with a stride of 1 and padding of 1 to maintain the feature map size. The variable feature extraction branch for clothing is F. global With M cloth The process involves extracting variable features such as color, texture, and style from the clothing. During training, a contrastive loss function L is designed to encourage the two branches to focus on their respective features. sep :

[0096]

[0097] Among them, the contrast loss function L contrast The calculation is as follows:

[0098]

[0099] In the formula, f1 and f2 are eigenvectors, τ is a temperature parameter set to 0.1, and L is minimized. sep This achieves effective decoupling of appearance features.

[0100] Step S3: Process the keypoint coordinate sequence in the video sequence to generate a multimodal gait representation. Specifically, this includes:

[0101] Step S31: Based on the joint coordinate sequence J i Construct a spatiotemporal graph G = (V, E), where nodes V = {v i,k The diagram corresponds to 18 joints, and edge E represents the connection relationship between the joints in time and space. In the time dimension, connections are established between the same joints in adjacent frames; in the spatial dimension, connections are established between adjacent joints based on the human skeletal structure.

[0102] Step S32: Spatiotemporal Dynamic Feature Extraction. The Spatiotemporal Graph Convolutional Network (ST-GCN) adopts a structure of alternating stacks of two layers of graph convolution (GCN) and two layers of temporal convolution (TCN). The specific network architecture is as follows:

[0103] During network training, the initial learning rate is set to 1×10. -3 It adopts an exponential decay strategy, the loss function is a weighted sum of contrast loss and cross-entropy loss, and the optimizer is AdamW.

[0104] Define a spatiotemporal graph G = (V, E), where node V corresponds to 18 predefined joints (such as shoulder, elbow, and hip joints), and E is the set of edges between nodes. In the temporal dimension, the same joint in adjacent frames is connected by temporal edges. In the spatial dimension, spatial edges are constructed based on the natural connections of the human skeleton (such as shoulder joint connecting to elbow joint, hip joint connecting to knee joint). Joint coordinate sequence. (T is the number of frames, M is the number of keypoints, and 2 is the coordinate dimension), converted into the adjacency matrix of the spatiotemporal graph. Non-zero elements represent the connections between nodes. Represents the set of real numbers. The superscript of is the dimension of the set of real numbers.

[0105] Each graph convolutional layer aggregates node features using a learnable weight matrix W, as shown in the formula:

[0106]

[0107] Where X' is the aggregation feature. For the normalized adjacency matrix, To add a self-loop adjacency matrix, Let X be the degree matrix, X be the input features, and σ be the ReLU activation function.

[0108] The temporal convolutional layer uses a one-dimensional convolutional kernel with a kernel size of 3 and a stride of 1 to model the temporal series features of each joint, capturing the motion patterns within the gait cycle. Spatiotemporal convolutions are performed alternately, ultimately outputting the spatiotemporal dynamic features of the gait. (C represents the number of feature channels.)

[0109] Step S33: Convert the video sequence into a gait energy map (GEI). The pixel value GEI(x,y) of each GEI is calculated as shown in the formula:

[0110]

[0111] Where T is the number of frames in the video sequence, I t (x,y) is the image of frame t. In this embodiment, a complete gait cycle is selected as the T value. To enhance the contrast of GEI, the calculated GEI is normalized so that its pixel value ranges from [0,255].

[0112] Step S34: Extract the dynamic features F from ST-GCN st The static features of GEI are concatenated and fused together. First, GEI is converted into a form that is compatible with F. st Feature vectors of the same dimension are then concatenated along the channel dimension to obtain the multimodal gait representation F. gait .

[0113] Step S4: The Scene-Aware Gated Network (SANet) consists of a scene feature extraction subnetwork and a gating weight generation subnetwork, with the following specific structure:

[0114] The input for scene feature extraction is body structure features. Variable characteristics of clothing and multimodal gait features First, the splicing feature F is obtained through a splicing operation. concat :

[0115]

[0116] The scene feature extraction subnetwork consists of 3 fully connected layers:

[0117] First fully connected layer: Input dimension D1+D2+D3(F) concatThe output dimension is 1024, the activation function is LeakyReLU, and a Dropout layer is added to prevent overfitting.

[0118] The second fully connected layer has an input dimension of 1024, an output dimension of 512, and uses the Swish activation function.

[0119] The third fully connected layer has an input dimension of 512 and an output dimension of 256, generating scene features. This feature encodes joint information on the degree of change in pedestrian appearance and gait stability.

[0120] The fusion weight generation gated weight generation subnetwork consists of one fully connected layer and a Softmax activation function:

[0121] The fully connected layer has an input dimension of 256, an output dimension of 2, and a weight matrix. bias Calculate the gating score

[0122] The Softmax function normalizes the gated scores into fusion weights α and β, satisfying α + β = 1, as shown in the formula:

[0123]

[0124] Where α is the weight of appearance features and β is the weight of gait features. When pedestrians change clothes frequently (e.g., significant changes in clothing color and style), β automatically increases, enhancing the reliance on stable gait features; when the frequency of clothing changes is low, α increases, making full use of the body structure information in appearance features.

[0125] Feature fusion and appearance features With gait features F gait The final fused features are obtained by weighted summation:

[0126] F final =α·F appearance +β·F gait .

[0127] Step S5: Perform recognition and classification, and fuse the features F final The input is fed into the classification layer (classifier), which is a fully connected layer with the number of neurons equal to the number of pedestrian identity categories in the dataset. The cross-entropy loss function L is used. ce The training process is performed, and the calculation is shown in the formula:

[0128] In the formula, C represents the number of pedestrian categories, and y c For real labels, p cTo predict class probabilities, a batch size of 32 was set during training, with a total of 200 training batches (epochs). During the testing phase, the fused feature F was used... final The cosine similarity between pedestrians is calculated, and pedestrians with similarity scores higher than a set threshold are identified as having the same identity, thus achieving the task of re-identifying pedestrians who have changed clothes.

[0129] This application effectively decouples clothing-independent body structure features and clothing-variable features through a semantic segmentation-guided feature separation module, reducing the interference of clothing changes on appearance features. It utilizes a spatiotemporal graph convolutional network combined with gait energy maps to generate multimodal gait representations, enhancing the expressive power of gait features. Furthermore, it achieves adaptive fusion of appearance and gait features through a scene-aware gating network, enabling the model to dynamically adjust feature weights based on the degree of clothing change, achieving cross-modal feature complementarity. Experimental results demonstrate that this application significantly improves performance in pedestrian re-identification tasks involving clothing changes, effectively addressing the challenges of clothing changes in real-world scenarios.

[0130] Based on the same inventive concept, this application also provides a pedestrian re-identification device for implementing the aforementioned pedestrian re-identification method for changing clothes. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the pedestrian re-identification device for changing clothes provided below can be found in the limitations of the pedestrian re-identification method for changing clothes described above, and will not be repeated here.

[0131] In one exemplary embodiment, such as Figure 6 As shown, a pedestrian re-identification device for changing clothes is provided, comprising:

[0132] The appearance feature decoupling module is used to decompose the appearance features of each pedestrian in the target video data into body structure features and clothing variable features.

[0133] The gait feature enhancement module is used to generate spatiotemporal dynamic features of joint motion within the gait cycle based on target video data using a trained spatiotemporal graph convolutional network, and to generate multimodal gait representations of each pedestrian by combining the gait energy map; the gait energy map is generated based on the target video data.

[0134] The feature fusion module is used to input the body structure features, clothing variable features, and multimodal gait representations of each pedestrian into a trained scene-aware gating network to obtain the fused features of each pedestrian.

[0135] The recognition module is used to determine whether each pedestrian is the same person based on the fusion characteristics of each pedestrian.

[0136] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 7As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores pedestrian re-identification data related to changing clothes. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a pedestrian re-identification method for changing clothes.

[0137] Those skilled in the art will understand that Figure 7 The structures shown are merely block diagrams of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0138] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0139] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0140] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0141] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0142] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units, etc., and are not limited to these.

[0143] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0144] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for re-identifying pedestrians changing clothes, characterized in that, The method for re-identifying pedestrians changing clothes includes: The appearance features of each pedestrian in the target video data are decomposed into body structure features and clothing variable features; body structure features include body shape and joint position; clothing variable features include color, texture and style. Based on the target video data, a trained spatiotemporal graph convolutional network is used to generate spatiotemporal dynamic features of joint motion within the gait cycle, and combined with the gait energy map to generate multimodal gait representations for each pedestrian; the gait energy map is generated based on the target video data. The body structure features, clothing variable features, and multimodal gait representations of each pedestrian are input into a trained scene-aware gating network to obtain the fused features of each pedestrian. Based on the merging characteristics of each pedestrian, determine whether the pedestrians are the same person; The appearance features of each pedestrian in the target video data are decomposed into body structure features and variable clothing features, specifically including: The target video data is processed to obtain the sequence of key point coordinates of pedestrians, body structure region mask, and clothing region mask in each single frame image of the target video data. Extract the global features of each single frame image, and combine them with the corresponding body structure region mask and clothing region mask to obtain the body structure region mask and clothing region mask of the pedestrian in each single frame image; A trained dual-stream network is used to extract body structure features and clothing variable features from the body structure region mask and clothing region mask of each single frame image.

2. The method for re-identifying pedestrians changing clothes according to claim 1, characterized in that, The target video data is processed to obtain the sequence of keypoint coordinates of pedestrians, body structure region masks, and clothing region masks in each single frame image of the target video data, specifically including: A trained human pose estimation model is used to extract each preset joint point from each single frame image, and each preset joint point constitutes a joint point coordinate sequence. The trained semantic segmentation model is used to extract body structure region masks and clothing region masks from each single frame image.

3. The method for re-identifying pedestrians changing clothes according to claim 1, characterized in that, Global image features are extracted from each single frame image, and combined with the corresponding body structure region mask and clothing region mask to obtain the body structure region mask and clothing region mask of the pedestrian in each single frame image, specifically including: Global image features of each single frame were extracted using ResNet50 as the backbone network. The body structure region mask and clothing region mask are optimized through thresholding and morphological operations to obtain the body structure region mask and clothing region mask of the pedestrian in each single frame image.

4. The method for re-identifying pedestrians changing clothes according to claim 1, characterized in that, The trained two-stream network uses a contrastive loss function during training. The contrastive loss function is expressed as follows: ; ； in, Indicates comparative loss, This represents the contrastive loss function. and This represents two input variables, with the superscript T indicating transpose. and Represents the body structure feature vectors of the same pedestrian at different times; and This represents the variable feature vector of the clothing of the same pedestrian at different times, where N is the total number of negative samples, i represents the index of the currently processed positive sample, and j represents the index of the negative sample. This represents the body structure feature vector or clothing variable feature vector of the j-th negative sample. This represents the temperature parameter.

5. The method for re-identifying pedestrians changing clothes according to claim 1, characterized in that, Based on the target video data, a trained spatiotemporal graph convolutional network is used to generate spatiotemporal dynamic features of joint movements within the gait cycle. Combined with the gait energy map, multimodal gait representations for each pedestrian are generated, specifically including: A spatiotemporal graph is constructed based on the key point coordinate sequence. In the time dimension, the spatiotemporal graph establishes connections between the same key points of each adjacent frame image. In the spatial dimension, for each single frame image, connections are established between adjacent key points based on the human skeletal structure. A trained spatiotemporal dynamic feature extraction spatiotemporal graph convolutional network is used to extract spatiotemporal dynamic features from the spatiotemporal graph; the trained spatiotemporal dynamic feature extraction spatiotemporal graph convolutional network is a structure with two layers of graph convolution and two layers of temporal convolution stacked alternately; Convert the target video data into a gait energy map; The spatiotemporal dynamic features and gait energy map are concatenated along the channel dimension to obtain the multimodal gait representation.

6. A device for re-identifying pedestrians changing clothes, characterized in that, The pedestrian re-identification device for changing clothes applies the pedestrian re-identification method for changing clothes as described in any one of claims 1-5, and the pedestrian re-identification device for changing clothes comprises: The appearance feature decoupling module is used to decompose the appearance features of each pedestrian in the target video data into body structure features and clothing variable features; The gait feature enhancement module is used to generate spatiotemporal dynamic features of joint motion within the gait cycle based on target video data using a trained spatiotemporal graph convolutional network, and to generate multimodal gait representations of each pedestrian by combining the gait energy map; the gait energy map is generated based on the target video data. The feature fusion module is used to input the body structure features, clothing variable features, and multimodal gait representations of each pedestrian into the trained scene perception gating network to obtain the fused features of each pedestrian; The recognition module is used to determine whether each pedestrian is the same person based on the fusion characteristics of each pedestrian.

7. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the pedestrian re-identification method for changing clothes as described in any one of claims 1-5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the pedestrian re-identification method for changing clothes as described in any one of claims 1-5.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the pedestrian re-identification method for changing clothes as described in any one of claims 1-5.

Images