Clothes changing pedestrian re-identification method, device and equipment, medium and product

Abstract

The invention discloses a clothes changing pedestrian re-identification method, device and equipment, a medium and a product, and relates to the technical field of target detection, and the method comprises the steps: decomposing the appearance characteristics of each pedestrian in target video data into body structure characteristics and clothes variable characteristics; on the basis of target video data, generating space-time dynamic characteristics of joint movement in a gait cycle by adopting a trained space-time diagram convolutional network, and generating multi-modal gait representation of each pedestrian in combination with a gait energy diagram; the gait energy diagram is generated according to target video data; inputting the body structure features, the clothes variable features and the multi-modal gait representation of each pedestrian into a trained scene perception gating network to obtain fusion features of each pedestrian; according to the fusion features of the pedestrians, whether the pedestrians are the same person or not is judged, cross-modal feature complementation can be achieved, and the accuracy of re-recognition of the clothes changing pedestrians is improved.

Description

Technical Field

[0001] The present application relates to the field of target detection technology, and in particular to a method, device, equipment, medium and product for re-identifying pedestrians who have changed clothes. Background Art

[0002] Person re-identification (Re-ID) aims to match and identify pedestrians across multiple camera scenarios, and has broad application prospects in intelligent security, smart transportation, video surveillance, and other fields. However, when pedestrians change their clothing, traditional appearance-based person re-ID methods often suffer from a significant drop in recognition accuracy due to the significant changes in clothing appearance. Clothing is an important component of a pedestrian's appearance, and changing clothes can alter key visual features such as color, texture, and style, making it difficult for models based on single appearance features to cope with such changes.

[0003] As a biometric characteristic, gait is relatively stable and unique, and is not easily affected by clothing changes. Gait features reflect information such as a pedestrian's posture, rhythm, and movement patterns while walking, complementing appearance features. Despite the stability of gait features, their dynamic complementarity with appearance features has not been fully explored. Summary of the Invention

[0004] The purpose of this application is to provide a method, device, equipment, medium and product for re-identifying pedestrians who change clothes, which can achieve cross-modal feature complementarity and improve the accuracy of re-identification of pedestrians who change clothes.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a method for re-identifying a pedestrian who has changed clothes, comprising:

[0007] Decompose the appearance features of each pedestrian in the target video data into body structure features and clothing variable features;

[0008] Based on the target video data, a trained spatiotemporal graph convolutional network is used to generate spatiotemporal dynamic features of joint motion within the gait cycle, and a multimodal gait representation of each pedestrian is generated by combining the gait energy graph generated based on the target video data.

[0009] The body structure features, clothing variable features, and multimodal gait representation of each pedestrian are input into the trained scene perception gating network to obtain the fused features of each pedestrian;

[0010] Based on the fusion features of each pedestrian, it is determined whether the pedestrians are the same person.

[0011] Optionally, the appearance features of each pedestrian in the target video data are decomposed into body structure features and clothing variable features, specifically including:

[0012] Processing the target video data to obtain a joint point coordinate sequence, a body structure region mask, and a clothing region mask of the pedestrian in each single frame image in the target video data;

[0013] Extract the global image features of each single frame image, combine them with the corresponding body structure region mask and clothing region mask, and obtain the pedestrian's body structure region mask and clothing region mask in each single frame image;

[0014] The trained two-stream network is used to extract body structure features and clothing variable features from the body structure region mask and clothing region mask of each single frame image.

[0015] Optionally, processing the target video data to obtain a joint point coordinate sequence, a body structure region mask, and a clothing region mask of a pedestrian in each single frame image in the target video data specifically includes:

[0016] A trained human posture estimation model is used to extract each preset joint point from each single frame image, and each preset joint point constitutes a joint point coordinate sequence;

[0017] The trained semantic segmentation model is used to extract body structure region masks and clothing region masks from each single frame image.

[0018] Optionally, extracting the global image features of each single-frame image and combining the corresponding body structure region mask and clothing region mask to obtain the body structure region mask and clothing region mask of the pedestrian in each single-frame image specifically includes:

[0019] ResNet50 is used as the backbone network to extract the global features of each single frame image;

[0020] The body structure region mask and clothing region mask are optimized through threshold processing and morphological operations to obtain the pedestrian's body structure region mask and clothing region mask in each single frame image.

[0021] Optionally, the loss function used by the trained two-stream network during training is a contrast loss function;

[0022]

[0023] Among them, L sep represents the contrast loss, L contrast () represents the contrast loss function, f1 and f2 represent two input variables, and the superscript T represents transposition. and Represents the body structure feature vector of the same pedestrian at different times; and represents the variable feature vector of clothing of the same pedestrian at different times, N is the total number of negative samples, i represents the index of the positive sample currently being processed, j represents the index of the negative sample, and f 2,j represents the body structure feature vector or clothing variable feature vector of the jth negative sample, and τ represents the temperature parameter.

[0024] Optionally, based on the target video data, a trained spatiotemporal graph convolutional network is used to generate spatiotemporal dynamic features of joint motion within the gait cycle, and combined with the gait energy graph to generate a multimodal gait representation of each pedestrian, specifically including:

[0025] Constructing a spatiotemporal graph based on the joint point coordinate sequence, wherein the spatiotemporal graph establishes connections between the same joint points in adjacent frames in the time dimension, and establishes connections between adjacent joint points in each single frame according to the human skeletal structure in the spatial dimension;

[0026] A trained spatiotemporal dynamic feature extraction spatiotemporal graph convolutional network is used to extract spatiotemporal dynamic features from the spatiotemporal graph; the trained spatiotemporal dynamic feature extraction spatiotemporal graph convolutional network is a structure in which two layers of graph convolution and two layers of time convolution are alternately stacked.

[0027] Convert target video data into gait energy graph;

[0028] The spatiotemporal dynamic features and the gait energy map are spliced ​​in the channel dimension to obtain the multimodal gait representation.

[0029] In a second aspect, the present application provides a device for re-identifying pedestrians who have changed clothes, comprising:

[0030] Appearance feature decoupling module, used to decompose the appearance features of each pedestrian in the target video data into body structure features and clothing variable features;

[0031] A gait feature enhancement module is used to generate spatiotemporal dynamic features of joint motion within the gait cycle based on the target video data using a trained spatiotemporal graph convolutional network, and to generate a multimodal gait representation of each pedestrian in combination with a gait energy graph generated based on the target video data;

[0032] The feature fusion module is used to input the body structure features, clothing variable features and multimodal gait representation of each pedestrian into the trained scene perception gating network to obtain the fused features of each pedestrian;

[0033] The recognition module is used to determine whether the pedestrians are the same person based on their fusion features.

[0034] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above-described methods for re-identifying pedestrians changing clothes.

[0035] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-mentioned methods for re-identifying pedestrians changing clothes.

[0036] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned methods for re-identifying pedestrians who change clothes.

[0037] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0038] The present application provides a method, apparatus, equipment, medium and product for re-identifying pedestrians who change clothes, which decomposes the appearance features of each pedestrian in the target video data into body structure features and clothing variable features, effectively decouples body structure features and clothing variable features that are unrelated to clothing, and reduces the interference of clothing change on appearance features. Based on the target video data, a trained spatiotemporal graph convolutional network is used to generate the spatiotemporal dynamic features of joint point movements within the gait cycle, and a gait energy graph is combined to generate a multimodal gait representation of each pedestrian, thereby enhancing the expressive ability of gait features. The scene-aware gating network is used to achieve adaptive fusion of appearance and gait features, and the feature weights can be dynamically adjusted according to the degree of clothing change, thereby achieving cross-modal feature complementarity and improving the accuracy of re-identification of pedestrians who change clothes. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0040] Figure 1 A flowchart of a method for re-identifying a pedestrian who has changed clothes provided in one embodiment of the present application;

[0041] Figure 2 A detailed flow chart of a method for re-identifying a pedestrian who has changed clothes provided in one embodiment of the present application

[0042] Figure 3 This is a diagram showing the overall framework of a method for re-identifying a person who has changed clothes, provided in one embodiment of the present application;

[0043] Figure 4 A schematic diagram of the structure of an appearance feature decoupling module provided in one embodiment of the present application;

[0044] Figure 5 A schematic diagram of the structure of a gait feature enhancement module provided in one embodiment of the present application;

[0045] Figure 6 A schematic diagram of the functional modules of a device for re-identifying pedestrians changing clothes provided in one embodiment of the present application;

[0046] Figure 7 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0048] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0049] Existing methods for re-identifying pedestrians after changing clothes have the following limitations: feature coupling problem: clothing and body features in appearance features are not effectively decoupled, resulting in feature drift after changing clothes; modal fusion rigidity: fixed weight fusion cannot adapt to the dynamic changes in feature importance in different clothing-changing scenarios; insufficient utilization of spatiotemporal information: gait feature extraction only relies on traditional spatiotemporal networks, which does not fully model the joint motion trajectory within the gait cycle.

[0050] This application proposes a three-level framework of dynamic feature decoupling, enhancement and fusion:

[0051] Appearance feature decoupling: Through the semantic segmentation-guided feature separation module, the appearance features are decomposed into clothing-independent body structure features (such as body shape and joint position) and clothing variable features, suppressing the interference of changing clothes.

[0052] Gait feature enhancement: A spatiotemporal graph convolutional network (ST-GCN) is introduced to model the motion trajectory of joints within the gait cycle, and the gait energy graph (GEI) is combined to generate a multimodal gait representation.

[0053] Adaptive fusion mechanism: A scene-aware gating network (SANet) is designed to dynamically adjust the fusion weights of appearance and gait features according to the degree of clothing change, achieving cross-modal feature complementarity.

[0054] In an exemplary embodiment, the present application provides a method for re-identifying a pedestrian who has changed clothes, such as Figure 1-Figure 3 As shown, the method for re-identifying pedestrians who change clothes includes steps 101 and 102.

[0055] Step 101: Decompose the appearance features of each pedestrian in the target video data into body structure features and clothing variable features.

[0056] Step 102: Based on the target video data, a trained spatiotemporal graph convolutional network is used to generate spatiotemporal dynamic features of joint point motion within the gait cycle, and a multimodal gait representation of each pedestrian is generated in combination with a gait energy graph; the gait energy graph is generated based on the target video data.

[0057] Step 103: Input the body structure features, clothing variable features, and multimodal gait representation of each pedestrian into the trained scene perception gating network to obtain the fused features of each pedestrian;

[0058] Step 104: Determine whether the pedestrians are the same person based on the fusion features of the pedestrians.

[0059] In an exemplary embodiment, Figure 4 As shown, step 101 specifically includes steps 201 to 203.

[0060] Step 201: Process the target video data to obtain a joint point coordinate sequence, a body structure region mask, and a clothing region mask of the pedestrian in each single frame image in the target video data.

[0061] Step 202: extract the global image features of each single-frame image, combine the corresponding body structure region mask and clothing region mask, and obtain the body structure region mask and clothing region mask of the pedestrian in each single-frame image.

[0062] Step 203: Using the trained two-stream network, extract body structure features and clothing variable features from the body structure region mask and clothing region mask of each single frame image.

[0063] In an exemplary embodiment, the target video data is processed to obtain a joint point coordinate sequence, a body structure region mask, and a clothing region mask of a pedestrian in each single-frame image in the target video data, specifically including: using a trained human posture estimation model to extract each preset joint point from each single-frame image, and each preset joint point constitutes a joint point coordinate sequence; using a trained semantic segmentation model to extract a body structure region mask and a clothing region mask from each single-frame image.

[0064] The target video data is the video data containing the clothes-changing scene, and the video data containing the clothes-changing scene is processed:

[0065] Sample single-frame images at a fixed frame rate and unify the resolution to 256×128.

[0066] A human pose estimation model (such as OpenPose) is used to extract the coordinate sequence of the pedestrian's joint points in each single frame image, and a semantic segmentation model (DeepLabv3+) is used to generate semantic area masks such as clothing and body.

[0067] The images are subjected to data augmentation (random flipping, cropping) and normalization to provide more sample images and improve the model training effect.

[0068] In an exemplary embodiment, extracting the global image features of each single-frame image and combining the corresponding body structure region mask and clothing region mask to obtain the body structure region mask and clothing region mask of the pedestrian in each single-frame image specifically includes:

[0069] ResNet50 is used as the backbone network to extract the global features of each single frame image;

[0070] The body structure region mask and clothing region mask are optimized through threshold processing and morphological operations to obtain the pedestrian's body structure region mask and clothing region mask in each single frame image.

[0071] In step 203, the body structure features include body shape and joint positions, and the clothing variable features include color, texture and style.

[0072] In an exemplary embodiment, the loss function used by the trained two-stream network during training is a contrast loss function.

[0073] The contrast loss function is expressed as:

[0074] Among them, L sep represents the contrast loss, L contrast () represents the contrast loss function, f1 and f2 represent two input variables, and the superscript T represents transposition. and Represents the body structure feature vector of the same pedestrian at different times; and represents the variable feature vector of clothing of the same pedestrian at different times, N is the total number of negative samples in the current batch, i represents the index of the positive sample currently being processed, corresponding to the i-th sample in the batch, and the value range is 1≤i≤B (B is the batch size); j represents the negative sample index, and if j≠i, the index of other pedestrians in the current positive sample pair is excluded; f 2,jrepresents the body structure feature vector or clothing variable feature vector of the jth negative sample; τ represents the temperature parameter, which is used to scale the probability distribution of feature vector similarity and has a value of 0.1.

[0075] In an exemplary embodiment, Figure 5 As shown, step 102 specifically includes steps 301 to 304.

[0076] Step 301: Construct a spatiotemporal graph based on the joint point coordinate sequence. The spatiotemporal graph establishes connections between the same joint points of adjacent frame images in the time dimension, and establishes connections between adjacent joint points in each single frame image according to the human skeletal structure in the spatial dimension.

[0077] Step 302: Use a trained spatiotemporal dynamic feature extraction spatiotemporal graph convolutional network to extract spatiotemporal dynamic features from the spatiotemporal graph; the trained spatiotemporal dynamic feature extraction spatiotemporal graph convolutional network is a structure in which two layers of graph convolution and two layers of time convolution are alternately stacked.

[0078] Step 303: Convert the target video data into a gait energy graph.

[0079] Step 304: splicing the spatiotemporal dynamic features and the gait energy map in the channel dimension to obtain the multimodal gait representation.

[0080] This application uses joint point coordinates to construct a spatiotemporal graph, models the joint point motion trajectory within the gait cycle through a spatiotemporal graph convolutional network, and extracts spatiotemporal dynamic features.

[0081] The target video data, i.e., video sequence, is converted into gait energy graph (GEI), and the spatiotemporal dynamic features are fused with the GEI static features to generate a multimodal gait representation containing temporal dynamics and global energy distribution.

[0082] Step 103 combines the body structure features, clothing changeability features, and multimodal gait feature representations, and inputs them into a fully connected network to extract scene features related to the degree of clothing change.

[0083] The gating unit generates the fusion weight of appearance and gait features according to the scene characteristics. When the degree of clothing change is high, the weight of gait features is enhanced, and vice versa, the weight of appearance features is increased to achieve feature complementarity.

[0084] In the training phase, the cross-entropy loss function is used to optimize the clothing-changing pedestrian re-identification model. In the testing phase, the pedestrian feature similarity is calculated by cosine similarity to complete the cross-camera clothing-changing pedestrian re-identification task.

[0085] The clothing-changing person re-identification model consists of a sequentially connected appearance feature decoupling module, a gait feature enhancement module, a scene-aware gating network, and a classifier. The appearance feature decoupling module implements step 101, the gait feature enhancement module implements step 102, and the scene-aware gating network and classifier implement steps 103 and 104, respectively.

[0086] In the test phase, "each pedestrian" is an independent detection target in a single frame, and the characteristics of each pedestrian are the fusion result of body and clothing features. In the test phase, the fusion features (including body structure and clothing features) are extracted for each pedestrian detected in the single-frame image, and the vector direction consistency of the pedestrian features under different cameras is calculated through cosine similarity to determine whether they are the same person. The main consideration is to balance feature discriminability (optimization in the training phase) and matching efficiency (lightweight calculation in the test phase). The pre-trained model is used to complete the feature matching forward propagation to extract features without updating parameters.

[0087] In an exemplary embodiment, a method for re-identifying pedestrians after changing clothes is implemented based on the deep learning framework PyTorch.

[0088] Step S1: Sample the video at a frame rate of 15 frames per second from the public person re-identification dataset to obtain a set of single-frame images. I i represents the i-th frame image, and N is the number of single-frame images. The resolution of all images is uniformly adjusted to 256×128 pixels to facilitate subsequent network processing.

[0089] The open source human pose estimation model OpenPose is used. Its network structure is based on a convolutional neural network (CNN) and is initialized by parameters pre-trained on the COCO dataset. i Input the OpenPose model, which can detect 18 key joints, i.e. preset joints, and obtain the joint point coordinate sequence. J i is the j-th joint point coordinate sequence, j i,k is the kth joint point coordinate in the jth joint point coordinate sequence, where each joint point coordinate j i,k =(x i,k ,y i,k ), x i,k and y i,k Respectively represent the horizontal and vertical coordinates of the joint point in the image. The preset joint points include shoulder joint, elbow joint and hip joint.

[0090] The semantic segmentation model DeepLabv3+ is used, which uses ResNet50 as the backbone network and is pre-trained on the Cityscapes dataset. iInput DeepLabv3+ and output the mask S of different semantic areas such as clothing, skin, hair, etc. i , mask S i is the same as image I i Multi-channel images of the same size, each channel corresponds to a semantic category.

[0091] For image I i Perform data enhancement operations such as random flipping and cropping, and normalize the image to the [0,1] interval to facilitate subsequent network processing.

[0092] Step S2: Input the preprocessed image into the appearance feature decoupling module, which consists of a semantic segmentation guidance branch and a feature separation branch. Specifically, it includes:

[0093] Step S21: Extract image I using ResNet50 as the backbone network i The global feature F global ,ResNet50 contains 5 residual blocks, and the output feature map size of the network is 8×4×2048. During the training process, random weight initialization is adopted, and batch normalization technology is used to reduce internal covariate shift and improve the stability of network training.

[0094] Step S22: The mask S obtained by semantic segmentation i , optimized through threshold processing and morphological operations to obtain a more accurate body structure area mask M body and clothing area mask M cloth Among them, M body and M cloth is related to the global feature F global The corresponding binary mask, the area with value 1 indicates that the corresponding feature is valid, and 0 indicates that it is invalid.

[0095] Step S23: Two-stream network structure of feature separation branch, both branches are composed of 3 convolutional layers and 2 fully connected layers. global With M body Perform element-by-element multiplication to obtain the body structure region feature F body , and then pass through the convolution layer to further extract clothing-independent body structure features such as body shape and joint position. The convolution kernel sizes of the convolution layer are 3×3, 3×, and 3×3 respectively, with a step size of 1 and a padding of 1 to keep the feature map size unchanged. Clothing variable feature extraction branch F global With M cloth Perform operations to extract variable features such as color, texture, and style of clothing. During the training process, in order to encourage the two streams to focus on the corresponding features, a contrast loss function L is designed. sep :

[0096]

[0097] Among them, the contrast loss function L contrast The calculation is as follows:

[0098]

[0099] Where f1 and f2 are eigenvectors, τ is the temperature parameter, which is set to 0.1. sep , achieving effective decoupling of appearance features.

[0100] Step S3: Process the joint point coordinate sequence in the video sequence to generate a multimodal gait representation. Specifically including:

[0101] Step S31: According to the joint point coordinate sequence J i Construct a spatiotemporal graph G = (V, E), where the node V = {v i,k The edges E represent the connections between the joints in time and space. In the temporal dimension, connections are established between the same joints in adjacent frames; in the spatial dimension, connections are established between adjacent joints based on the human skeletal structure.

[0102] Step S32: Spatiotemporal dynamic feature extraction Spatiotemporal graph convolutional network (ST-GCN) adopts a structure of alternating stacking of two layers of graph convolution (GCN) and two layers of temporal convolution (TCN). The specific network architecture is as follows:

[0103] When training the network, the initial learning rate is set to 1×10 -3 , an exponential decay strategy is adopted, the loss function is the weighted sum of contrast loss and cross entropy loss, and the optimizer is AdamW.

[0104] Define a spatiotemporal graph G = (V, E), where the node V corresponds to 18 preset joints (such as shoulder joints, elbow joints, hip joints, etc.) and E is the set of edges between each node. The same joints in adjacent frames in the time dimension are connected by temporal edges, and spatial edges are constructed based on the natural connection relationship of the human skeleton in the spatial dimension (such as shoulder joints connecting elbow joints, hip joints connecting knee joints). Joint point coordinate sequence (T is the number of frames, M is the number of joint points, and 2 is the coordinate dimension), converted to the adjacency matrix of the space-time graph The non-zero elements represent the connection relationship between nodes. represents the set of real numbers, The superscript of is the dimension of the set of real numbers.

[0105] Each layer of graph convolution aggregates node features through a learnable weight matrix W, the formula is:

[0106]

[0107] Among them, X' is the aggregation feature, is the normalized adjacency matrix, is the adjacency matrix with self-loops added, is the degree matrix, X is the input feature, and σ is the ReLU activation function.

[0108] The temporal convolution layer uses a one-dimensional convolution kernel with a kernel size of 3 and a step size of 1 to model the time series features of each joint point and capture the motion patterns within the gait cycle. The spatiotemporal convolution is performed alternately to ultimately output the spatiotemporal dynamic features of the gait. (C is the number of feature channels).

[0109] Step S33: Convert the video sequence into a gait energy image GEI. Each pixel value GEI(x,y) of GEI is calculated as shown in the formula:

[0110]

[0111] Where T is the number of video sequence frames, I t (x, y) t-th frame image. In this embodiment, a complete gait cycle is selected as the value T. To enhance the contrast of GEI, the calculated GEI is normalized so that its pixel value range is [0, 255].

[0112] Step S34: The dynamic features F extracted by ST-GCN st The static features of GEI are combined and fused, and GEI is first converted into F st The feature vectors of the same dimension are then concatenated in the channel dimension to obtain the multimodal gait representation F gait .

[0113] Step S4: The scene-aware gating network (SANet) consists of a scene feature extraction subnetwork and a gating weight generation subnetwork. The specific structure is as follows:

[0114] Scene feature extraction input is body structure features Variable features of clothing and multimodal gait features First, the splicing feature F is obtained by splicing operation concat :

[0115]

[0116] The scene feature extraction sub-network consists of three fully connected layers:

[0117] The first fully connected layer: input dimension D1+D2+D3(F concat), the output dimension is 1024, the activation function is LeakyReLU, and a Dropout layer is added to prevent overfitting.

[0118] The second fully connected layer has an input dimension of 1024, an output dimension of 512, and an activation function of Swish.

[0119] The third fully connected layer: input dimension 512, output dimension 256, generates scene features This feature encodes the joint information of the degree of pedestrian appearance change and gait stability.

[0120] The fusion weight generation gate weight generation sub-network contains 1 fully connected layer and Softmax activation function:

[0121] Among them, the fully connected layer: input dimension 256, output dimension 2, weight matrix Bias Calculate the gate score

[0122] Softmax function: Normalize the gated score to the fusion weights α, β, satisfying α + β = 1. The formula is:

[0123]

[0124] Where α is the weight of the appearance feature, and β is the weight of the gait feature. When the pedestrian changes clothes frequently (e.g., clothing color or style changes significantly), β automatically increases, increasing reliance on stable gait features. When the pedestrian changes clothes less frequently, α increases, fully utilizing the body structure information in the appearance features.

[0125] Feature fusion appearance features and gait characteristics F gait The final fusion feature is obtained by weighted summation:

[0126] F final =α·F appearance +β·F gait .

[0127] Step S5: Perform identification and classification, and combine the fusion feature F final Input to the classification layer (classifier), the classification layer uses a fully connected layer, the number of neurons is the number of pedestrian identity categories in the dataset. The cross entropy loss function L is used. ce For training, the calculation is as shown in the formula:

[0128] Where C is the number of pedestrian categories, y c is the true label, p cTo predict the class probability. During the training process, the batch size is set to 32, and a total of 200 training batches (epochs) are trained. In the test phase, according to the fusion feature F final The cosine similarity between pedestrians is calculated, and pedestrians with similarity higher than the set threshold are judged as the same identity, thus achieving the task of re-identifying pedestrians who have changed clothes.

[0129] This application uses a semantic segmentation-guided feature separation module to effectively decouple clothing-independent body structure features and clothing variability features, reducing the interference of clothing changes on appearance features. It also uses a spatiotemporal graph convolutional network combined with a gait energy graph to generate a multimodal gait representation, enhancing the expressive power of gait features. It also uses a scene-aware gating network to achieve adaptive fusion of appearance and gait features, enabling the model to dynamically adjust feature weights based on the degree of clothing changes, achieving cross-modal feature complementarity. Experimental results show that this application has a significant performance improvement in the clothing-changing pedestrian re-identification task and can effectively address the clothing-changing challenges in real-world scenarios.

[0130] Based on the same inventive concept, embodiments of the present application also provide a device for re-identifying a person changing clothes, for implementing the aforementioned method for re-identifying a person changing clothes. The solution provided by this device is similar to the solution described in the aforementioned method. Therefore, the specific limitations of one or more embodiments of the device for re-identifying a person changing clothes provided below can be found in the aforementioned limitations of the method for re-identifying a person changing clothes, and will not be further elaborated here.

[0131] In an exemplary embodiment, Figure 6 As shown, a device for re-identifying pedestrians who change clothes is provided, comprising:

[0132] The appearance feature decoupling module is used to decompose the appearance features of each pedestrian in the target video data into body structure features and clothing variable features.

[0133] The gait feature enhancement module is used to generate spatiotemporal dynamic features of joint point motion within the gait cycle based on the target video data using a trained spatiotemporal graph convolutional network, and to generate a multimodal gait representation of each pedestrian in combination with a gait energy graph; the gait energy graph is generated based on the target video data.

[0134] The feature fusion module is used to input the body structure features, clothing variable features and multimodal gait representation of each pedestrian into the trained scene perception gating network to obtain the fused features of each pedestrian.

[0135] The recognition module is used to determine whether the pedestrians are the same person based on their fusion features.

[0136] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 7As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data for re-identification of pedestrians who have changed clothes. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for re-identifying pedestrians who have changed clothes is implemented.

[0137] Those skilled in the art will understand that Figure 7 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application and does not constitute a limitation on the computer device to which the solution of the present application is applied. A specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps of the above-mentioned method embodiments when executing the computer program.

[0138] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0139] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0140] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0141] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0142] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, data processing logic of programmable logic devices, and the like.

[0143] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0144] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for re-identifying pedestrians who have changed clothes, characterized in that: The method for re-identifying a pedestrian who has changed clothes comprises: Decompose the appearance features of each pedestrian in the target video data into body structure features and clothing variable features; Based on the target video data, a trained spatiotemporal graph convolutional network is used to generate spatiotemporal dynamic features of joint motion within the gait cycle, and a multimodal gait representation of each pedestrian is generated by combining the gait energy graph generated based on the target video data. The body structure features, clothing variable features, and multimodal gait representation of each pedestrian are input into the trained scene perception gating network to obtain the fused features of each pedestrian; Based on the fusion features of each pedestrian, it is determined whether the pedestrians are the same person.

2. The method for re-identifying pedestrians changing clothes according to claim 1, characterized in that: The appearance features of each pedestrian in the target video data are decomposed into body structure features and clothing variable features, including: Processing the target video data to obtain a joint point coordinate sequence, a body structure region mask, and a clothing region mask of the pedestrian in each single frame image in the target video data; Extract the global image features of each single frame image, combine them with the corresponding body structure region mask and clothing region mask, and obtain the pedestrian's body structure region mask and clothing region mask in each single frame image; The trained two-stream network is used to extract body structure features and clothing variable features from the body structure region mask and clothing region mask of each single frame image.

3. The method for re-identifying pedestrians changing clothes according to claim 2, characterized in that: The target video data is processed to obtain a joint point coordinate sequence, a body structure region mask, and a clothing region mask of the pedestrian in each single frame image in the target video data, specifically including: A trained human posture estimation model is used to extract each preset joint point from each single frame image, and each preset joint point constitutes a joint point coordinate sequence; The trained semantic segmentation model is used to extract body structure region masks and clothing region masks from each single frame image.

4. The method for re-identifying pedestrians changing clothes according to claim 2, characterized in that: Extract the global image features of each single frame image and combine them with the corresponding body structure region mask and clothing region mask to obtain the pedestrian's body structure region mask and clothing region mask in each single frame image. Specifically, it includes: ResNet50 is used as the backbone network to extract the global features of each single frame image; The body structure region mask and clothing region mask are optimized through threshold processing and morphological operations to obtain the pedestrian's body structure region mask and clothing region mask in each single frame image.

5. The method for re-identifying pedestrians changing clothes according to claim 2, characterized in that: The loss function used by the trained two-stream network during training is a contrast loss function; The contrast loss function is expressed as: Among them, L sep represents the contrast loss, L contrast () represents the contrast loss function, f1 and f2 represent two input variables, and the superscript T represents transposition. and Represents the body structure feature vector of the same pedestrian at different times; and represents the variable feature vector of clothing of the same pedestrian at different times, N is the total number of negative samples, i represents the index of the positive sample currently being processed, j represents the index of the negative sample, and f 2,j represents the body structure feature vector or clothing variable feature vector of the jth negative sample, and τ represents the temperature parameter.

6. The method for re-identifying pedestrians changing clothes according to claim 2, characterized in that: Based on the target video data, the trained spatiotemporal graph convolutional network is used to generate the spatiotemporal dynamic features of the joint motion within the gait cycle. Combined with the gait energy graph, a multimodal gait representation of each pedestrian is generated, including: Constructing a spatiotemporal graph based on the joint point coordinate sequence, wherein the spatiotemporal graph establishes connections between the same joint points in adjacent frames in the time dimension, and establishes connections between adjacent joint points in each single frame according to the human skeletal structure in the spatial dimension; A trained spatiotemporal dynamic feature extraction spatiotemporal graph convolutional network is used to extract spatiotemporal dynamic features from the spatiotemporal graph; the trained spatiotemporal dynamic feature extraction spatiotemporal graph convolutional network is a structure in which two layers of graph convolution and two layers of time convolution are alternately stacked. Convert target video data into gait energy graph; The spatiotemporal dynamic features and the gait energy map are spliced ​​in the channel dimension to obtain the multimodal gait representation.

7. A device for re-identifying pedestrians who change clothes, characterized in that: The clothing-changing pedestrian re-identification device applies the clothing-changing pedestrian re-identification method according to any one of claims 1 to 6, and the clothing-changing pedestrian re-identification device comprises: Appearance feature decoupling module, used to decompose the appearance features of each pedestrian in the target video data into body structure features and clothing variable features; A gait feature enhancement module is used to generate spatiotemporal dynamic features of joint motion within the gait cycle based on the target video data using a trained spatiotemporal graph convolutional network, and to generate a multimodal gait representation of each pedestrian in combination with a gait energy graph generated based on the target video data; The feature fusion module is used to input the body structure features, clothing variable features and multimodal gait representation of each pedestrian into the trained scene perception gating network to obtain the fused features of each pedestrian; The recognition module is used to determine whether the pedestrians are the same person based on their fusion features.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method for re-identifying a pedestrian who has changed clothes according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for re-identifying a pedestrian who has changed clothes according to any one of claims 1 to 6 is implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method for re-identifying a pedestrian who has changed clothes according to any one of claims 1 to 6 is implemented.

Images