Dressing pedestrian re-identification method based on image style migration and residual network

Abstract

The invention relates to the technical field of computer vision, in particular to a clothes changing pedestrian re-identification method based on image style migration and a residual network. Comprising the following steps: preprocessing a clothes changing pedestrian data set by using a clothes expansion model in combination with a label smoothing mode so as to expand the clothes changing pedestrian data set; creating a clothes changing pedestrian re-identification model for the expanded clothes changing pedestrian data set, increasing time dimension to extract space-time dynamic features, and introducing a multi-scale structure to construct class residual connection with scale levels; performing local feature extraction and global feature extraction on pedestrians through a joint attention module, and inputting to-be-queried pedestrian images in the clothes changing pedestrian data set into the clothes changing pedestrian re-recognition model to extract query features; and constraining the clothes changing pedestrian re-identification model through a joint loss function for training. According to the invention, style migration is carried out on the clothes part of the pedestrian to increase the diversity of clothes types, so that the clothes of the pedestrian have randomness, and the generalization ability of the model in a clothes changing scene is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method for re-identifying pedestrians who have changed clothes based on image style migration and a residual network. Background Art

[0002] To ensure urban safety and order, the establishment of video surveillance systems such as smart cities and intelligent transportation has become a current development trend and an urgent need. Person Re-identification (Re-ID) technology aims to solve the problem of pedestrian association and matching in cross-camera scenarios. It processes monitoring data through pattern recognition and computer vision technology to achieve automatic detection, tracking and identification without human intervention. Re-ID can be used as an important supplement to face recognition technology. It can continuously track pedestrians whose faces cannot be clearly photographed across cameras, improve the spatiotemporal continuity of data, and is widely used in human-computer interaction, smart home and other fields, showing broad application prospects. In practical applications, Re-ID, as a research direction of pedestrian intelligent cognition, faces many technical difficulties and objective factors. Pedestrian image data comes from different times, places, and devices. There are problems such as posture changes, background differences, lighting differences, pixel differences, occlusions, clothing changes, modality changes, and different perspectives, which bring huge challenges to pedestrian re-identification.

[0003] However, most of the current pedestrian re-identification methods focus too much on the texture and color features of clothes, and pay less attention to the biometric features of the non-clothing areas of pedestrians, resulting in a sharp drop in performance of many methods when pedestrians change clothes. Summary of the invention

[0004] In view of the shortcomings of the prior art mentioned above, the purpose of the present invention is to provide a method for re-identifying pedestrians who change clothes based on image style transfer and residual network, which solves the problem that excessive focus on clothing texture and color features and less attention to the biometric features of non-clothing areas of pedestrians leads to a sharp drop in performance of many methods when pedestrians change clothes.

[0005] To achieve the above objectives and other related objectives, the present invention provides a method for re-identifying a person who has changed clothes based on image style transfer and residual network, comprising:

[0006] S1. Use the clothing expansion model combined with the label smoothing method to preprocess the clothing-changing pedestrian dataset to expand the clothing-changing pedestrian dataset;

[0007] S2. Create a clothing-changing pedestrian re-identification model for the expanded clothing-changing pedestrian dataset, add the time dimension to extract spatiotemporal dynamic features, and introduce a multi-scale structure to construct a residual connection with scale levels;

[0008] S3, according to the clothing-changing pedestrian re-identification model, extract local features and global features of pedestrians through a joint attention module, and input the image of the pedestrian to be queried in the clothing-changing pedestrian dataset into the clothing-changing pedestrian re-identification model to extract query features;

[0009] S4. The clothing-changing pedestrian re-identification model is trained by constraining the joint loss function, and the extracted pedestrian features are retrieved and matched. The joint loss function is a combination of a clothing loss function and a pedestrian loss function.

[0010] In one embodiment of the present invention, the clothing-changing pedestrian re-identification model is a multi-scale ResNet503D re-identification module.

[0011] In one embodiment of the present invention, the step S1 of preprocessing the clothing-changing pedestrian dataset by using the clothing expansion model combined with a label smoothing method to expand the clothing-changing pedestrian dataset includes:

[0012] The clothing expansion model is used to preprocess the low-resolution data in the clothing-changing pedestrian dataset, that is, to use a generative adversarial network to constrain the style transfer range, and to obtain a new clothing style through style transfer from the images of the pedestrian clothing area in the clothing-changing pedestrian dataset to expand the clothing-changing pedestrian dataset.

[0013] In one embodiment of the present invention, the multi-scale ResNet503D re-identification module adds a time dimension without changing the main structure of ResNet50.

[0014] In one embodiment of the present invention, the step S2 creates a clothing-changing pedestrian re-identification model for the expanded clothing-changing pedestrian dataset, adds a time dimension to extract spatiotemporal dynamic features, and introduces a multi-scale structure to construct a residual connection with scale levels, including:

[0015] S21, expanding the ResNet50 network layer of the multi-scale ResNet503D re-identification module, and reducing the output weight of the convolutional layer through zero gamma initialization of the BN layer in the shallow network layer, simplifying the clothing-changing pedestrian re-identification model for initial training;

[0016] S22, introducing a multi-scale structure into the ResNet503D residual block of the multi-scale ResNet503D re-identification module, performing group convolution on the features input to the ResNet503D residual block, constructing a residual connection with scale levels to replace the original single 3×3×3 convolution kernel, and the results of the previous group of group convolutions and the next group of group inputs constitute a residual structure input to the next group of convolutions to obtain multiple equivalent feature scales;

[0017] S23. Based on the separability of the 3D convolution of the ResNet503D residual block, a pseudo 3D convolution form is adopted in the residual block, and the 3D convolution kernel 3×3×3 in the residual block is represented by a cascade of a 2D convolution kernel 1×3×3 in the spatial domain and a 1D convolution kernel 3×1×1 in the temporal domain. The spatial information of each frame is extracted by convolution in the spatial domain, and the temporal information between frames is extracted by convolution in the temporal domain. The output of the spatial domain convolution is used as the input of the temporal convolution. The k×k×k 3D convolution is asymmetrically decomposed into k 3 The number of parameters in the decomposed 1×k×k and k×k×1 convolutions is only k 2 +k parameters;

[0018] S24, the multiple equivalent feature scales obtained by the group convolution are fused and then upsampled to restore the feature map dimension, and the formula is:

[0019]

[0020] Where i∈{1,2,...,s}, s is the number of groups, T i is the time domain convolution, S i is the spatial domain convolution, x i is the i-th group input, y i is the output corresponding to the i-th group.

[0021] In one embodiment of the present invention, the joint loss function introduces a hyperparameter to adjust the loss function weight, CosFaceLoss is applied to the pedestrian dataset, and feature distinction is performed between pedestrian feature types in the pedestrian dataset through cosine angles.

[0022] In one embodiment of the present invention, the extraction of local features and global features of pedestrians by the joint attention module in step S3 includes:

[0023] S31, the joint attention module includes a potential attention module and a cross attention module;

[0024] S32, for the local feature extraction, the latent attention module uses a self-attention mechanism to extract latent features, captures rough non-pedestrian partial features based on the appearance similarity between different pixels of the clothing-changing pedestrian dataset, and adds a latent attention module after stage 1 of the multi-scale ResNet503D re-identification module. The multi-channel convolution of the latent attention module uses a residual-like structure to fuse and transfer features of different pixels;

[0025] S33, for the global feature extraction, the cross attention module is used to downsample the input features to obtain H, where H passes through three convolution modules to generate Q, K, and V respectively, where C′ is one eighth of C. For each pixel point μ in the feature map Q, the vector Q is obtained at the same position and different channel levels. μ , extract the vector in the same row and column as position μ from K Among them, the parameter of the i-th position is Ω i,u , i=[1,...,H+W-1], after multiplication we get:

[0026] After normalization, we get A∈[(H+W-1)×W×H], and for V∈[C×W×H], we get the vector set on the channel axis Multiply it with the obtained A and add it to the original input H to get the final output H′. The formula is as follows:

[0027]

[0028] The cross attention module connects each pixel position with the global pixel position to obtain the global feature, and after repeated calculations, we get:

[0029] H u ″=[f(A,u,θ)+1]·f(A′,u,θ)·H θ

[0030] H″←[f(A,u x ,θ y ,θ x ,θ y )·f(A′,u x ,u y ,u x ,θ y )+f(A,θ x ,u y ,θ x ,θ y )·f(A′,u x ,u y ,θ x ,u y )]·H θ

[0031] Among them, H″ is the output feature after repetition, (θ x ,θ y ) is the pixel position coordinate x randomly selected in the first cycle i ,(u x ,θ y ) is the same as x i The pixel position y coordinate of the same row i , (θ x ,u y ) is the same as xi The pixel position coordinate z in the same column i ,(u x ,u y ) is y i and z i The coordinates of the intersection points in the same row and column;

[0032] The cross attention module is looped three times to make the output features have spatiotemporal correlation, the potential attention module is connected to the cross attention module in a cascade form to fuse local features with global features, and the three-loop cross attention module is introduced into the second residual block and the fourth residual block of stage2 and stage3 of the multi-scale ResNet503D re-identification module, respectively.

[0033] The present invention also provides a clothing-changing pedestrian re-identification system based on image style transfer and residual network, comprising:

[0034] A preprocessing module is used to preprocess the clothing-changing pedestrian dataset using a clothing expansion model combined with a label smoothing method to expand the clothing-changing pedestrian dataset;

[0035] The first extraction module is used to create a clothing-changing pedestrian re-identification model for the expanded clothing-changing pedestrian dataset, add a time dimension to extract spatiotemporal dynamic features, and introduce a multi-scale structure to construct a residual connection with scale levels;

[0036] A second extraction module is used to extract local features and global features of pedestrians through a joint attention module according to the clothing-changing pedestrian re-identification model, and input the image of the pedestrian to be queried in the clothing-changing pedestrian dataset into the clothing-changing pedestrian re-identification model to extract query features;

[0037] The training module is used to train the clothing-changing pedestrian re-identification model by constraining the joint loss function and retrieve and match the extracted pedestrian features. The joint loss function is a combination of a clothing loss function and a pedestrian loss function.

[0038] As described above, the clothing-changing pedestrian re-identification method based on image style transfer and residual network of the present invention has the following beneficial effects:

[0039] (1) The clothing-changing pedestrian re-identification method based on image style transfer and residual network of the present invention can extract more robust and discriminative features suitable for clothing-changing scenarios. In addition, the present invention uses image style transfer as a data preprocessing method to transfer the style of pedestrian clothing to increase the diversity of clothing types, making pedestrian clothing random, thereby increasing the generalization ability of the model in clothing-changing scenarios.

[0040] (2) The clothing-changing pedestrian re-identification method based on image style transfer and residual network of the present invention uses ResNet503D as the backbone network, introduces the time dimension, and replaces the large convolution kernels by stacking to expand the receptive field. In the residual block, a pseudo three-dimensional convolution form is used to decouple the 3D convolution into an asymmetric form of spatial domain convolution and temporal domain convolution, which reduces the number of parameters and calculations of the model. The use of a residual-like structure increases the variety of the model's receptive field and improves the accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A flowchart of a method for re-identifying a person who has changed clothes based on image style transfer and a residual network is provided in an embodiment of the present application.

[0042] Figure 2 A multi-scale residual block structure diagram based on separable 3D convolution for a method for re-identifying a person changing clothes based on image style transfer and residual network provided in an embodiment of the present application.

[0043] Figure 3 A potential attention and cross-attention cascade structure diagram of a method for re-identifying a person changing clothes based on image style transfer and residual network provided in an embodiment of the present application.

[0044] Figure 4 A network structure diagram of a method for re-identifying a person after changing clothes based on image style transfer and residual network provided in an embodiment of the present application. DETAILED DESCRIPTION

[0045] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0046] This section will describe in detail the specific embodiments of the present invention. The preferred embodiments of the present invention are shown in the accompanying drawings. The purpose of the accompanying drawings is to supplement the description of the text part of the specification with graphics, so that people can intuitively and vividly understand each technical feature and the overall technical solution of the present invention, but it cannot be understood as a limitation on the scope of protection of the present invention.

[0047] In the description of the present invention, it should be noted that the orientations or positional relationships indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", etc. are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention; the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be the internal communication of two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0048] It should be noted that the illustrations provided in the following embodiments are only used to illustrate the basic concept of the present invention in a schematic manner, and thus the illustrations only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0049] See also Figure 1 , Figure 1 A flowchart of a method for re-identifying a person changing clothes based on image style transfer and residual network provided in an embodiment of the present application. The present invention provides a method for re-identifying a person changing clothes based on image style transfer and residual network, comprising:

[0050] Step S1: In order to solve the problem of pedestrians changing clothes, a clothing expansion model is introduced. The clothing expansion model is used in combination with a label smoothing method to preprocess the clothing-changing pedestrian dataset, so as to expand the clothing-changing pedestrian dataset and increase the diversity of pedestrian clothing types.

[0051] Step S2: create a clothing-changing pedestrian re-identification model for the expanded clothing-changing pedestrian dataset, add a time dimension to extract spatiotemporal dynamic features, and introduce a multi-scale structure to construct a residual connection with scale levels.

[0052] Step S3: according to the clothing-changing pedestrian re-identification model, local and global features of pedestrians are extracted through a joint attention module, and the image of the pedestrian to be queried in the clothing-changing pedestrian dataset is input into the clothing-changing pedestrian re-identification model to extract query features.

[0053] Step S4, training the clothing-changing pedestrian re-identification model by constraining the joint loss function, and retrieving and matching the extracted pedestrian features, wherein the joint loss function is a combination of a clothing loss function and a pedestrian loss function.

[0054] Specifically, in step S1, image style transfer is performed on the clothing-changing pedestrian dataset, a given pedestrian clothing style is transferred to the original pedestrian dataset, and a generative adversarial network is used to perform migration position constraints, so that the pedestrian clothing style is changed while retaining the pedestrian characteristics, the diversity of pedestrian clothing is increased, and the impact of clothing on re-identification is reduced.

[0055] In step S2, the time dimension is added to capture more dynamic features from the image dataset. The multi-scale ResNet503D network is used as the backbone network, the ResNet50 network layer is expanded to 3D form, and the 7×7×7 downsampling convolution of the backbone stage0 is replaced by three layers of 3×3×3 convolution layers. Downsampling is performed in the first layer of 3×3×3 convolution layers to reduce the amount of calculation and parameters. Stacked convolution with small convolution kernels can ensure that the network can still extract information from a larger range of input images. Continuous convolution increases the depth of the network and integrates three nonlinear activation layers to replace a single nonlinear activation layer, which increases the discrimination ability. A multi-scale structure is introduced in the residual block of ResNet503D, and the features of the input residual block are grouped and convolved. A residual connection with scale levels is constructed to replace the original single 3×3×3 convolution kernel. The results of the previous group of group convolutions and the next group of group inputs form a residual structure input to the next group of convolutions. Due to the combination effect, the equivalent receptive field increases, and multiple equivalent feature scales are obtained. Based on the separability of 3D convolution, a pseudo 3D convolution form is used in the residual block. The 3D convolution kernel 3×3×3 in the residual block is represented by a cascade of a 2D convolution kernel 1×3×3 in the spatial domain and a 1D convolution kernel 3×1×1 in the temporal domain. The spatial domain convolution extracts the spatial information of each frame, and the temporal domain convolution extracts the timing information between frames. The spatial domain convolution output is directly used as the input of the temporal convolution. By performing asymmetric decomposition of the 3D convolution, the k×k×k 3D convolution of the k 3 The number of parameters in the decomposed 1×k×k and k×k×1 convolutions is only k 2 +k parameters, significantly reducing the number of parameters. The features obtained by group convolution are fused and then upsampled to restore the feature map dimension. The formula is:

[0056]

[0057] Where i∈{1,2,...,s}, s is the number of groups, T i is the time domain convolution, S i is the spatial domain convolution, x i is the i-th group input, y i To correspond to the i-th group of outputs, the last residual block of the backbone network is replaced to ensure more effective feature outputs.

[0058] In step S3, according to the clothing-changing pedestrian re-identification module, local and global features of pedestrians are extracted through a joint attention module, and the image of the pedestrian to be queried in the clothing-changing pedestrian dataset is input into the clothing-changing pedestrian re-identification model to extract query features.

[0059] In step S4, the joint loss function is a combination of a clothing loss function and a pedestrian loss function, which constrains the clothing-changing pedestrian re-identification model to be trained and retrieves and matches the extracted pedestrian features.

[0060] Specifically, the local feature extraction and global feature extraction of pedestrians by the joint attention module in step S3 includes:

[0061] S31. The joint attention module includes a potential attention module and a cross attention module.

[0062] S32. For the local feature extraction, the latent attention module uses a self-attention mechanism to extract latent features, captures rough non-pedestrian partial features based on the appearance similarity between different pixels of the clothing-changing pedestrian dataset, and adds a latent attention module after stage 1 of the multi-scale ResNet503D re-identification module. The multi-channel convolution of the latent attention module uses a residual-like structure to fuse and transmit the features of different pixels.

[0063] S33, global features use stacked cross attention modules to downsample the input features to obtain H, where H passes through three convolution modules to generate Q, K, and V respectively, where C′ is one eighth of C. For each pixel point μ in the feature map Q, a vector Q can be obtained at the same position and different channel levels. μ , extract the vector in the same row and column as position μ from K The parameter of the i-th position is Ω i,u , i=[1,...,H+W-1], after multiplication we get:

[0064]

[0065] After normalization, we get A∈[(H+W-1)×W×H]; for V∈[C×W×H], we can also get a vector set on the channel axis Multiply it with the obtained A and add it to the original input H to get the final output H′. The formula is as follows:

[0066]

[0067] By repeating the cross attention module, each pixel position is connected to the global pixel position to obtain close context information and rich global feature information. After repetition, we get:

[0068] H u ″=[f(A,u,θ)+1]·f(A′,u,θ)·H θ

[0069] H″←[f(A,u x ,θ y ,θ x ,θ y )·f(A′,u x ,u y ,u x ,θ y )+f(A,θ x ,u y ,θ x ,θ y )·f(A′,u x ,u y ,θ x ,u y )]·H θ ,

[0070] Among them, H″ is the output feature after repetition, (θ x ,θ y ) is the pixel position coordinate x randomly selected in the first cycle i ,(u x ,θ y ) is the same as x i The pixel position y coordinate of the same row i , (θ x ,u y ) is the same as x i The pixel position coordinate z in the same column i ,(u x ,u y ) is y i and z i The intersection coordinates of the same row and column. In order to extract the time dimension information contained in the network, the attention module is cycled three times to make the features have spatiotemporal correlation. The potential module is connected to the cross module in a cascade form to fuse the local features with the global features. In the subsequent stage 2, a three-cycle cross attention module is introduced to strengthen the correlation of global features and increase the individual differences of the extracted pedestrian features.

[0071] See also Figure 2 , Figure 2A multi-scale residual block structure diagram based on separable 3D convolution for a method of re-identifying a person who has changed clothes based on image style transfer and residual network provided in an embodiment of the present application. During training, a multi-scale structure is introduced into the expanded 3D residual block, and the input features are divided into 4 groups from the channel dimension while ensuring that the total number of channels remains unchanged. A residual connection with a scale hierarchy is constructed to replace the original single 3×3×3 convolution kernel. The input feature map of the previous group is fused with another group of input feature maps and then input into the next group of convolution layers, forming a local residual structure in structure. Based on separable 3D convolution, a pseudo three-dimensional convolution form is used in the residual block, and the 3D convolution kernel 3×3×3 is represented by a cascade of a 2D convolution kernel 1×3×3 in the spatial domain and a 1D convolution kernel 3×1×1 in the temporal domain. Each group of spatial domain convolution downsampling reduces the feature map size to half of the original size. After all grouped convolution output features are fused, upsampling is performed to restore the feature map dimension, and the output can be obtained as follows:

[0072]

[0073] See also Figure 3 , Figure 3 This is a diagram of the cascade structure of potential attention and cross attention for a method of clothing-changing pedestrian re-identification based on image style transfer and residual network provided in an embodiment of the present application. A potential attention module is added after stage 1 of ResNet503D, and a residual-like structure is adopted for multi-channel convolution. Through a simple 1×1×1 convolution layer and the introduction of BN layer and ReLu activation layer, the direct connection channel is retained to fuse shallow features with deep features, and learn based on the appearance similarity between different pixels. The output of the potential attention module is directly input to the cross attention module, which is cascaded three times, and a three-cycle cross attention module is introduced in the subsequent stage 2 of ResNet503D to ensure that the deep network extracts more global features.

[0074] See also Figure 4 , Figure 4 This is a network structure diagram of a method for re-identifying pedestrians after changing clothes based on image style transfer and residual network provided in an embodiment of the present application. In order to retain rich features, the last layer of downsampling of ResNet503D is deleted, and the output feature maps are integrated and connected using global average pooling and global maximum pooling. The input image size is 256×128, and the preprocessing does not change the image size. The batch size is set to 32, and each batch contains 8 pedestrians and 8 pictures of each pedestrian. Adam trains the model for 300 epochs, and LCA is used for training after the 150th epoch. The initial learning rate is set to 0.00035, and the learning rate is divided by 10 every 20 epochs. The output results are evaluated periodically to promptly detect problems such as overfitting or underfitting of the model and make timely adjustments.

[0075] The improved network model was tested and evaluated, and its performance was verified by comparing it with the original network and other target detection networks. The results are as follows:

[0076]

[0077] Compared with the original network, the improved multi-scale ResNet50 re-identification module of the present invention improves the Top1 and mAP by 2.1% and 1.9% in the re-identification direction, and improves the Top1 and mAP by 1.9% and 2.5% in the clothing change re-identification direction. At the same time, compared with the two classic temporal information modeling methods of C2D, I3D and Non-Local, and the temporal information modeling method AP3D designed specifically for video re-identification, it has advantages in evaluation indicators.

[0078] Similar to the principle of the method for re-identifying a person changing clothes based on image style transfer and residual network of the present invention, the present invention also provides a system for re-identifying a person changing clothes based on image style transfer and residual network, comprising:

[0079] A preprocessing module is used to preprocess the clothing-changing pedestrian dataset using a clothing expansion model combined with a label smoothing method to expand the clothing-changing pedestrian dataset;

[0080] The first extraction module is used to create a clothing-changing pedestrian re-identification model for the expanded clothing-changing pedestrian dataset, add a time dimension to extract spatiotemporal dynamic features, and introduce a multi-scale structure to construct a residual connection with scale levels;

[0081] A second extraction module is used to extract local features and global features of pedestrians through a joint attention module according to the clothing-changing pedestrian re-identification model, and input the image of the pedestrian to be queried in the clothing-changing pedestrian dataset into the clothing-changing pedestrian re-identification model to extract query features;

[0082] The training module is used to train the clothing-changing pedestrian re-identification model by constraining the joint loss function and retrieve and match the extracted pedestrian features. The joint loss function is a combination of a clothing loss function and a pedestrian loss function.

[0083] In summary, the clothing-changing pedestrian re-identification method based on image style transfer and residual network of the present invention expands the diversity of pedestrian clothing types through preprocessing, introduces the time dimension, uses a separable 3D convolutional multi-scale residual structure to extract multi-equivalent scale features, and stacks cross-attention modules to enhance global pixel correlation, thereby improving the re-identification accuracy. In addition, the computational complexity and parameter amount of the model are further reduced by decoupling convolutional layers and indirect association.

[0084] The technical principle of the present invention is described above in conjunction with specific embodiments. These descriptions are only for explaining the principle of the present invention and cannot be interpreted as limiting the scope of protection of the present invention in any way. Based on the explanations herein, those skilled in the art can associate other specific embodiments of the present invention without creative work, and these equivalent variations or substitutions are all included in the scope defined by the claims of this application.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for re-identifying people with changed clothes based on image style transfer and residual network, characterized in that: include: S1. Use the clothing expansion model combined with the label smoothing method to preprocess the clothing-changing pedestrian dataset to expand the clothing-changing pedestrian dataset; S2. Create a clothing-changing pedestrian re-identification model for the expanded clothing-changing pedestrian dataset, add the time dimension to extract spatiotemporal dynamic features, and introduce a multi-scale structure to construct a residual connection with scale levels; S3, according to the clothing-changing pedestrian re-identification model, extract local features and global features of pedestrians through a joint attention module, and input the image of the pedestrian to be queried in the clothing-changing pedestrian dataset into the clothing-changing pedestrian re-identification model to extract query features; S4. The clothing-changing pedestrian re-identification model is trained by constraining the joint loss function, and the extracted pedestrian features are retrieved and matched. The joint loss function is a combination of a clothing loss function and a pedestrian loss function.

2. The method for re-identifying people after changing clothes based on image style transfer and residual network according to claim 1, characterized in that: The clothing-changing pedestrian re-identification model is a multi-scale ResNet503D re-identification module.

3. The method for re-identifying people with changed clothes based on image style transfer and residual network according to claim 1, characterized in that: The step S1 of preprocessing the clothing-changing pedestrian dataset by using the clothing expansion model combined with the label smoothing method to expand the clothing-changing pedestrian dataset includes: The clothing expansion model is used to preprocess the low-resolution data in the clothing-changing pedestrian dataset, that is, to use a generative adversarial network to constrain the style transfer range, and to obtain a new clothing style through style transfer from the images of the pedestrian clothing area in the clothing-changing pedestrian dataset to expand the clothing-changing pedestrian dataset.

4. The method for re-identifying people after changing clothes based on image style transfer and residual network according to claim 2, characterized in that: The multi-scale ResNet503D re-identification module adds the time dimension without changing the main structure of ResNet50.

5. The method for re-identifying people after changing clothes based on image style transfer and residual network according to claim 4 is characterized in that: In step S2, a clothing-changing pedestrian re-identification model is created for the expanded clothing-changing pedestrian dataset, the time dimension is added to extract spatiotemporal dynamic features, and a multi-scale structure is introduced to construct a residual connection with scale levels, including: S21, expanding the ResNet50 network layer of the multi-scale ResNet503D re-identification module, and reducing the output weight of the convolutional layer through zero gamma initialization of the BN layer in the shallow network layer, simplifying the clothing-changing pedestrian re-identification model for initial training; S22, introducing a multi-scale structure into the ResNet503D residual block of the multi-scale ResNet503D re-identification module, performing group convolution on the features input to the ResNet503D residual block, constructing a residual connection with scale levels to replace the original single 3×3×3 convolution kernel, and the results of the previous group of group convolutions and the next group of group inputs constitute a residual structure input to the next group of convolutions to obtain multiple equivalent feature scales; S23, based on the separability of the 3D convolution of the ResNet503D residual block, a pseudo 3D convolution form is adopted in the residual block, and the 3D convolution kernel 3×3×3 in the residual block is represented by a cascade of a 2D convolution kernel 1×3×3 in the spatial domain and a 1D convolution kernel 3×1×1 in the temporal domain, and the spatial information of each frame is extracted by convolution in the spatial domain, and the temporal information between frames is extracted by convolution in the temporal domain. The output of the spatial domain convolution is used as the input of the temporal convolution, and the k×k×k 3D convolution is asymmetric decomposed into k 3 The number of parameters in the decomposed 1×k×k and k×k×1 convolutions is only k 2 +k parameters; S24, the multiple equivalent feature scales obtained by the group convolution are fused and then upsampled to restore the feature map dimension, and the formula is: Where i∈{1,2,...,s}, s is the number of groups, T i is the time domain convolution, S i is the spatial domain convolution, x i is the i-th group input, y i is the output corresponding to the i-th group.

6. The method for re-identifying people with changed clothes based on image style transfer and residual network according to claim 1, characterized in that: The joint loss function introduces a hyperparameter to adjust the loss function weight, applies CosFaceLoss to the pedestrian dataset, and distinguishes features among the pedestrian feature types in the pedestrian dataset through cosine angles.

7. The method for re-identifying people after changing clothes based on image style transfer and residual network according to claim 5, characterized in that: The local feature extraction and global feature extraction of pedestrians by the joint attention module in step S3 includes: S31, the joint attention module includes a potential attention module and a cross attention module; S32, for the local feature extraction, the latent attention module uses a self-attention mechanism to extract latent features, captures rough non-pedestrian partial features based on the appearance similarity between different pixels of the clothing-changing pedestrian dataset, and adds a latent attention module after stage 1 of the multi-scale ResNet503D re-identification module. The multi-channel convolution of the latent attention module uses a residual-like structure to fuse and transfer features of different pixels; S33, for the global feature extraction, the cross attention module is used to downsample the input features to obtain H, where H passes through three convolution modules to generate Q, K, and V respectively, where C′ is one eighth of C. For each pixel point μ in the feature map Q, the vector Q is obtained at the same position and different channel levels. μ , extract the vector in the same row and column as position μ from K Among them, the parameter of the i-th position is Ω i,u , i=[1,...,H+W-1], after multiplication we get: After normalization, we get A∈[(H+W-1)×W×H], and for V∈[C×W×H], we get the vector set on the channel axis Multiply it with the obtained A and add it to the original input H to get the final output H′. The formula is as follows: The cross attention module connects each pixel position with the global pixel position to obtain the global feature, and after repeated calculations, we get: H u ″=[f(A,u,θ)+1]·f(A′,u,θ)·H θ H″←[f(A,u x ,i y ,i x ,i y )·f(A′,u x ,u y ,u x ,i y )+f(A,θ x ,u y ,i x ,i y )·f(A′,u x ,u y ,i x ,u y )]·H θ Among them, H″ is the output feature after repetition, (θ x ,θ y ) is the pixel position coordinate x randomly selected in the first cycle i ,(u x ,θ y ) is the same as x i The pixel position y coordinate of the same row i , (θ x ,u y ) is the same as x i The pixel position coordinate z in the same column i ,(u x ,u y ) is y i and z i The coordinates of the intersection points of the same row and column; The cross attention module is looped three times to make the output features have spatiotemporal correlation, the potential attention module is connected to the cross attention module in a cascade form to fuse local features with global features, and the three-loop cross attention module is introduced into the second residual block and the fourth residual block of stage2 and stage3 of the multi-scale ResNet503D re-identification module, respectively.

8. A clothing-changing pedestrian re-identification system based on image style transfer and residual network, characterized in that: include: A preprocessing module is used to preprocess the clothing-changing pedestrian dataset using a clothing expansion model combined with a label smoothing method to expand the clothing-changing pedestrian dataset; The first extraction module is used to create a clothing-changing pedestrian re-identification model for the expanded clothing-changing pedestrian dataset, add a time dimension to extract spatiotemporal dynamic features, and introduce a multi-scale structure to construct a residual connection with scale levels; A second extraction module is used to extract local features and global features of pedestrians through a joint attention module according to the clothing-changing pedestrian re-identification model, and input the image of the pedestrian to be queried in the clothing-changing pedestrian dataset into the clothing-changing pedestrian re-identification model to extract query features; The training module is used to train the clothing-changing pedestrian re-identification model by constraining the joint loss function and retrieve and match the extracted pedestrian features. The joint loss function is a combination of a clothing loss function and a pedestrian loss function.

Images