Clothing uniformity and diversity fusion based clothes-changing pedestrian re-identification method
Patent Information
- Application Number
- CN202311311592.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-10-11
- Publication Date
- 2026-11-03
- Estimated Expiration
- 2043-10-11
AI Technical Summary
Existing pedestrian re-identification technologies that rely on clothing color and texture features result in a significant decrease in recognition effectiveness when pedestrians change clothes, making it difficult to effectively identify pedestrians changing clothes.
A method based on the fusion of uniformity and diversity of clothing is adopted. The uniformity and diversity features of clothing are obtained through a semantic segmentation model, and non-clothing features are extracted by combining the ResNet50 network. The model is trained by a rich-scale attention mechanism and a joint loss function to mask clothing information and enhance non-clothing features for pedestrian re-identification.
In the pedestrian re-identification dataset for changing clothes, we achieved efficient recognition that does not depend on the color and texture of clothing, which improved the recognition accuracy and robustness and reduced the interference of background and clothing features on the recognition.
Smart Images
Figure CN117333901B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and deep learning technology, and in particular to a method for re-identifying pedestrians changing clothes based on the fusion of uniform and diverse clothing styles. Background Technology
[0002] Pedestrian re-identification is a computer vision task whose main goal is to identify and re-identify the same pedestrian in different cameras or scenes. It is an image retrieval problem. When the same pedestrian appears in different cameras or scenes, the system needs to re-identify that pedestrian. This typically involves searching a database for features similar to the current pedestrian's characteristics and determining their identity. Pedestrian re-identification currently plays a crucial role in smart cities, intelligent security, and human-computer interaction. It can be used to monitor pedestrian flow within an area, track criminal suspects, and improve traffic management. In recent years, with the development of deep learning, the pedestrian re-identification problem has attracted even more attention. In summary, pedestrian re-identification aims to determine the identity of pedestrians by comparing their physical features, overcoming the differences between pedestrian images captured in different scenes and at different times, to meet the needs of pedestrian identity verification and tracking in various applications.
[0003] Pedestrian re-identification in clothing-changing scenarios arose as researchers discovered that criminal suspects often alter their clothes, such as changing their pants, or wearing masks and hats, to evade surveillance cameras after committing a crime. This significantly reduced the performance of previous pedestrian re-identification algorithms in the clothing-changing domain, leading to increased attention to the problem. Researchers focused on extracting non-clothing human features, employing methods such as altering pedestrian clothing, using GANs to modify clothing and extract non-clothing features, and utilizing advanced features like 3D and contour analysis for feature extraction to facilitate pedestrian re-identification in clothing-changing scenarios.
[0004] Currently, researchers in the field of pedestrian re-identification after clothing changes mainly focus on non-clothing features such as contours, gait, and 3D models. However, these features often heavily rely on models that generate 3D, contour, and gait information. With the increasing sophistication of segmentation networks like Mask R-CNN and pose estimation networks like Dense Pose, accurate segmentation masks of pedestrian body regions can be obtained, and even more fine-grained regions of each individual body part can be precisely obtained. This provides new insights for solving the problem of pedestrian re-identification after clothing changes.
[0005] Therefore, in order to address the above problems, this invention proposes a method for re-identifying pedestrians changing clothes based on the fusion of uniform and diverse clothing styles. Summary of the Invention
[0006] This invention addresses the shortcomings of existing technologies by proposing a pedestrian re-identification method based on the fusion of uniform and diverse clothing. This invention enables the model to identify pedestrians without relying on appearance features such as clothing color and texture, achieving excellent results in relevant pedestrian re-identification datasets.
[0007] The technical solution to the problem solved by this invention is as follows: This invention provides a method for re-identifying pedestrians changing clothes based on the fusion of uniform and diverse clothing, including the following steps:
[0008] S1. Obtain the human body semantic segmentation map through the semantic segmentation model to obtain images of clothing consistency and diversity. Use these as network inputs and train them to obtain clothing consistency features, clothing diversity features, and original image features.
[0009] S2. Use the designed rich-scale attention mechanism to extract non-clothing human body features:
[0010] The features of the clothing consistency image extracted from S1 are divided into four parts. The first part is convolved and then added to the second part. The second part is convolved and then added to the third part. The third part is convolved and then added to the fourth part. The channel features of the feature map are reweighted by weighted feature vectors to make the model pay more attention to the human body features in the non-clothing area. Then, this feature is passed through a max pooling layer to obtain the non-clothing human body features. The non-clothing human body features are divided into two parts: one part is the upper body non-clothing human body features that are retained, and the other part is the lower body non-clothing human body features that are discarded.
[0011] S3. The original image features extracted using the ResNet50 residual network model are fused with the non-clothing features of the upper body:
[0012] The original image is fed into the ResNet50 model for feature extraction. By fusing the extracted features with the upper body non-clothing features, the upper body region in the generalized features of the original image is weakened, so that the non-clothing human body features are inversely enhanced. Then, the inversely enhanced non-clothing human body features are passed through a fully connected layer for classification loss calculation.
[0013] S4. Training the network by constraining the joint loss function:
[0014] Three links are set up in the network. The first link learns human features independent of clothing by using a parameter-independent ResNet50 model. The second link learns global features of the original image for pedestrian retrieval. The third link learns features weakened by clothing and background. Then, joint training is performed using a loss function, which is divided into four parts: triplet loss, classification loss, mean squared error loss, and adaptive Euclidean distance loss. After calculation, the four parts of the loss are added together for joint training to obtain more accurate global features for pedestrian similarity matching.
[0015] S5. Use the trained model to extract features for retrieval and matching:
[0016] The image to be retrieved is matched one by one with each pedestrian image in the retrieval database to obtain similarity; the pedestrian images in the retrieval database are arranged in descending order of similarity, and the pedestrian image with the highest similarity is taken as the recognition result corresponding to the pedestrian image to be processed.
[0017] In S1, a human semantic segmentation algorithm is used to pre-obtain 18 human body parts from the corresponding image. These 18 parts are fused and divided into clothing and non-clothing. Only the clothing area is blacked out. The clothing area is located using the fused label, and a corresponding binary mask is generated. The mask is then multiplied element-wise with the original image to mask the clothing information, thus obtaining an image with the clothing area masked, resulting in a clothing consistency image. The clothing area is randomly coded, and the background area is set to 0, resulting in a clothing diversity image. The clothing consistency image, the original image, and the clothing diversity image are input into the backbone network. The backbone network parameters for the original image and the clothing diversity image are shared, while the backbone network parameters for the clothing consistency image are unique, resulting in three global features: clothing consistency image features, original image features, and clothing diversity features.
[0018] The specific formulas used in S1 are as follows:
[0019] F a =ResNet50(A)
[0020] F b =ResNet50(B)
[0021] F c =ResNet50(C)
[0022] Where A is an image with consistent clothing, B is the original image, C is an image with diverse clothing, ResNet is the feature extraction backbone network, and F... a It is a characteristic of consistent clothing images, F b It is a feature of the original image, F c It is a characteristic of clothing diversity.
[0023] The specific formulas used in S2 are as follows:
[0024]
[0025]
[0026]
[0027] f w =concat(f1,f′2,f′3,f′4)
[0028] F w =mp(F2(F1(avg(f w ))))
[0029] In this context, convolutions conv1, conv2, and conv3 are all convolution operations with a 3x3 kernel and 1 unit of padding; avg is average pooling, F1 is weighted de-pooling, F2 is weighted pooling, and mp is max pooling with a 12x8 kernel. It's an addition; `concat` concatenates features along the channel dimension. `f1`, `f2`, `f3`, and `f4` are the features F of the clothing consistency image. a Image features after being divided into four equal parts horizontally.
[0030] The specific formulas used in S3 are as follows:
[0031]
[0032] Among them, F′ b It is a characteristic after fusion, F w It is an upper body feature that is not covered by clothing, F b These are features of the original image. It is addition. It is the outer product.
[0033] The specific formulas used in S4 are as follows:
[0034] L total =L global +L id +L ems +F sc
[0035] Among them, L total It's a loss of the entire network, L id It is classification loss, L global It is the ternary loss used as a measure of loss, L ems It is used as an adaptive Euclidean distance loss, F sc It is the feature F of the original image band clothing diversity characteristics F c Sum after calculating the mean squared error loss;
[0036]
[0037] Where b is the size of the training batch, and p() is the input object I. i The predicted probability, y i It is its corresponding pedestrian category label;
[0038]
[0039] Among them, f a f p f n These are the features extracted from the anchor, positive, and negative images, respectively. Here, positive and negative samples refer to pedestrians with the same and different identities as the anchor image, respectively; α is the boundary hyperparameter, max is the maximum value, min is the minimum value, P is the number of classes in each training batch, and K is the number of images per class. + This means that when the value in brackets [] is greater than 0, that value is taken as the loss; when it is less than 0, the loss is 0.
[0040]
[0041] Where, x 1i and x 2i The two are the upper body features (excluding clothing) F. w and clothing diversity F c The feature is that θ is a dynamic threshold and N is the number of sample pairs;
[0042]
[0043] L sem The mean squared error loss is given by f(x), b is the batch size, ||*||² represents normalization, and F is the mean squared error loss. b It is a feature of the original image, F a It is a characteristic of consistent clothing images, F c It is a characteristic of clothing diversity.
[0044] The specific formulas used in S5 are as follows:
[0045]
[0046] Where I1 and I2 are two input images, The Euclidean distance between the two images is... and These are the feature vectors corresponding to them obtained through forward propagation of the network.
[0047] The above technical solution has the following advantages or beneficial effects:
[0048] 1) To address the problem of pedestrians changing clothes, a clothing consistency network is proposed to learn generalized pedestrian appearance features, so that the model does not rely on appearance features such as clothing color and texture when identifying pedestrians.
[0049] 2) The original image features extracted by the ResNet50 network model are fused with the non-clothing features of the upper body, which increases the diversity of the samples and weakens the background features.
[0050] 3) By reasonably integrating clothing consistency and diversity, the model can focus more on foreground information that is not related to clothing with greater accuracy.
[0051] 4) The advanced semantic human body re-identification method based on clothing shielding network proposed in this paper has achieved excellent results in relevant human body re-identification datasets. Attached Figure Description
[0052] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.
[0053] Figure 1 This is a flowchart of the present invention.
[0054] Figure 2 The network framework diagram of the pedestrian re-identification method based on clothing uniformity and diversity fusion designed for this invention is shown.
[0055] Figure 3 Example of helping pedestrians change clothes.
[0056] Figure 4 The results are the search results for this invention on the PRCC dataset. Detailed Implementation
[0057] To make the objectives, technical solutions, and advantages of this invention clearer and more understandable, the invention will be further described below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.
[0058] Example 1
[0059] like Figures 1 to 4 As shown, this invention provides a method for re-identifying pedestrians changing clothes based on the fusion of uniform and diverse clothing styles, including the following steps:
[0060] S1. Obtain the human body semantic segmentation map through the semantic segmentation model to obtain images of clothing consistency and diversity. Use these as network inputs and train them to obtain clothing consistency features, clothing diversity features, and original image features.
[0061] S2. Use the designed rich-scale attention mechanism to extract non-clothing human body features:
[0062] The features of the clothing consistency image extracted from S1 are divided into four parts. The first part is convolved and then added to the second part. The second part is convolved and then added to the third part. The third part is convolved and then added to the fourth part. The channel features of the feature map are reweighted by weighted feature vectors to make the model pay more attention to the human body features in the non-clothing area. Then, this feature is passed through a max pooling layer to obtain the non-clothing human body features. The non-clothing human body features are divided into two parts: one part is the upper body non-clothing human body features that are retained, and the other part is the lower body non-clothing human body features that are discarded.
[0063] S3. The original image features extracted using the ResNet50 residual network model are fused with the non-clothing features of the upper body:
[0064] The original image is fed into the ResNet50 model for feature extraction. By fusing the extracted features with the upper body non-clothing features, the upper body region in the generalized features of the original image is weakened, so that the non-clothing human body features are inversely enhanced. Then, the inversely enhanced non-clothing human body features are passed through a fully connected layer for classification loss calculation.
[0065] S4. Training the network by constraining the joint loss function:
[0066] Three links are set up in the network. The first link learns human features independent of clothing by using a parameter-independent ResNet50 model. The second link learns global features of the original image for pedestrian retrieval. The third link learns features weakened by clothing and background. Then, joint training is performed using a loss function, which is divided into four parts: triplet loss, classification loss, mean squared error loss, and adaptive Euclidean distance loss. After calculation, the four parts of the loss are added together for joint training to obtain more accurate global features for pedestrian similarity matching.
[0067] S5. Use the trained model to extract features for retrieval and matching:
[0068] The image to be retrieved is matched one by one with each pedestrian image in the retrieval database to obtain similarity; the pedestrian images in the retrieval database are arranged in descending order of similarity, and the pedestrian image with the highest similarity is taken as the recognition result corresponding to the pedestrian image to be processed.
[0069] Specifically, in S1, the semantic segmentation algorithm commonly used in human pose estimation is used to obtain the semantic segmentation image of the corresponding part of the image in advance. After obtaining the semantic segmentation image, the upper body information of the image is masked and blacked out to obtain the clothing consistency image; the upper body is randomly masked and the background is uniformly masked to obtain the clothing diversity image. This is used as the network input, and after training, the clothing consistency feature, clothing diversity feature, and original image feature are obtained.
[0070] By utilizing a human semantic segmentation algorithm, 18 human body parts were pre-obtained from the corresponding image. These 18 human body parts are: "background", "hat", "hair", "sunglasses", "top", "skirt", "pants", "dress", "belt", "left foot", "right foot", "face", "left leg", "right leg", "left arm", "right arm", "bag", and "scarf". After fusing these 18 parts, they were categorized into: clothing, background, and others. The clothing part was first masked, while the rest remained unchanged. The clothing region was located using the fused labels, and a corresponding binary mask was generated. This mask was then multiplied element-wise with the original image. The process involves masking clothing information to obtain an image where the clothing area is obscured, resulting in a clothing consistency image. Similarly, clothing areas can be randomly masked, and background areas can be set to 0 to obtain a clothing diversity image. The clothing consistency image, the original image, and the clothing diversity image are input into a backbone network. The backbone network parameters for the original image and the clothing diversity image are shared, while the parameters for the clothing consistency image are unique. Three global features are obtained: clothing consistency image features, original image features, and clothing diversity features. The backbone network uses a ResNet50 network model, pre-trained on the ImageNet image dataset.
[0071] The specific formulas used in S1 are as follows:
[0072] F a =ResNet50(A)
[0073] F b =ResNet50(B)
[0074] F c =ResNet50(C)
[0075] Where A is an image with consistent clothing, B is the original image, C is an image with diverse clothing, ResNet is the feature extraction backbone network, and F... a It is a characteristic of consistent clothing images, F b It is a feature of the original image, F c It is a characteristic of clothing diversity.
[0076] The specific formulas used in S2 are as follows:
[0077]
[0078]
[0079]
[0080] f w =concat(f1,f′2,f′3,f′4)
[0081] F w =mp(F2(F1(avg(f w ))))
[0082] In this context, convolutions conv1, conv2, and conv3 are all convolution operations with a 3x3 kernel and 1 padding; avg is average pooling, F1 is a weighted de-weighting operation, F2 is a weighted operation, mp is max pooling with a 12x8 kernel, ⊕ is addition, concat concatenates features along the channel dimension, and f1, f2, f3, and f4 are F-values of the clothing consistency image. a Image features after being divided into four equal parts horizontally.
[0083] The specific formulas used in S3 are as follows:
[0084]
[0085] Among them, F′ b It is a characteristic after fusion, F w It is an upper body feature that is not covered by clothing, F b These are features of the original image. It is addition. It is the outer product.
[0086] S4 specifically trains the network by constraining it with a joint loss function. The network has three links: the first uses a parameter-independent ResNet50 model to learn human features independent of clothing; the second learns global features from the original image; and the third learns features weakened by clothing and background. This invention uses the global features from the second link for pedestrian retrieval. The loss function consists of four parts: one part uses the global features extracted by ResNet50 as a triplet loss, which serves as a metric loss to shorten intra-class feature distances and widen inter-class feature distances; another part uses the features after applying a fully connected layer to the global features as a classification loss; and a third part is the mean squared error loss: comparing the original image with the clothing... The first part uses a mean squared error loss to calculate the uniformity and diversity of clothing features. The second part uses an adaptive Euclidean distance loss: using the upper body non-clothing features and clothing diversity features, an adaptive Euclidean distance loss is calculated to adaptively narrow the gap between the two features. The three global features and the upper body non-clothing features are then passed through a fully connected layer to obtain classification features, which are used to calculate the classification loss. To reduce the error of the SCHP segmentation model in blackening the background region, the upper body non-clothing features and global features are calculated using an adaptive Euclidean distance loss function. The four losses are summed and jointly trained to obtain more accurate global features for pedestrian similarity matching.
[0087] This invention designs an adaptive Euclidean distance loss function. This loss function makes the Euclidean distance (minus a dynamic threshold) between the feature vectors of each pair of samples greater than or equal to zero, then squares it, and averages the losses of all sample pairs. This loss function encourages the model to make the distance between similar samples less than the dynamic threshold, while keeping the distance between dissimilar samples greater than or equal to zero. This reduces the noise introduced by SCHP segmentation of the background region. Through backpropagation and optimization, the model can learn to adapt to the dynamic threshold for different data distributions and tasks to minimize this loss function.
[0088] Simultaneously, mean squared error loss is used to bring the source image features and the black clothing features closer together, thereby weakening the clothing features in the source image features and compensating for the noise brought by the SCHP semantic segmentation model.
[0089] By training, the spatial distance between two features is minimized as much as possible, so that the effective semantic information before and after the clothing change remains consistent; finally, the four parts of the loss are weighted and summed to allow the model to learn more accurate non-clothing features of the human body.
[0090] The specific formulas used in S4 are as follows:
[0091] L total =L global +L id +Lems +F sc
[0092] Among them, L total It's a loss of the entire network, L id It is classification loss, L global It is the ternary loss used as a measure of loss, L ems It is used as an adaptive Euclidean distance loss, F sc It is the feature F of the original image b and clothing diversity characteristics F c Sum after calculating the mean squared error loss;
[0093]
[0094] Where b is the size of the training batch, and p() is the input object I. i The predicted probability, y i It is its corresponding pedestrian category label;
[0095]
[0096] Among them, f a f p f n These are the features extracted from the anchor, positive, and negative images, respectively. Here, positive and negative samples refer to pedestrians with the same and different identities as the anchor image, respectively; α is the boundary hyperparameter, max is the maximum value, min is the minimum value, P is the number of classes in each training batch, and K is the number of images per class. + This means that when the value in brackets [] is greater than 0, that value is taken as the loss; when it is less than 0, the loss is 0.
[0097]
[0098] Where, x 1i and x 2i The two are the upper body features (excluding clothing) F. w and clothing diversity F c The feature is that θ is a dynamic threshold and N is the number of sample pairs;
[0099]
[0100] L sem The mean squared error loss is given by f(x), b is the batch size, ||*||² represents normalization, and F is the mean squared error loss. b It is a feature of the original image, F a It is a characteristic of consistent clothing images, F c It is a characteristic of clothing diversity.
[0101] S5 specifically utilizes a trained deep learning model to extract global features of pedestrians, and uses these features for retrieval and matching. Specifically, it matches the image to be retrieved with each pedestrian image in the retrieval database one by one to obtain similarity; it calculates the Euclidean distance between the image to be retrieved and each pedestrian image in the database, and arranges the pedestrian images in the database in descending order of similarity, moving the images with higher similarity to the beginning of the retrieval sequence and placing the images with lower similarity at the end. The pedestrian image with the highest similarity is then used as the recognition result corresponding to the pedestrian image to be processed. The model's performance is evaluated using two metrics: the average accuracy (mAP) of the retrieval model and the rank-n (indicating whether the top n images are matched).
[0102] The specific formulas used in S5 are as follows:
[0103]
[0104] Where I1 and I2 are two input images, The Euclidean distance between the two images is... and These are the feature vectors corresponding to them obtained through forward propagation of the network.
[0105] Example 2
[0106] The table below shows the experimental results using only black clothing, all black clothing, and black upper body and white lower body. Here, mAP represents the average retrieval accuracy of all retrieved models, and rank-1 indicates the first successful hit.
[0107]
[0108] Example 3
[0109] To demonstrate the effectiveness of this invention, experiments were conducted on the publicly available person re-identification datasets PRCC and LTCC. The experiments were evaluated using mAP and rank-1. Here, mAP represents the average retrieval accuracy of all retrieved models, and rank-1 indicates the first successful match. CCIL is a newly proposed method for person re-identification using clothing changes, presented at the 2023 CVPR conference, and CC-ReID is a method proposed at the 2022 CVPR conference.
[0110]
[0111] Performance comparison of different algorithms with the method of this invention, as follows: Figure 4 As shown in the figure, the multi-granularity pedestrian re-identification method based on clothing shielding network proposed in this invention has excellent performance.
[0112] As can be seen from the above technical solution, this invention provides a method for re-identifying pedestrians in different clothing based on the fusion of uniform and diverse clothing features. The improved clothing consistency method makes the model focus more on non-clothing features. Features are extracted through a designed rich-scale adaptive module, making the model pay more attention to non-clothing human body features, and extracting non-clothing human body features from the upper body. This feature is fused with features extracted from the original image, thus desensitizing clothing features in the original image. A clothing diversity module randomly encodes clothing areas to increase clothing diversity, and the background area undergoes consistency processing. A designed loss function is used to calculate the loss with the original image features, weakening the background and clothing area features of the model, thereby focusing more on non-clothing human body features. This solves the noise problem caused by directly using semantic segmentation models and also effectively addresses the clothing-changing problem.
[0113] Although the specific embodiments of the invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the invention. Based on the technical solutions of the invention, various modifications or variations that can be made by those skilled in the art without creative effort are still within the scope of protection of the invention.
Claims
1. A method for pedestrian re-identification based on the fusion of uniform and diverse clothing, characterized in that, Includes the following steps: S1. Obtain the human body semantic segmentation map through the semantic segmentation model to obtain images of clothing consistency and diversity. Use these as network inputs and train them to obtain clothing consistency features, clothing diversity features, and original image features. The specific process is as follows: Using a human semantic segmentation algorithm, 18 human body parts in the corresponding image are obtained in advance. These 18 parts are fused and divided into clothing and non-clothing. Only the clothing area is blacked out. The clothing area is located using the fused label and the corresponding binary mask is generated. The element-wise dot product operation is performed with the original image to mask the clothing information, thereby obtaining an image in which the clothing area is masked, resulting in a clothing consistency image. The clothing area is randomly masked, and the background area is set to 0 to obtain a clothing diversity image. The clothing consistency image, the original image, and the clothing diversity image are input into the backbone network. The backbone network parameters of the original image and the clothing diversity image are shared, while the backbone network parameters of the clothing consistency image are unique. Three global features are obtained respectively: clothing consistency image feature, original image feature, and clothing diversity feature. = = = in, The pictures show the same clothes. This is the original image. These are pictures showcasing the diversity of clothing. It is a feature extraction backbone network. The consistent characteristics of the clothing images These are features of the original image. It is a characteristic of clothing diversity; S2. Use the designed rich-scale attention mechanism to extract non-clothing human body features: The features of the clothing consistency image extracted from S1 are divided into four parts. The first part is convolved and then added to the second part. The second part is convolved and then added to the third part. The third part is convolved and then added to the fourth part. The channel features of the feature map are reweighted by weighted feature vectors to make the model pay more attention to the human body features in the non-clothing area. Then, this feature is passed through the max pooling layer to obtain the non-clothing human body features. The non-clothing human body features are divided into two parts: one part is the upper body non-clothing human body features that are retained, and the other part is the lower body non-clothing human body features that are discarded. The specific formula is as follows: Among them, convolution , , All are convolution operations with a 3x3 kernel and 1 padding. It is average pooling. It's a demotion operation. It is a weighted operation. This is a max pooling operation with a pooling kernel size of 12*8. It is addition. It involves stitching features along the channel dimension. , , , It is a feature of clothing consistency images Image features after being divided into four equal parts horizontally; It refers to the upper body, not the clothing. S3. The original image features extracted using the ResNet50 residual network model are fused with the non-clothing features of the upper body: The original image is fed into the ResNet50 model for feature extraction. By fusing the extracted features with the upper body non-clothing features, the upper body region in the generalized features of the original image is weakened, so that the non-clothing human body features are inversely enhanced. Then, the inversely enhanced non-clothing human body features are passed through a fully connected layer for classification loss calculation. S4. Training the network by constraining the joint loss function: Three links are set up in the network. The first link learns human features independent of clothing by using a parameter-independent ResNet50 model. The second link learns global features of the original image for pedestrian retrieval. The third link learns features weakened by clothing and background. Then, joint training is performed using a loss function, which is divided into four parts: triplet loss, classification loss, mean squared error loss, and adaptive Euclidean distance loss. After calculation, the four parts of the loss are added together and jointly trained to obtain global features for pedestrian similarity matching. S5. Use the trained model to extract features for retrieval and matching: The image to be retrieved is matched one by one with each pedestrian image in the retrieval database to obtain similarity; the pedestrian images in the retrieval database are arranged in descending order of similarity, and the pedestrian image with the highest similarity is taken as the recognition result corresponding to the pedestrian image to be processed.
2. The method for re-identifying pedestrians changing clothes based on the fusion of uniform and diverse clothing as described in claim 1, characterized in that: The specific formulas used in S3 are as follows: in, These are characteristics resulting from the fusion. These are features of the original image. It is addition. It is the outer product.
3. The pedestrian re-identification method based on clothing uniformity and diversity fusion according to claim 1, characterized in that: The specific formulas used in S4 are as follows: The loss function for the entire network is: in, It's a loss of the entire network. It is classification loss. It is the ternary loss as a measure of loss. It is used as an adaptive Euclidean distance loss. Features of the original image and the diversity of clothing Sum after calculating the mean squared error loss; in, It is the size of the training batch. It is the input object The predicted probability, It is its corresponding pedestrian category label; in, , , These are features extracted from the anchor, positive, and negative images, respectively. Here, positive and negative samples refer to pedestrians with the same and different identities as the anchor image, respectively. These are boundary hyperparameters. It takes the maximum value. It takes the minimum value. It is the number of categories trained in each batch. It is the number of images in each category. It means When the intrinsic value is greater than 0, that value is taken as the loss; when it is less than 0, the loss is 0. in, and These are the non-clothing features of the upper body. and clothing diversity Features It is a dynamic threshold. It is the number of sample pairs; For mean squared error loss, b This represents the size of the training batch, where ||*||2 represents normalization. These are features of the original image. The consistent characteristics of the clothing images It is a characteristic of clothing diversity.
4. The pedestrian re-identification method based on clothing uniformity and diversity fusion according to claim 1, characterized in that: The specific formulas used in S5 are as follows: in, and Given two input images, Let Euclidean distance be the distance between the two images. and These are the feature vectors corresponding to them obtained through forward propagation of the network.