A method for re-identifying pedestrians in mixed-dress clothing
By constructing a location-aware, clothing feature-refined network and a multi-granular progressive identity feature enhancement network, and combining generative adversarial networks and a feature distance dynamic fusion strategy guided by clothing similarity, the problem of pedestrian re-identification in mixed-dress scenarios is solved, achieving efficient and robust pedestrian recognition and feature extraction, which can be applied to multiple smart fields.
Patent Information
- Application Number
- CN202510639689.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-19
- Publication Date
- 2026-03-06
- Estimated Expiration
- 2045-05-19
AI Technical Summary
Existing pedestrian re-identification methods struggle to effectively extract identity and clothing features in mixed-dress scenarios, and their performance degrades when clothing changes, making it impossible to accurately identify pedestrians.
We construct a location-aware, clothing feature-refined network model and a multi-granular progressive identity feature enhancement network model. By using a generative adversarial network to separate identity and clothing cues, we design a feature distance dynamic fusion strategy guided by clothing similarity to improve feature extraction and recognition performance.
It significantly improves pedestrian re-identification performance in mixed-dress scenarios, and can provide efficient and robust pedestrian localization and trajectory association over long periods of time. It is applicable to fields such as smart security, social governance, and smart communities, and can be extended to applications that require multimodal feature fusion, such as smart cities, intelligent transportation, and smart e-commerce.
Smart Images
Figure CN120564120B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of artificial intelligence and computer vision, and more specifically to a method for re-identifying pedestrians in mixed-dress clothing. Background Technology
[0002] Person re-identification (ReID), a key technology in intelligent surveillance systems, aims to achieve individual tracking across cameras and scenes through feature matching. In traditional short-term applications, appearance feature matching-based person re-identification methods perform well. These methods rely on the assumption that pedestrians wear consistent clothing and emphasize appearance features to achieve image matching by designing effective feature representation and metric learning techniques.
[0003] However, in the real world, changes in pedestrian clothing are inevitable. When the monitoring time span is long, pedestrians may change their clothing multiple times, which significantly alters their appearance. Traditional pedestrian re-identification methods are based on the assumption of consistent clothing, i.e., non-clothing-changing pedestrian re-identification. This leads to severe performance degradation when faced with clothing changes. For example, when a pedestrian changes clothing, a matching model based on appearance features may fail to correctly identify the pedestrian, resulting in tracking failure. To address this challenge, Cloth-Changing Person Re-Identification (CC-ReID) has emerged and become an important research topic.
[0004] Existing methods for pedestrian re-identification after clothing changes mainly focus on extracting identity features unrelated to clothing, and can be broadly classified into two categories: (1) Methods that explicitly integrate clothing-invariant cues: These methods use prior knowledge to guide the extraction of identity features robust to clothing changes. These prior knowledge include 3D body information, contour details, gait patterns, key points, and human body analysis data. By utilizing this information, the model can learn identity features unaffected by clothing changes. (2) Methods that decouple clothing and identity features: These methods employ adversarial learning, sampling strategies, or augmentation techniques to make the learning of clothing features and identity features independent in the implicit space at the data level or during model training, thereby capturing discriminative features that remain unchanged by clothing changes and achieving accurate pedestrian identification.
[0005] Despite some progress in pedestrian re-identification under different clothing conditions, real-world applications often present more complex scenarios where pedestrians are dressed differently or not, creating mixed-clothing environments. In such scenarios, both existing pedestrian re-identification methods (clothing changes and consistent clothing) have limitations. Traditional non-clothing methods rely excessively on appearance features, making them susceptible to misinterpretation when pedestrians' clothing changes, thus failing to accurately identify them. Clothing-changing methods, while suppressing clothing-related information to focus on identity cues, discard useful visual cues such as color and texture, making accurate identification difficult when clothing remains unchanged. Regardless of whether pedestrians' clothing remains consistent or changes, neither type of method can prioritize all correct results from the database.
[0006] Therefore, how to effectively extract identity and clothing features in real-world mixed-dress scenarios and dynamically fuse feature distances to improve pedestrian re-identification performance has become a pressing issue for those skilled in the art. Summary of the Invention
[0007] In view of the above problems, the present invention provides a method for re-identifying pedestrians with mixed clothing, which deeply mines highly discriminative identity and clothing features, and designs a feature distance dynamic fusion strategy guided by clothing similarity, so as to at least solve some of the technical problems mentioned in the background art.
[0008] To achieve the above objectives, the present invention adopts the following technical solution:
[0009] This invention provides a method for re-identifying pedestrians with mixed clothing, comprising the following steps:
[0010] Step 1: Construct a location-aware, clothing feature-refined network model:
[0011] Based on the input pedestrian image, a clothing region mask is obtained using a human body analysis model, enabling the clothing backbone network to predict the clothing region; the clothing region mask is then used as the ground truth label to backpropagate and update the parameters of the clothing backbone network.
[0012] The original clothing features are extracted from pedestrian images using a clothing backbone network, and a clothing prototype library is built based on the original clothing features.
[0013] Step 2: Construct a multi-granularity progressive identity feature enhancement network model:
[0014] By leveraging the concept of generative adversarial networks, generators and discriminators are used to separate identity and clothing clues, ensuring that the original identity features extracted by the identity backbone network retain key identity information.
[0015] Construct a progressive optimization paradigm with three levels: instance-instance, instance-prototype, and prototype-prototype, update the parameters of the identity backbone network, and assist in updating the parameters of the clothing backbone network.
[0016] Step 3: Image retrieval process:
[0017] Based on the trained clothing backbone network and identity backbone network, as well as the constructed clothing prototype library, the clothing similarity between the query image and the library image is calculated. Then, based on the clothing similarity, the identity feature distance and clothing feature distance are dynamically fused, and the fused distance is used as the retrieval similarity metric. The retrieval results are output in descending order of fused distance.
[0018] Furthermore, the specific implementation process of step 1 is as follows:
[0019] Step 11: Obtain training samples containing multiple pedestrian images, process the input pedestrian images using a human body analysis model, and obtain human body analysis results including clothing areas and non-clothing areas;
[0020] Step 12: Construct clothing region mask: Set the value of clothing region in the human body parsing result to 1, and set the value of non-clothing region in the human body parsing result to 0;
[0021] This invention lays the foundation for subsequent extraction of clothing and identity features through human body analysis, enabling subsequent models to focus more on clothing areas or identity areas unrelated to clothing, thereby improving the accuracy and effectiveness of clothing and identity feature extraction and providing a key preprocessing step for the entire pedestrian re-identification method.
[0022] Step 13: Use the clothing backbone network to extract the original clothing features from the input pedestrian image; input the original clothing features into the constructed pixel-level clothing localization network model, perform 1x1 convolution operation and channel-dimensional Softmax activation function on the original clothing features to obtain the pixel-level attention map;
[0023] Step 14: Using the clothing region mask as the real label, calculate the loss on the pixel-level attention map using the cross-entropy loss function, and backpropagate to update the parameters of the clothing backbone network;
[0024] Step 15: Construct a two-part clothing appearance enhancement network model. Input the original clothing features into the constructed two-part clothing appearance enhancement network model. Divide the original clothing features into two parts along the height dimension. Then, refine the features of the two parts using a self-attention mechanism and splice them together to obtain spliced clothing features. Use the clothing region mask as the ground truth label. Use the cross-entropy loss function to calculate the loss of the spliced clothing features and backpropagate to update the parameters of the clothing backbone network.
[0025] This invention first uses a clothing backbone network to obtain the original clothing features, and then constructs a pixel-level clothing localization network model and a two-part clothing appearance enhancement network model to improve the ability to perceive the location of clothing and the ability to extract the appearance features of clothing, respectively. The two work together to form a refined network model to accurately capture clothing details.
[0026] Step 16: Calculate the static center of the original clothing features of each batch of input pedestrian images based on the clothing tags as clothing prototypes, and continuously update the clothing prototype library using an exponential moving average method.
[0027] Furthermore, the specific implementation process of step 2 is as follows:
[0028] Step 21: Combine the human body analysis results and the clothing area mask to obtain the pedestrian clothing erased image;
[0029] Step 22: Use an identity backbone network to extract the corresponding original identity features and clothing erasure features from the pedestrian image and the pedestrian clothing erasure image, respectively; use a generative adversarial network to preserve the identity information in the original identity features: extract clothing domain features from the original identity features through a clothing generator; extract identity domain features from the clothing domain features through an identity generator; use a discriminator to determine whether a given feature belongs to the clothing domain feature or the identity domain feature, thereby achieving soft decoupling between the identity domain feature and the clothing domain feature;
[0030] Step 23: Based on the original identity features and clothing erasure features, update the parameters of the identity backbone network using a multi-granularity progressive optimization strategy, and assist in updating the parameters of the clothing backbone network.
[0031] Furthermore, step 23 specifically includes:
[0032] Step 231: Obtain the original identity features and clothing erasure features of each batch of input pedestrian images; take the static center of the original identity features and clothing erasure features of each pedestrian category as the identity prototype and clothing erasure prototype corresponding to that pedestrian category, and use exponential moving average to continuously update the corresponding identity prototype library and clothing erasure prototype library.
[0033] Step 232: Align the distribution of the original identity features and clothing erasure features using KL divergence, treat each feature as an instance, and apply triplet constraints to the original identity feature instance, the original clothing feature instance, and the clothing erasure feature instance, denoted as instance-instance constraint;
[0034] This invention realizes the transfer of identity knowledge from the clothing erasure branch to the identity branch through instance-instance constraints, and at the same time helps to create a more obvious inter-class feature distribution, laying the foundation for subsequent prototype-based optimization to form a compact and discriminative representation.
[0035] Step 233: For the original identity feature, original clothing feature, and clothing erasure feature, minimize the distance between a single instance and its corresponding prototype. At the same time, perform interactive operations on the instances and prototypes of the original identity feature and clothing erasure feature, which is called instance-prototype constraint.
[0036] This invention effectively reduces intra-class variance and feature distribution differences between the original RGB image and the clothing erased image by constraining the instance-prototype relationship, making the features learned by the network model more compact and discriminative.
[0037] Step 234: For each target pedestrian image, obtain its corresponding clothing prototype. Using contrast loss, minimize the distance between the clothing prototype and its related identity prototype and clothing eraser prototype, while maximizing the distance between the clothing prototype and its unrelated identity prototype and clothing eraser prototype. This is called the prototype-prototype constraint.
[0038] This invention treats clothing as an inherent subset of the overall identity by constraining the prototype-prototype relationship. This ensures that when calculating the similarity between the query image and the library image, the inherent identity information can continuously guide and constrain the evaluation of clothing features, while avoiding the identity prototype being mistakenly pulled to the clothing prototype, thereby preventing specific clothing information from being mixed into the extracted identity features.
[0039] Step 235: Update the parameters of the clothing backbone network and the identity backbone network using backpropagation with instance-instance constraints and instance-prototype constraints; update the parameters of the clothing backbone network using backpropagation with prototype-prototype constraints.
[0040] This invention extracts original identity features using a backbone network, employs generative adversarial networks to achieve soft decoupling between identity and clothing features while preserving identity information, and adopts a multi-granular progressive optimization strategy to adjust feature distribution, reduce intra-class variance, strengthen identity features and assist in learning clothing features, and finally constructs an identity enhancement network model.
[0041] Furthermore, the specific implementation process of step 3 is as follows:
[0042] Step 31: Input the query image and the library image into the trained clothing backbone network and identity backbone network, and extract the clothing features and identity features corresponding to the query image and the library image, respectively;
[0043] Step 32: Based on the clothing features corresponding to the query image and the library image, the K-nearest neighbor algorithm is used to retrieve the K most similar clothing prototypes from the constructed clothing prototype library, respectively, to obtain the query image prototype set and the library image prototype set; the Jaccard distance between the query image prototype set and the library image prototype set is calculated as the clothing similarity.
[0044] Step 33: Based on the clothing features and identity features corresponding to the query image and the library image respectively, calculate the clothing feature distance and identity feature distance using the cosine distance metric; use clothing similarity as the weight of the clothing feature distance, and perform weighted fusion of the clothing feature distance and identity feature distance to obtain the fused distance;
[0045] This invention inputs query images and library images into a trained backbone network to obtain corresponding identity and clothing features. It then uses a clothing prototype library and the K-nearest neighbor algorithm to calculate clothing similarity. Finally, using cosine distance as a metric, it combines the clothing similarity as a weight to weight the identity feature distance and clothing feature distance, thereby achieving accurate pedestrian re-identification and matching.
[0046] Step 34: Use the fusion distance as a retrieval similarity metric and output the retrieval results in ascending order of fusion distance.
[0047] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a method for re-identifying pedestrians with mixed clothing, which has the following beneficial effects:
[0048] This invention deeply mines highly discriminative identity and clothing features and designs a dynamic feature distance fusion strategy guided by clothing similarity, effectively improving pedestrian re-identification performance in mixed-dress scenarios. Furthermore, the principles of this invention can be applied to fields such as smart security, social governance, and smart communities, providing strong technical support for tasks such as pedestrian localization, trajectory association, and behavior analysis over long periods. Simultaneously, the principles of this invention have broad applicability and can be further extended to scenarios requiring the extraction and dynamic fusion and selective use of multimodal discriminative features, such as smart cities, intelligent transportation, smart e-commerce, and cross-scenario user behavior mining, thereby achieving efficient and robust feature extraction and utilization, and enhancing the intelligence level of related applications.
[0049] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description
[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0051] Figure 1 This is a schematic flowchart of the pedestrian re-identification method with mixed clothing provided in an embodiment of the present invention.
[0052] Figure 2This is a schematic diagram of the location-aware clothing feature refinement network model and the multi-granular progressive identity feature enhancement network model provided in the embodiments of the present invention.
[0053] Figure 3 Example diagram of the dynamic fusion and retrieval process of feature distance based on clothing similarity during the inference stage provided in this embodiment of the invention. Detailed Implementation
[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0055] This invention discloses a method for re-identifying pedestrians with mixed clothing, comprising the following steps:
[0056] Step 1: Construct a location-aware, clothing feature-refined network model:
[0057] Based on the input pedestrian image, the clothing region mask is obtained based on the human body analysis model, enabling the clothing backbone network to predict the clothing region; the clothing region mask is used as the real label, and the parameters of the clothing backbone network are updated by backpropagation, training the clothing backbone network to have pixel-level localization capability for clothing positions.
[0058] The original clothing features are extracted from pedestrian images using a clothing backbone network, and a clothing prototype library is built based on the original clothing features.
[0059] Step 2: Construct a multi-granularity progressive identity feature enhancement network model:
[0060] By leveraging the concept of generative adversarial networks, generators and discriminators are used to separate identity and clothing clues, ensuring that the original identity features extracted by the identity backbone network retain key identity information.
[0061] A progressive optimization paradigm is constructed at three levels: instance-instance, instance-prototype, and prototype-prototype. The parameters of the identity backbone network are updated to strengthen identity features; and the parameter updates of the clothing backbone network are assisted.
[0062] Step 3: Image retrieval process:
[0063] Based on the trained clothing backbone network and identity backbone network, as well as the constructed clothing prototype library, the clothing similarity between the query image and the library image is calculated. Then, based on the clothing similarity, the identity feature distance and clothing feature distance are dynamically fused, and the fused distance is used as the retrieval similarity metric. The retrieval results are output in descending order of fused distance.
[0064] The pedestrian re-identification method for mixed-dress scenarios provided by this invention, which takes into account both changing and non-changing clothing scenarios, effectively improves the pedestrian re-identification performance in mixed-dress scenarios. The three steps described above will be explained in detail below.
[0065] In step S1 above, the specific implementation process is as follows:
[0066] Step 11: Obtain training samples containing multiple pedestrian images, process the input pedestrian images through the human body analysis model, and obtain human body analysis results including clothing regions (e.g., tops and bottoms) and non-clothing regions (e.g., heads).
[0067] Among them, the human body parsing model can be selected from models such as SCHP, ACE2P, and M2FP, and the human body parsing model can be pre-trained on datasets such as ATR, LIP, and PPP.
[0068] Step 12: Construct clothing region mask: Set the value of clothing region in the human body parsing result to 1, and set the value of non-clothing region in the human body parsing result to 0;
[0069] In this embodiment of the invention, human body analysis lays the foundation for subsequent extraction of clothing and identity features, enabling the subsequent model to focus more on clothing areas or identity areas unrelated to clothing, thereby improving the accuracy and effectiveness of clothing and identity feature extraction and providing a key preprocessing step for the entire pedestrian re-identification method.
[0070] Step 13: Use the clothing backbone network to extract the original clothing features from the input pedestrian image; input the original clothing features into the constructed pixel-level clothing localization network model; in the pixel-level clothing localization network model, perform 1x1 convolution operation and channel-dimensional Softmax activation function on the original clothing features to obtain the pixel-level attention map;
[0071] Step 14: Using the clothing region mask as the ground truth label, calculate the loss on the pixel-level attention map using the cross-entropy loss function, as shown in the following formula:
[0072]
[0073] Among them, L loc denoted by ; N represents the total number of pedestrian images in a batch; H represents the length of the pixel-level attention map; W represents the width of the pixel-level attention map; i represents the index of the pedestrian image; p represents each pixel in the pixel-level attention map. This represents the clothing region mask at position p in the i-th pedestrian image; This represents the attention value at position p in the i-th pedestrian image;
[0074] The loss is backpropagated to update the parameters of the garment backbone network;
[0075] Step 15: Construct a two-part clothing appearance enhancement network model. Input the original clothing features into the constructed two-part clothing appearance enhancement network model. Divide the original clothing features into two parts along the height dimension. Then, refine the features of the two parts using a self-attention mechanism and splice them together to obtain spliced clothing features. Use the clothing region mask as the ground truth label. Use the cross-entropy loss function to calculate the loss of the spliced clothing features and backpropagate to update the parameters of the clothing backbone network.
[0076] In this embodiment of the invention, the original clothing features are first obtained using a clothing backbone network. Then, a pixel-level clothing localization network model and a two-part clothing appearance enhancement network model are constructed to improve the ability to perceive the location of clothing and the ability to extract the appearance features of clothing, respectively. The two work together to form a refined network model to accurately capture clothing details.
[0077] Step 16: Calculate the static center of the original clothing features of each batch of input pedestrian images based on the clothing tags as clothing prototypes, and continuously update the clothing prototype library using an exponential moving average method.
[0078] In step 2 above, the specific implementation process is as follows:
[0079] Step 21: Combine the human body analysis results and the clothing area mask to obtain the pedestrian clothing erased image;
[0080] Step 22: Use an identity backbone network to extract the corresponding original identity features and clothing erasure features from the pedestrian image and the pedestrian clothing erasure image, respectively; use a generative adversarial network to preserve the identity information in the original identity features: extract clothing domain features from the original identity features through a clothing generator; extract identity domain features from the clothing domain features through an identity generator; use a discriminator to determine whether a given feature belongs to the clothing domain feature or the identity domain feature, thus achieving soft decoupling between the identity domain feature and the clothing domain feature; represented as:
[0081]
[0082]
[0083] in, Indicates generator loss; G represents the discriminator loss; N represents the total number of pedestrian images in a batch; D represents the discriminator, G... clo This indicates a clothing generator; Represents the original identity features of the i-th pedestrian image; The original identity features of the i-th pedestrian image are used by the clothing generator and the identity generator to generate the identity domain features; Represents the original clothing features of the i-th pedestrian image;
[0084] These two loss functions are backpropagated to update the network parameters of the generator and discriminator;
[0085] Step 23: Based on the original identity features and clothing erasure features, update the parameters of the identity backbone network using a multi-granularity progressive optimization strategy, and assist in updating the parameters of the clothing backbone network; Step 23 specifically includes:
[0086] Step 231: Obtain the original identity features and clothing erasure features of each batch of input pedestrian images; take the static center of the original identity features and clothing erasure features of each pedestrian category as the identity prototype and clothing erasure prototype corresponding to that pedestrian category, and continuously update the corresponding identity prototype library and clothing erasure prototype library using exponential moving average; taking identity features as an example, the process includes:
[0087]
[0088] in, This indicates the method of constructing the identity prototype; This represents the prototype of the pedestrian with identity label k; t represents the current update round. N represents the update coefficient; id This represents the number of pedestrians in each identity tag within a batch; y i This represents the identity label of the i-th pedestrian image; Represents the original identity features of the i-th pedestrian image;
[0089] Step 232: At the instance-to-instance level, align the distributions of the original identity features and the clothing erasure features using KL divergence, as follows:
[0090]
[0091] Among them, L kl The KL divergence loss is used to align the distribution of original identity features and clothing erasure features: N represents the total number of pedestrian images in a batch; i represents the index of the pedestrian image; This represents the category probability output by the identity classifier for the original identity features corresponding to the i-th pedestrian image; This represents the category probability output by the identity classifier for the clothing erasure feature corresponding to the i-th pedestrian image;
[0092] Treat each feature as an instance, and apply triplet constraints to the original identity feature instance, the original clothing feature instance, and the clothing erasure feature instance, denoted as instance-instance constraint;
[0093] In this embodiment of the invention, the transfer of identity knowledge from the clothing erasure branch to the identity branch is realized through the constraints between instances, which also helps to create a more obvious inter-class feature distribution, laying the foundation for subsequent prototype-based optimization to form a compact and discriminative representation.
[0094] Step 233: At the instance-prototype level, for original identity features, original clothing features, and clothing erasure features, minimize the distance between a single instance and its corresponding prototype. Taking identity instances and prototypes as an example, the process is as follows:
[0095]
[0096] in, The loss represents the merging loss between identity instances and identity prototypes; N represents the total number of pedestrian images in a batch; i represents the index of the pedestrian image; ||·||2 represents L2 regularization; Represents the original identity features of the i-th pedestrian image; This represents the identity prototype corresponding to the identity label in the i-th pedestrian image;
[0097] Simultaneously, interactive operations are performed on instances and prototypes of the original identity features and clothing erasure features, denoted as instance-prototype constraints; the process is as follows:
[0098]
[0099] L int The loss represents the interaction operation between instances and prototypes of the original identity features and clothing erasure features: N represents the total number of pedestrian images in a batch; i represents the index of the pedestrian image; S(·,·) represents the cosine similarity between two vectors; Represents the original identity features of the i-th pedestrian image; This represents the clothing erasure feature of the i-th pedestrian image; This represents the identity prototype corresponding to the identity label in the i-th pedestrian image; P represents the clothing erase prototype corresponding to the identity tag in the i-th pedestrian image; id The number of identity prototypes is indicated; j represents the identity prototype index or the clothing erase prototype index. This represents the j-th identity prototype; This indicates that the j-th clothing prototype has been erased.
[0100] In this embodiment of the invention, the constraint between instance and prototype effectively reduces the intra-class variance and the feature distribution gap between the original RGB image and the clothing erased image, making the features learned by the network model more compact and more discriminative.
[0101] Step 234: At the prototype-prototype level, for each target pedestrian image and its corresponding identity label, obtain its corresponding clothing prototype. Using contrastive loss, minimize the distance between the clothing prototype and its related identity prototype and clothing erasure prototype, while maximizing the distance between the clothing prototype and its unrelated identity prototype and clothing erasure prototype. This is denoted as the prototype-prototype constraint. The process is as follows:
[0102]
[0103] in, P represents the constraint loss between the identity prototype and the clothing prototype; clo This indicates the number of clothing prototypes constructed; c represents the index of the clothing prototype. This represents the identity prototype corresponding to the identity tag to which the c-th clothing prototype belongs; This represents the c-th clothing prototype; sg(·) represents gradient stopping;
[0104] In this embodiment of the invention, in the constraints before prototype-prototype, clothing is regarded as an inherent subset of the overall identity, ensuring that when calculating the similarity between the query image and the library image, the inherent identity information can continuously guide and constrain the evaluation of clothing features, while avoiding the identity prototype being mistakenly pulled to the clothing prototype, thereby preventing specific clothing information from being mixed into the extracted identity features.
[0105] Step 235: Update the parameters of the clothing backbone network and the identity backbone network using backpropagation with instance-instance constraint and instance-prototype constraint losses; update the parameters of the clothing backbone network using backpropagation with prototype-prototype constraint losses.
[0106] In this embodiment of the invention, the original identity features are extracted with the help of a backbone network, and a generative adversarial network is used to achieve soft decoupling between identity and clothing features to maintain identity information. A multi-granularity progressive optimization strategy is adopted to adjust the feature distribution, reduce intra-class variance, strengthen identity features and assist in the learning of clothing features, and finally construct an identity enhancement network model.
[0107] In step 3 above, the specific implementation process is as follows:
[0108] Step 31: Input the query image and the library image into the trained clothing backbone network and identity backbone network, and extract the clothing features and identity features corresponding to the query image and the library image, respectively;
[0109] Step 32: Based on the clothing features corresponding to the query image and the library image, the K-nearest neighbor algorithm is used to retrieve the K most similar clothing prototypes from the constructed clothing prototype library, resulting in the query image prototype set Q and the library image prototype set G; the Jaccard similarity between the query image prototype set Q and the library image prototype set G is calculated as the clothing similarity score; the process is as follows:
[0110]
[0111] This represents the clothing similarity between the query image and the library images; q represents the query image, and g represents the library images. This represents the set of KNN prototypes retrieved from the clothing prototype library for the query image; This represents the set of KNN prototypes retrieved from the clothing prototype library for the library image;
[0112] Step 33: Based on the clothing features and identity features corresponding to the query image and the library image respectively, calculate the clothing feature distance and identity feature distance using the cosine distance metric; use clothing similarity as the weight for the clothing feature distance, and perform weighted fusion of the clothing feature distance and identity feature distance to obtain the fused distance; the process is as follows:
[0113]
[0114] in, The final distance between the query image and the library image is represented by D(·,·), which represents the cosine distance between the two vectors. Indicates the identity characteristics of the queried image; Represents the identity features of the library images; Indicates the clothing features of the query image; Represents clothing features in the library images;
[0115] In this embodiment of the invention, query images and database images are input into a trained backbone network to obtain corresponding identity and clothing features. Clothing similarity is calculated using a clothing prototype database and the K-nearest neighbor algorithm. Then, using cosine distance as a metric, clothing similarity is used as a weight to weight the identity feature distance and clothing feature distance, achieving accurate pedestrian re-identification matching. For example, with K=10, a query image and a database image retrieve the K most similar clothing prototypes. The intersection of the prototype sets has 4 elements, and the union has 16 elements. Using Jaccard similarity calculation, the clothing similarity between these two images is 0.25. Therefore, the total distance between these two images is 0.25 * clothing feature distance + (1-0.25) * identity feature distance. In this way, clothing similarity is well measured, and the overall strategy achieves good performance in mixed-dress scenarios.
[0116] Step 34: Use the fusion distance as a retrieval similarity metric and output the retrieval results in ascending order of fusion distance (i.e., in descending order of retrieval similarity).
[0117] In one specific embodiment, a pedestrian re-identification method for mixed-dress scenarios is disclosed. Addressing the performance limitations of existing pedestrian re-identification methods based on consistent and changing clothing in mixed-dress scenarios, this method employs pixel-level clothing localization and a two-part clothing appearance enhancement method to strengthen clothing feature extraction. It also utilizes a generative adversarial network-based identity information preservation and multi-granular progressive optimization strategy to enhance identity feature extraction. During the retrieval process, for both the query image and the database images, clothing similarity is calculated between the images using clothing features and a clothing prototype library. This clothing similarity serves as a guide for dynamically fusing identity feature distance and clothing feature distance. The fused distance is then used as the retrieval similarity metric to output the retrieval results, effectively improving pedestrian re-identification performance in mixed-dress scenarios.
[0118] The overall process is as follows Figure 1 The process includes three steps: training a location-aware, clothing feature refinement network and building a clothing prototype library; training a multi-granular, progressive identity feature enhancement network; and dynamically fusing identity and clothing feature distances to output retrieval results. Specific steps include:
[0119] S1: Train a position-aware, clothing feature refinement network to build a clothing prototype library;
[0120] Specifically, the human body analysis model is used to process the input pedestrian image to obtain region analysis results containing different categories; a clothing region mask is constructed: the value belonging to the clothing region in the human body analysis result is set to 1, and the value belonging to the non-clothing region in the human body analysis result is set to 0.
[0121] The overall structure of the training position-aware clothing feature refinement network is as follows: Figure 2 As shown, in terms of location awareness, the binary clothing mask is aligned with the clothing feature map through bilinear interpolation, and then a pixel-level attention map is generated through 1×1 convolution and channel softmax activation. Focused cross-entropy loss is used to ensure that the model learns accurate clothing localization without background interference.
[0122] In terms of appearance enhancement, clothing features are decomposed into two parts, each of which is independently refined through a self-attention mechanism and then connected to reconstruct the overall fine-grained clothing representation.
[0123] The static center of the original clothing features of each batch of input pedestrian images is calculated based on the clothing tags as the clothing prototype, and the clothing prototype library is continuously updated using an exponential moving average method.
[0124] S2: Training a multi-granular progressive identity feature enhancement network;
[0125] Specifically, key identity clues are first preserved from clothing information using an identity preservation method based on generative adversarial networks, and then a multi-granular progressive optimization strategy is applied to further enhance the identity representation.
[0126] Identity preservation based on generative adversarial networks separates identity and clothing clues through adversarial training of the generator and discriminator.
[0127] The multi-granularity progressive optimization strategy optimizes at three levels: instance-to-instance, instance-to-prototype, and prototype-to-prototype. Instance-to-instance optimization aligns the distribution of clothing mask branches and identity branches using KL divergence and applies triplet loss to increase inter-class distance. Instance-to-prototype optimization minimizes the distance between instance features and their corresponding prototypes and promotes interaction between instance features and prototypes. Prototype-to-prototype optimization employs a prototype-based contrastive learning method to minimize the distance between the clothing prototype and its related identity prototypes and clothing erasure prototypes, while maximizing the distance between the clothing prototype and its unrelated identity prototypes and clothing erasure prototypes. Instance-to-instance and instance-to-prototype losses are backpropagated to update the parameters of the clothing and identity backbone networks. In the prototype-to-prototype constraints, clothing is considered an intrinsic subset of the overall identity, and loss backpropagation only updates the parameters of the clothing backbone network. This ensures that when calculating the similarity between the query image and the library images, intrinsic identity information continuously guides and constrains the evaluation of clothing features, while preventing identity prototypes from being mistakenly pulled towards clothing prototypes, thus preventing the extraction of specific clothing information from being mixed into the identity features.
[0128] S3: By calculating the clothing similarity between the query image and the library images, the identity feature distance and clothing feature distance are dynamically fused. The fused distance is used as the retrieval similarity metric, and the retrieval results are output in descending order of similarity.
[0129] Specifically, Figure 3 This is an example of a dynamic fusion process of identity and clothing features based on clothing similarity. Given a query image and a library image, the process first uses a clothing encoder and an identity encoder in the network to extract their identity and clothing features respectively. Using the K-nearest neighbor algorithm, based on the extracted clothing features, the K clothing prototypes closest to the query image and the library image are retrieved from a pre-built clothing prototype library. The set of prototypes corresponding to the query image is denoted as Q, and the set of prototypes corresponding to the library image is denoted as G. The Jaccard similarity is used to quantify the clothing similarity between the query image and the library image; the larger the value, the higher the clothing similarity between the two images, meaning they are more likely to be the same clothing, and vice versa. The calculated clothing similarity is then incorporated into the overall distance calculation between the query image and the library image. The overall distance is obtained by a weighted combination of the identity distance and the clothing distance. Finally, the overall distance is used as the retrieval similarity metric, and the final retrieval result is output.
[0130] When the clothing similarity is high, it means that the clothing in the query image and the image in the library are likely the same. In this case, the weight of clothing distance in the formula is high, and the model will pay more attention to clothing features based on appearance during the matching process. When the clothing similarity value is low, it means that the clothing may have changed. The weight of identity distance increases, and the model will shift the matching focus to identity clues that are unrelated to clothing.
[0131] For the distance-based dynamic fusion process, the parameter K in K-nearest neighbors can be adjusted. A larger K value results in greater intersection between the K-nearest neighbor clothing prototypes of the query image and the database image, leading to higher clothing similarity and potentially affecting the distance weighting. However, a small K value may result in disjoint sets of K-nearest neighbor clothing prototypes between the query image and the database image, in which case the distance-based dynamic fusion strategy degenerates into retrieval based solely on identity features. In practical applications, appropriate parameters need to be selected, such as determining the value of K based on the number of pre-trained clothing prototypes.
[0132] Furthermore, the principles of this invention can be applied to fields such as smart security, social governance, and smart communities, providing strong technical support for tasks such as pedestrian positioning, trajectory association, and behavior analysis over long periods. Simultaneously, the principles of this invention have broad applicability and can be further extended to scenarios requiring the extraction and dynamic fusion and selective use of multimodal discriminative features, such as smart cities, intelligent transportation, smart e-commerce, and cross-scenario user behavior mining. This enables efficient and robust feature extraction and utilization, improving the intelligence level of related applications.
[0133] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0134] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A mixed-dressed pedestrian re-identification method, characterized in that, The method comprises the following steps: Step 1: constructing a position-aware-clothing feature refinement network model: According to the input pedestrian image, the clothing region mask is obtained based on the human parsing model, so that the clothing backbone network predicts the clothing region; The clothing region mask is used as the true label to update the parameters of the clothing backbone network through back propagation; The clothing backbone network is used to extract the original clothing features from the pedestrian image, and a clothing prototype library is constructed based on the original clothing features; Step 2: constructing a multi-granularity progressive identity feature enhancement network model: With the help of the idea of generative adversarial network, the generator and the discriminator are used to separate the identity and the clothing clues, so that the original identity features extracted by the identity backbone network can maintain the key identity information; An instance-instance, instance-prototype, and prototype-prototype progressive optimization paradigm is constructed to update the parameters of the identity backbone network and assist the parameter update of the clothing backbone network; Step 3: image retrieval process: Based on the trained clothing backbone network and identity backbone network, and the constructed clothing prototype library, the clothing similarity of the query image and the library image is calculated; and according to the clothing similarity, the identity feature distance and the clothing feature distance are dynamically fused, the fusion distance is taken as the retrieval similarity measurement, and the retrieval result is output in the order from large to small according to the fusion distance.
2. The method of claim 1, wherein, The specific implementation process of step 1 is as follows: Step 11: obtaining training samples containing multiple pedestrian images, processing the input pedestrian image through the human parsing model to obtain human parsing results containing clothing regions and non-clothing regions; Step 12: constructing a clothing region mask: setting the value belonging to the clothing region in the human parsing result to 1, and setting the value belonging to the non-clothing region in the human parsing result to 0; Step 13: using the clothing backbone network to extract the original clothing features from the input pedestrian image; inputting the original clothing features into the constructed pixel-level clothing positioning network model to perform 1x1 convolution operation and channel dimension Softmax activation function on the original clothing features to obtain a pixel-level attention map; Step 14: using the clothing region mask as the true label, calculating the loss of the pixel-level attention map using the cross-entropy loss function, and updating the parameters of the clothing backbone network through back propagation; Step 15: constructing a two-part clothing appearance enhancement network model, inputting the original clothing features into the constructed two-part clothing appearance enhancement network model, dividing the original clothing features into upper and lower parts along the height dimension, and performing feature refinement on the upper and lower parts respectively using the self-attention mechanism, then splicing the two parts to obtain spliced clothing features; using the clothing region mask as the true label, calculating the loss of the spliced clothing features using the cross-entropy loss function, and updating the parameters of the clothing backbone network through back propagation; Step 16: calculating the static center of the original clothing features of each batch of input pedestrian images according to the clothing label as the clothing prototype, and continuously updating the clothing prototype library using the exponential moving average method.
3. The method of claim 1, wherein, The specific implementation process of step 2 is as follows: Step 21: obtaining a pedestrian clothing erasing image by combining the human parsing result and the clothing region mask; Step 22: Extract the corresponding original identity features and clothing-erased features from the pedestrian image and the pedestrian clothing-erased image respectively using the identity backbone network; Utilize the generative adversarial network to maintain the identity information in the original identity features: extract the clothing domain features from the original identity features through the clothing generator; extract the identity domain features from the clothing domain features through the identity generator; Judge whether the given features belong to the clothing domain features or the identity domain features through the discriminator, and realize the soft decoupling of the identity domain features and the clothing domain features; Step 23: Based on the original identity features and the clothing-erased features, update the parameters of the identity backbone network using a multi-granularity progressive optimization strategy, and assist the parameter update of the clothing backbone network.
4. The method of claim 3, wherein, The step 23 specifically comprises: Step 231: Obtain the original identity features and the clothing-erased features of each batch of input pedestrian images; take the static center of the original identity features and the clothing-erased features of each pedestrian class as the identity prototype and the clothing-erased prototype corresponding to the pedestrian class respectively, and continuously update the corresponding identity prototype library and clothing-erased prototype library by using the exponential moving average; Step 232: Align the distributions of the original identity features and the clothing-erased features through the KL divergence, take each feature as an instance, and apply a triplet constraint to the original identity feature instance, the original clothing feature instance and the clothing-erased feature instance, denoted as instance-instance constraint; Step 233: For the original identity features, the original clothing features and the clothing-erased features, minimize the distance between the single instance and the corresponding prototype, and simultaneously interact the instances and the prototypes of the original identity features and the clothing-erased features, denoted as instance-prototype constraint; Step 234: For the identity label corresponding to each target pedestrian image, obtain the corresponding clothing prototype, minimize the distance between the clothing prototype and the identity prototype and the clothing-erased prototype related to the clothing prototype, and maximize the distance between the clothing prototype and the identity prototype and the clothing-erased prototype unrelated to the clothing prototype, denoted as prototype-prototype constraint; Step 235: Update the parameters of the clothing backbone network and the identity backbone network by backward propagation of the losses of the instance-instance constraint and the instance-prototype constraint; update the parameters of the clothing backbone network by backward propagation of the loss of the prototype-prototype constraint.
5. The method of claim 1, wherein, The specific implementation process of the step 3 is: Step 31: input the query image and the library image into the trained clothing backbone network and identity backbone network to extract the corresponding clothing features and identity features of the query image and the library image; Step 32: based on the clothing features corresponding to the query image and the library image, retrieve the top K most similar clothing prototypes from the constructed clothing prototype library respectively by using the K-nearest neighbor algorithm, to obtain the query image prototype set and the library image prototype set; Calculate the Jaccard distance of the query image prototype set and the library image prototype set as the clothing similarity; Step 33: based on the clothing features and the identity features corresponding to the query image and the library image respectively, calculate the clothing feature distance and the identity feature distance respectively by using the cosine distance measurement method; The clothing similarity is taken as a weight of clothing feature distance, and the clothing feature distance and identity feature distance are weighted and fused to obtain a fusion distance; Step 34: taking the fusion distance as a retrieval similarity measure, outputting a retrieval result in order of the fusion distance from small to large.