Method for re-identifying a clothes changer based on isolating interference factors

Abstract

The application discloses a method for realizing re-identification of clothes-changing personnel based on isolation of interference factors, and relates to the technical field of intelligent image processing. The method comprises the following steps: acquiring a to-be-inquired image and a gallery image set; inputting the to-be-inquired image into a trained clothes-changing model to obtain a to-be-inquired feature vector, and inputting the gallery image set into the trained clothes-changing model to obtain a screening result feature vector set; respectively calculating the distance between the to-be-inquired feature vector and each screening result feature vector in the screening result feature vector set to obtain a plurality of distance sets; determining the minimum value in the plurality of distance sets and determining the screening result feature vector corresponding to the minimum value; and determining the gallery image set image corresponding to the to-be-inquired image according to the screening result feature vector. The extracted features can ignore the differences between clothes and cameras, thereby effectively distinguishing whether two images belong to the same pedestrian.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence image processing, and particularly relates to a method for realizing clothes-changing person re-identification by isolating interference factors. BACKGROUND

[0002] The pedestrian re-identification (ReID) task aims to train a deep model to identify the same pedestrian in a short-term scene with limited time and spatial span. The clothes-changing pedestrian re-identification (CC-ReID) task focuses on a more challenging and realistic long-term surveillance scene, and retrieves the images of pedestrians whose clothes may change. Obviously, in addition to the general challenges such as moderate view / posture changes, occlusions, low resolution, etc., CC-ReID also faces more severe challenges such as clothing changes caused by long-term use and drastic view changes. Therefore, the clothes of the pedestrian and the camera view are often associated with the identity information, which interferes with the learning of robust identity features. Separating the identity features from these interference factors is the key to solving CC-ReID.

[0003] Existing CC-ReID methods mainly focus on learning identity-related feature representations that are stable, rather than interference factors such as cloth and environmental features. These methods can be roughly divided into single-modal based and multi-modal based methods. Single-modal methods only learn identity features that are independent of cloth from RGB images, but this is insufficient to eliminate the influence of interference factors. Multi-modal methods capture identity features that are independent of cloth by exploring other synthetic data modalities (such as shape, gait motion, cloth-erased images, or cloth-swapped images), thereby improving performance. These methods mainly convert the interference factor disentanglement problem into a cross-modal feature alignment problem from a data-driven perspective. However, methods using cloth-erased images or cloth-swapped images lack an intrinsic consideration of the influence of clothing features. SUMMARY

[0004] The present application realizes that the extracted features can ignore the differences between clothes and cameras, thereby effectively distinguishing whether two images belong to the same pedestrian, by providing a method for realizing clothes-changing person re-identification by isolating interference factors.

[0005] The present application provides a method for realizing clothes-changing person re-identification by isolating interference factors, which comprises:

[0006] obtaining a to-be-queried image and a gallery image set;

[0007] The image to be queried is input into the trained clothing-changing model to obtain the query feature vector, and the image set from the image library is input into the trained clothing-changing model to obtain the filtered result feature vector set; wherein, the clothing-changing model includes: a dual-stream identity feature learning framework and a generative adversarial interference factor decoupling network, the dual-stream identity feature learning framework is used to extract identity features unrelated to clothing fabric; the generative adversarial interference factor decoupling network is used to counteract the interference of interference factors on the identity features;

[0008] Calculate the distance between the feature vector to be queried and each feature vector in the set of filtered result feature vectors to obtain a distance set consisting of multiple distances;

[0009] Determine the minimum value in the distance set, and determine the feature vector of the filtering result corresponding to the minimum value;

[0010] Based on the feature vector of the filtering results, the target image corresponding to the image to be queried in the image library is determined.

[0011] In one possible implementation, during the training of the clothing-changing model, the first loss function L of the dual-stream identity feature learning framework is calculated. dual Adjust the parameters of the two-stream identity feature learning framework; among them, calculate the first loss function L of the two-stream identity feature learning framework during training. dual ,include:

[0012] Using a dual-stream identity feature learning framework to study the original image X i Perform raw image stream processing to obtain raw stream processing results; wherein, the raw stream processing results include: a first feature vector f i And the first M ped dimensional probability vector p i Among them, M ped This represents the number of pedestrian categories in the training set;

[0013] The original image X is processed using a dual-stream identity feature learning framework. i Perform clothing removal flow processing to obtain clothing removal flow processing results; wherein, the clothing removal flow processing results include: a second feature vector. Second M ped dimensional probability vector

[0014] Using the original stream processing results and the clothing erasure stream processing results, the first loss function L of the dual-stream identity feature learning framework is calculated. dual .

[0015] In one possible implementation, during the training of the clothing-changing model, the cross-entropy loss function L of the adversarial interference factor decoupling network is calculated. int-ce and the allocation loss function L dise The parameters of the anti-interference factor decoupling network are adjusted, wherein the cross-entropy loss function L of the anti-interference factor decoupling network is calculated. int-ce and the allocation loss function L dise ,include:

[0016] The interference factor identification branch in the generative adversarial interference factor decoupling network is used to analyze the original image X. i Interference features are extracted to obtain the interference factor features f. i int The generated adversarial interference factor decoupling network includes an interference factor identification branch, a feature deentanglement module, and an interference factor decoupling target.

[0017] Utilizing the characteristics f of the interference factors i int Calculate the cross-entropy loss function L of the interference factor identification branch. int-ce The allocation loss function L of the target decoupled from the interference factors dise .

[0018] In one possible implementation, the training loss function of the clothing-changing model is expressed as:

[0019] L = L dual +αL int-ce +ηL dise ;

[0020] Among them, L dual L represents the loss function of the dual-stream identity feature learning framework; int-ce L represents the cross-entropy loss function; dise Represents the allocation loss function L dise ; α and η are both constants.

[0021] In one possible implementation, the dual-stream identity feature learning framework is used to learn the original image X. i Perform raw image stream processing to obtain the raw stream processing results, including:

[0022] Using a shared image encoder and identity feature extractor For the original image X i After processing, the first feature vector f is obtained. i ;

[0023] Pedestrian classifier Using the first feature vector f iGenerate the first M ped dimensional probability vector p i , the feature vector f i and the first M ped dimensional probability vector p i As the result of the original stream processing.

[0024] In one possible implementation, the original image X is processed using a dual-stream identity feature learning framework. i The clothing wiping flow is processed to obtain the clothing wiping flow processing results, including:

[0025] Generate a binary mask S using the SCHP model. i ;

[0026] Using the binary mask S i And the image erasure formula, to obtain the original image X i The corresponding erased image X i ;

[0027] Using a shared image encoder and identity feature extractor For the erased image X i Feature extraction is performed to obtain the second feature vector.

[0028] Pedestrian classifier Using the second feature vector Generate the second M ped dimensional probability vector The second feature vector and the second M ped dimensional probability vector As a result of the clothing wiping process.

[0029] In one possible implementation, the first loss function L of the dual-stream identity feature learning framework is calculated using the original stream processing result and the clothing erasure stream processing result. dual ,include:

[0030] According to the first M ped dimensional probability vector p i and the second M ped dimensional probability vector Calculate the loss function of the pedestrian classifier in the dual-stream identity feature learning framework to obtain the pedestrian classifier loss function L. ce ;

[0031] Determine the first feature vector f i and the second feature vector Computational Identity Feature Extractor , to obtain an identity feature extractor loss function L tri ;

[0032] constructing a first prototype model o i corresponding to the first feature vector f m , constructing a second prototype model o corresponding to the second feature vector f , and updating the first prototype model o m and the second prototype model o by an exponential moving average method to obtain an updated first prototype model o m and an updated second prototype model o

[0033] calculating a prototype model cross-flow matching loss function L m according to the updated first prototype model o and the updated second prototype model o mm ;

[0034] calculating a minimum intra-person distance of the original image X i , and using the minimum intra-person distance to constrain the distance between the first feature vector f i and the second feature vector f by an intra-distance constraint function L cicl ; wherein the minimum intra-person distance is: the minimum distance of different images of the same identity person;

[0035] obtaining a weight coefficient ω i corresponding to the second M ped -dimensional probability vector f according to a weight calculation formula;

[0036] determining a first loss function L dual of the dual-flow identity feature learning framework according to a pedestrian classifier loss function L ce , an identity feature extractor loss function L tri , the prototype model cross-flow matching loss function L mm , and the intra-distance constraint function L cicl .

[0037] In a possible implementation, the interference factor identification branch in the generative adversarial interference factor decoupling network is used to extract interference features from the original image X i to obtain interference factor features f i int , including:

[0038] using a shared image encoder For the original image X i The process is performed to obtain the initial processing result;

[0039] Utilizing the interference factor identification branch for specific feature extractors Feature extraction is performed on the initial processing results to obtain the interference factor features f. i int .

[0040] In one possible implementation, the use of the interference factor feature f i int Calculate the cross-entropy loss function L of the interference factor identification branch. int-ce The allocation loss function L of the target decoupled from the interference factors dise ,include:

[0041] The interference factor identification branch utilizes a first classifier and a second classifier to classify the interference factor features f. i int Feature extraction is performed to obtain clothing p i clo and camera number p i cam ;

[0042] The interference factor identification branch is based on the clothing p i clo and the camera number p i cam Calculate the loss function in the generative adversarial interference decoupling network to obtain the cross-entropy loss function L. int-ce ;

[0043] Using a trained generative adversarial interference factor decoupling network, the interference factor features f are decoupled. i int Supplement to the first feature vector f i In this process, the intermediate feature vector is obtained;

[0044] A pedestrian classifier is determined based on the intermediate feature vector. Allocation loss function L dise .

[0045] In one possible implementation, the generative adversarial interference factor decoupling network is trained, including:

[0046] The feature de-entanglement module utilizes the first feature generator Regarding the characteristics f of the interference factors i int Perform domain transfer to obtain first transfer information, and then utilize the second feature generator. The first feature vector f i Domain transfer is performed to obtain second transfer information;

[0047] The first feature generator The corresponding first discriminator The authenticity of the first transfer information is discriminated to obtain a first discrimination result; and the second feature generator The corresponding second discriminator The authenticity of the second transfer information is discriminated to obtain a second discrimination result;

[0048] The first transfer information is domain restored according to the first discrimination result to obtain first restored information, and the second transfer information is domain restored according to the second discrimination result to obtain second restored information;

[0049] According to the interference factor feature f i int The first restored information, and the first feature vector f i The second restored information, a cycle consistency loss function is determined

[0050] According to the first discrimination result and the second discrimination result, a loss function of an adversarial is determined

[0051] According to the first discrimination result and the second discrimination result, a generator loss function L disc is determined.

[0052] According to the cycle consistency loss function The adversarial loss function And the generator loss function L disc , a trained generation adversarial interference factor decoupling network is obtained.

[0053] One or more technical solutions provided in the present application have at least the following technical effects or advantages:

[0054] This invention separates identity features from interfering factors and enhances their robustness by suppressing intra-pedestrian differences caused by clothing textures and reducing dependence on interfering factors during the discriminative learning process. First, a dual-stream identity feature learning framework is proposed, consisting of an original image stream and a cloth-wiping stream. The cloth-wiping stream is implemented based on the image parsing method SCHP, thereby achieving clothing-independent identity feature extraction. The purpose of this dual-stream framework is to suppress fluctuations in the identity feature space caused by different fabric textures, thus obtaining a fabric-independent identity feature representation. A cloth-independent contrastive learning objective is introduced, constraining the distance between the features extracted from the original image and the clothing-wiping stream to the minimum intra-pedestrian distance. This helps to construct a more compact intra-pedestrian relationship structure between identity features of pedestrian images with different clothing. Furthermore, to reduce the negative impact of inaccurate clothing-wiping image parsing results, a cloth-independent contrastive learning objective is adaptively reweighted based on the discriminability of the clothing-wiping pedestrian image.

[0055] To more thoroughly mitigate the impact of interfering factors, a generative adversarial network for decoupling interfering factors is proposed. Specifically, an interfering factor identification branch is first designed to capture features oriented towards interfering factors. Simultaneously, a feature deentanglement module is designed to extract identity-related information from the features oriented towards interfering factors. This module learns in a cyclically consistent adversarial manner, ensuring the homogeneity of the generated features with the identity features. Ultimately, the discriminative learning process for identity features is made independent of interfering factors. Attached Figure Description

[0056] Figure 1 This is a flowchart of the method steps for re-identifying a person changing clothes based on isolating interfering factors, as provided in an embodiment of the present invention.

[0057] Figure 2 This is a framework diagram of a method for re-identifying people changing clothes based on isolating interfering factors, as provided in an embodiment of the present invention. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0059] Problem definition: Given a dataset of pedestrians changing clothes for re-identification, denoted as . Includes from M ped N images of pedestrians, these images are composed of M camThis was captured by a single camera. In real-world scenarios, the same pedestrian might appear wearing different clothes, assuming... Total number of clothes M clo .here, Let represent the i-th pedestrian image, where H, W, and C represent the height, width, and number of channels, respectively. Where y i ∈[1,M ped ], and They represent X respectively i The invention aims to learn a feature extractor that can effectively distinguish whether two images belong to the same pedestrian by ignoring differences in clothing and camera information. The goal is to learn a feature extractor that extracts features that can ignore differences in clothing and camera information.

[0060] This invention provides a method for re-identifying people changing clothes by isolating interfering factors, such as... Figure 1 and Figure 2 As shown, the method includes the following steps S101 to S104.

[0061] S101, retrieve the image to be queried and the image set in the image library.

[0062] S102, input the image to be queried into the trained clothing-changing model to obtain the feature vector to be queried, and input the image set of the image library into the trained clothing-changing model to obtain the feature vector set of the filtering results.

[0063] The clothing-changing model includes a dual-stream identity feature learning framework and a generative adversarial interference factor decoupling network. The dual-stream identity feature learning framework is used to extract identity features that are unrelated to the clothing fabric; the generative adversarial interference factor decoupling network is used to counteract the interference of interference factors on the identity features.

[0064] Specifically, in step S102, the first loss function L of the dual-stream identity feature learning framework is calculated. dual The steps include S1021 to S1023.

[0065] S1021, using a dual-stream identity feature learning framework to process the original image X i Perform raw image stream processing to obtain the raw stream processing result; wherein, the raw stream processing result includes: the first feature vector f i And the first M ped dimensional probability vector p i The dual-stream identity feature learning framework includes: a shared image encoder. Identity Feature Extractor pedestrian classifier

[0066] Specifically, in step S1021, the dual-stream identity feature learning framework is used to process the original image X. i Perform raw image stream processing to obtain the raw stream processing results, including:

[0067] (1) Utilizing a shared image encoder and identity feature extractor For the original image X i After processing, the first feature vector f is obtained. i ;

[0068] (2) Pedestrian classifier Using the first eigenvector f i Generate the first M ped dimensional probability vector p i Among them, M ped This represents the number of pedestrian categories in the training set;

[0069] For example, the clothing-changing model consists of a shared image encoder composed of the first four layers (denoted as...). ) and the identity feature extractor consisting of the fifth layer of blocks (denoted as ).exist Then, a pedestrian classifier is added, denoted as . Pedestrian recognition used to identify input images.

[0070] The computation process of the original image stream is as follows: Given an input image X i It can be done Generate a first feature vector f i The pedestrian classifier predicts the first M... ped A probability vector of dimension p is denoted as p i ,Right now

[0071] S1022, using a dual-stream identity feature learning framework to process the original image X i Perform clothing removal flow processing to obtain the clothing removal flow processing result; wherein, the clothing removal flow processing result includes: the second feature vector. Second M ped 3D probability vector

[0072] Specifically, in step S1022, the dual-stream identity feature learning framework is used to process the original image X. i The clothing wiping flow is processed to obtain the clothing wiping flow processing results, including:

[0073] (1) Generate a binary mask S using the SCHP model. i ;

[0074] (2) Using a binary mask S iAnd the image erasure formula, to obtain the original image X i The corresponding erased image X i ;

[0075] (3) Utilizing a shared image encoder and identity feature extractor For the erased image X i Feature extraction is performed to obtain the second feature vector.

[0076] (4) Pedestrian classifier Using the second eigenvector Generate the second M ped dimensional probability vector

[0077] For example, the SCHP model can be used to extract clothing regions, and this model can generate a binary mask S. i Then, through Get X i The erased image (represented as) The masking operation, where ⊙ represents element-wise multiplication, removes clothing texture. The clothing-erased image is extracted using a shared image encoder and identity feature extractor. The second eigenvector Right now Finally, input into the pedestrian classifier Generate a second M ped dimensional probability vector Right now

[0078] S1023, using the original stream processing results and the clothing erasure stream processing results, the first loss function L of the dual-stream identity feature learning framework is calculated. dual .

[0079] Specifically, in step S1023, the first loss function L of the dual-stream identity feature learning framework is calculated using the original stream processing results and the clothing erasure stream processing results. dual ,include:

[0080] (1) According to the first M ped dimensional probability vector p i Second M ped dimensional probability vector Calculate the loss function of the pedestrian classifier in the dual-stream identity feature learning framework to obtain the pedestrian classifier loss function L. ce Specifically:

[0081]

[0082] Where, pi [m] and They represent p respectively i and The m-th element.

[0083] (2) Determine the first eigenvector f i Second eigenvector Computational Identity Feature Extractor The loss function is used to obtain the identity feature extractor loss function L. tri Here, the loss function L of the pedestrian classifier mentioned above... ce This allows the clothing-changing model to focus on extracting features that can distinguish pedestrian identities. Furthermore, the triplet loss function, represented as L... tri To regularize the output of the identity feature extractor and improve the distinguishability between pedestrians:

[0084]

[0085] In the formula, Indicates with X i Image index set with the same pedestrian identity Indicates with X i A set of image indices representing pedestrians with different identities. dist(f) i ,f j Calculate f i and f j The cosine distance between them, i.e., dist(f) i ,f j )=1-cos(f i ,f j ), ∈ is a constant.

[0086] Compared to the original image, the clothing-erased image has a clear feature distribution. This affects the representation learning of the shared image encoder and identity feature extractor. To address this issue, a cross-stream matching objective based on a prototype model is designed to approximate the feature distribution of both the original and clothing-erased images.

[0087] (3) Construct the first feature vector f i The corresponding first prototype model m Constructing the second feature vector The corresponding second prototype model And the first prototype model was analyzed using the exponential moving average method. m Second prototype model The update is performed to obtain the updated first prototype model. m and the updated second prototype model

[0088] Here, firstly, in order to estimate the feature distribution of the m-th pedestrian, two prototype models were constructed by accumulating features from the original image and the image with the clothing erased, respectively. m and Update using the exponential moving average method. m and

[0089]

[0090] The second term of the formula represents the index function. It aims to calculate the average value of each pedestrian identity feature extracted by the current model, and then iteratively update the pedestrian prototype model using this average identity feature. γ is a hyperparameter controlling the update rate of the pedestrian prototype model.

[0091] (4) Based on the updated first prototype model m and the updated second prototype model Calculate the cross-flow matching loss function L of the prototype model mm Here, the prototype model cross-flow matching loss function L mm Represented as:

[0092]

[0093] Here, "·" represents the inner product operation. The loss function described above attracts the identity features of one stream to the prototype model of the identity feature estimation of another stream, thereby achieving the goal of reducing the feature distribution gap between the original image and the clothing-erased image.

[0094] To mitigate the differences caused by clothing texture, an adaptive, clothing-independent contrastive learning loss function objective is proposed, which helps to cultivate identity features with a more compact pedestrian internal relational structure.

[0095] (5) Calculate the original image X i Find the minimum pedestrian interior distance and use the minimum pedestrian interior distance through the interior distance constraint function L. cicl Constraining the first eigenvector f i Second eigenvector The distance between them; here, firstly, the minimum intra-pedestrian distance for each original image is estimated in the identity feature space. Then, it is used to constrain the distance between the identity features of the original image and the clothing-erased image:

[0096]

[0097] Here, sg(·) is the stopping gradient operation.

[0098] (6) According to the weight calculation formula, calculate the weight of the second M. ped dimensional probability vector The corresponding weighting coefficient ω i Here, ω is a measure of X. i The weighting coefficients for identity identifiability are determined by the pedestrian classifier. The prediction results are calculated using the following formula:

[0099]

[0100] τ is the temperature coefficient. L cicl The objective is to constrain the feature distance between an image of a pedestrian wearing any clothing and the corresponding image with the clothing erased to be less than the minimum feature distance of the images within the pedestrian's body. This means that under the same pedestrian recognition conditions, the difference in identity features caused by clothing texture will not exceed the normal feature fluctuations caused by other factors such as pose changes. Therefore, this loss function is beneficial for cultivating identity features independent of clothing texture variations. The motivation for introducing weighting coefficients is that the segmented clothing regions may be inaccurate, severely compromising the identity discriminability of the erased clothing images.

[0101] (7) Based on the pedestrian classifier loss function L ce Loss function L for identity feature extractor tri Prototype model cross-flow matching loss function L mm and interior distance constraint function L cicl Determine the first loss function L of the dual-stream identity feature learning framework. dual Specifically, the overall goal of forming a dual-stream identity feature learning framework is L dual :

[0102] L dual =L ce +L tri +β(L mm +L cicl );

[0103] Here, β is a constant. This framework ensures that the learned identity features are distinctive among different pedestrians, regardless of different clothing textures. In the following steps, identity features will be decoupled from confounding factors more thoroughly.

[0104] Generative adversarial interference factor decoupling network is proposed to address the issue that pedestrian recognition relies to some extent on clothing and camera parameters. For example, a person may frequently wear the same clothes or appear in the same camera view over a period of time. Pedestrian classifiers may inadvertently depend on these factors to complete the pedestrian recognition task, which hinders the exploration of robust identity features. To effectively counteract the influence of interference factors such as clothing and camera angle changes on identity features, a generative adversarial interference factor decoupling network is proposed. This network consists of an interference factor identification branch, a generative adversarial feature decoupling module, and an interference factor decoupling objective.

[0105] Specifically, in step S102, the interference factor identification branch in the generative adversarial interference factor decoupling network is used to analyze the original image X. i Interference features are extracted to obtain the interference factor features f. i int ,include:

[0106] (1) Utilizing a shared image encoder For the original image X i The process is performed to obtain the initial processing result;

[0107] (2) Utilizing interference factors to identify specific feature extractors in branches Feature extraction is performed on the initial processing results to obtain the interference factor features f. i int .

[0108] Specifically, in step S102, during the training of the clothing-changing model, the first loss function L of the dual-stream identity feature learning framework is calculated. dual Adjust the parameters of the two-stream identity feature learning framework; among them, calculate the first loss function L of the two-stream identity feature learning framework during training. dual ,include:

[0109] (1) The interference factor identification branch uses the first classifier and the second classifier to identify interference factor features f. i int Feature extraction is performed to obtain clothing p i clo and camera number p i cam Here, a specific feature extractor is placed after the shared image encoder, denoted as... This configuration is advantageous for extracting features specifically designed to identify interfering factors. Represented as X i Extracted interfering factor features f i int ,Right now Subsequently, two specialized classifiers were used to classify f i intThe clothing and camera numbers are determined, and the output p is obtained respectively. i clo and p i cam .

[0110] The classifiers are trained using a cross-entropy loss function, designed to fine-tune their accuracy in predicting clothing and camera numbers.

[0111] (2) Interference factor identification branch based on clothing p i clo and camera number p i cam Calculate the loss function in the generator adversarial factor decoupling network to obtain the cross-entropy loss function L. int-ce Cross-entropy loss function L for the interference factor identification branch int-ce Represented as:

[0112]

[0113] (3) Using a trained generative adversarial interference factor decoupling network, the interference factor features f i int Add to the first feature vector f i In this process, the intermediate feature vector is obtained;

[0114] (4) Determine the pedestrian classifier based on the intermediate feature vector. Allocation loss function L dise Here, a generative adversarial feature deentanglement module is used. The generative adversarial interference factor decoupling network is trained, including:

[0115] The feature deentanglement module utilizes the first feature generator Characteristics of interfering factors f i int Perform domain transfer to obtain the first transfer information, and then use the second feature generator. For the first eigenvector f i Perform a domain transfer to obtain the second transfer information;

[0116] Using the first feature generator The corresponding first discriminator The authenticity of the first transfer information is determined to obtain the first determination result; then, the second feature generator is used... The corresponding second discriminator The authenticity of the second transfer information is determined to obtain the second determination result;

[0117] Based on the first discrimination result, the first transfer information is restored by domain recovery to obtain the first restored information; and based on the second discrimination result, the second transfer information is restored by domain recovery to obtain the second restored information.

[0118] Based on the characteristics of the interfering factors f i int With the first recovered information and the first feature vector f i Using the second recovery information, determine the cycle consistency loss function.

[0119] Based on the first and second discrimination results, determine the adversarial loss function.

[0120] Based on the first and second discrimination results, determine the generator loss function L. disc ;

[0121] Based on the cycle consistency loss function Adversarial loss function and generator loss function L disc The trained generative adversarial interference factor decoupling network is obtained.

[0122] During the training of the clothing-changing model, the cross-entropy loss function L of the adversarial interference factor decoupling network is calculated. int-ce and the allocation loss function L dise The parameters of the anti-interference factor decoupling network are adjusted, wherein the cross-entropy loss function L of the anti-interference factor decoupling network is calculated. int-ce and the allocation loss function L dise ,include:

[0123] (1) Using the interference factor identification branch in the generator anti-interference factor decoupling network to analyze the original image X i Interference features are extracted to obtain the interference factor features f. i int The generated adversarial interference factor decoupling network includes an interference factor identification branch, a feature deentanglement module, and an interference factor decoupling target.

[0124] (2) Utilizing the characteristics of the aforementioned interference factors f i int Calculate the cross-entropy loss function L of the interference factor identification branch. int-ce The allocation loss function L of the target decoupled from the interference factors dise .

[0125] For example, the training process includes: to extract identity-related information from the features oriented towards interference factors, a generative adversarial feature deentanglement module is introduced. This involves constructing a feature generator. It extracts identity-related information from features related to interference factors, another feature generator. The aim is to extract features related to interfering factors from identity characteristics. Using... and Two discriminators evaluate the authenticity of features in the identity feature domain and the interference factor feature domain, respectively.

[0126] The discriminator is optimized using an objective function that enhances its ability to identify features generated by the feature generator and their corresponding domain features:

[0127]

[0128] In the objective function described above, the first two terms constrain... The latter two constraints on learning The learning.

[0129] The constraint loss function (L) applied to the generator gen (By an adversarial loss function) A cycle consistency loss function composition:

[0130]

[0131] Among them, the adversarial loss function It effectively facilitates feature domain transformation, while the cycle consistency loss function It maintains the cohesive relationship between identity and features oriented towards interference factors.

[0132] The goal is to decouple interfering factors. To eliminate the influence of interfering factors, the output probabilities of the pedestrian classifier are reassigned. Then, the reassigned output probabilities are constrained using the following objective:

[0133]

[0134] in This loss function ensures that the classification decision process is not affected by interfering factors, thereby guiding the feature extractor to ignore features that are related to these factors.

[0135] Combining all the formulas, the training loss function of the clothing-changing model, which optimizes network parameters (excluding the generative adversarial feature deentanglement module), is expressed as:

[0136] L = L dual +αL int-ce +ηL dise ;

[0137] Among them, L dualL represents the loss function of the dual-stream identity feature learning framework; int-ce L represents the cross-entropy loss function; dise Represents the allocation loss function L dise ; α and η are both constants.

[0138] S103, calculate the distance between the feature vector to be queried and each feature vector in the set of filtered results, and obtain multiple distance sets.

[0139] For example, identify a set of images from a library (represented as...) ) and a set of query images (represented as Correct matching between ).

[0140] This is achieved by calculating the distance between all library images and the query image pair. G i and Q j Input the feature extractor and generate feature vectors g respectively. i and q j Subsequently, using a camera classifier trained as part of the interference factor deentanglement module, the values ​​represented as... and The camera tag is assigned to g i and q j Given that images taken with the same camera typically have higher visual similarity than those taken with different cameras, these predicted camera labels are used to adjust g. i and q j The cosine distance between them. This distance is calculated and denoted as d. ij .as follows:

[0141] S104, determine the minimum value among multiple distance sets, and determine the feature vector of the filtering result corresponding to the minimum value.

[0142] Here, if the number of cameras is inferred and Consistent, g is set through a constant threshold. i and q j The cosine distance between them is reduced to δ. Using the above distance measurement, it is easy to find the image in the library that is most similar to each query image, thus completing the pedestrian re-identification task.

[0143] S105, determine the image set corresponding to the image to be queried based on the feature vector of the filtering result.

[0144] This invention proposes a method for re-identifying individuals changing clothes by isolating interfering factors, thereby obtaining a discriminative fabric-independent identity feature representation. Specifically, an adaptive, clothing-independent contrastive learning objective is introduced to suppress fluctuations in clothing texture within the identity feature space. To further enhance the robustness of identity features, a generative adversarial interference factor decoupling network is introduced, which reduces the dependence on interfering factor-related identity information during the identity feature discriminative learning process. Extensive experimental results strongly demonstrate the effectiveness of the proposed method in learning CC-ReID discriminative features, outperforming state-of-the-art methods.

[0145] The effects of this invention can be further illustrated by the following simulation experiments.

[0146] Datasets: The datasets used are three commonly used CC-ReID datasets (PRCC, VC-Clothes, and LTCC). The PRCC dataset consists of 33,689 images from 221 identities. Everyone in shots A and B is wearing the same clothing. For camera C, people are wearing different clothes, and images were taken at different times. VC-Clothes is a synthetic dataset containing 19,060 images rendered by the GTA5 game engine. These images were captured from 512 pedestrian identities using 4 cameras. Each person has 1 to 3 outfits. LTCC is an indoor CC-ReID dataset that captured 17,138 images from 152 identities across 12 camera views, featuring 478 different outfits.

[0147] Evaluation Criteria: During the testing phase, standard rank-1 accuracy and mean accuracy (mAP) were used for evaluation. Based on previous research, the following fabric-changing settings were used: the gallery setting only included fabric-changing samples. For the three datasets, the rank-1 accuracy and mAP under the fabric-changing settings were reported.

[0148] Table 1. Experimental results under the clothing changing setting.

[0149]

[0150]

[0151] The various embodiments described in this specification are presented in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. All or part of this invention can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, mobile communication terminals, multiprocessor systems, microprocessor-based systems, programmable electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0152] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the present invention.

Claims

1. A method for re-identifying personnel changing clothes by isolating interfering factors, characterized in that, include: Retrieve the image to be queried and the image set from the gallery; The image to be queried is input into the trained clothing-changing model to obtain the query feature vector, and the image set from the image library is input into the trained clothing-changing model to obtain the filtered result feature vector set; wherein, the clothing-changing model includes: a dual-stream identity feature learning framework and a generative adversarial interference factor decoupling network, the dual-stream identity feature learning framework is used to extract identity features unrelated to clothing fabric; the generative adversarial interference factor decoupling network is used to counteract the interference of interference factors on the identity features; During the training of the clothing-changing model, the first loss function of the dual-stream identity feature learning framework is calculated. Adjust the parameters of the two-stream identity feature learning framework; among them, calculate the first loss function of the two-stream identity feature learning framework during training. ,include: Using a dual-stream identity feature learning framework on the original image Perform raw image stream processing to obtain raw stream processing results; wherein, the raw stream processing results include: a first feature vector. and first dimensional probability vector ;in, This represents the number of pedestrian categories in the training set; The original image was processed using a dual-stream identity feature learning framework. Perform clothing removal flow processing to obtain clothing removal flow processing results; wherein, the clothing removal flow processing results include: a second feature vector. Second dimensional probability vector ; Using the original stream processing results and the clothing erasure stream processing results, the first loss function of the dual-stream identity feature learning framework is calculated. ; During the training of the clothing-changing model, the cross-entropy loss function of the adversarial interference factor decoupling network is calculated. and allocation loss function The parameters of the anti-interference factor decoupling network are adjusted, wherein the cross-entropy loss function of the anti-interference factor decoupling network is calculated. and allocation loss function ,include: The interference factor identification branch in the generative adversarial interference factor decoupling network is used to analyze the original image. Interference features are extracted to obtain the characteristics of interfering factors. ; Utilizing the characteristics of the interference factors Calculate the cross-entropy loss function of the interference factor identification branch. The allocation loss function of the target decoupled from the interference factors. ; Here, the use of the characteristics of the interference factors Calculate the cross-entropy loss function of the interference factor identification branch. The allocation loss function of the target decoupled from the interference factors. ,include: The interference factor identification branch utilizes a first classifier and a second classifier to analyze the interference factor features. Feature extraction is performed to obtain clothing and camera number ; The interference factor identification branch is based on the clothing and the camera number Calculate the cross-entropy loss function of the generative adversarial interference factor decoupling network. ; The characteristics of the interference factors Supplement to the first feature vector In this process, the intermediate feature vector is obtained; A pedestrian classifier is determined based on the intermediate feature vector. Allocation loss function ; The training method for the generative adversarial interference factor decoupling network includes: The feature deentanglement module utilizes a first feature generator. Features of the interfering factors Perform domain transfer to obtain the first transfer information, and then use the second feature generator. For the first feature vector Perform a domain transfer to obtain the second transfer information; Using the first feature generator The corresponding first discriminator The authenticity of the first transfer information is determined to obtain a first determination result; the second feature generator is then used to... The corresponding second discriminator The authenticity of the second transfer information is determined to obtain a second determination result; Based on the first discrimination result, the first transfer information is restored by domain recovery to obtain the first restored information, and based on the second discrimination result, the second transfer information is restored by domain recovery to obtain the second restored information; Based on the characteristics of the interference factors With the first recovery information and the first feature vector Based on the second recovery information, determine the cycle consistency loss function. ; Based on the first and second discrimination results, determine the adversarial loss function. ; Based on the first discrimination result and the second discrimination result, determine the generator loss function. ; According to the cycle consistency loss function The loss function of the adversarial and the generator loss function The decoupling network for generating adversarial interference factors is trained. Calculate the distance between the feature vector to be queried and each feature vector in the set of filtered result feature vectors to obtain a distance set consisting of multiple distances; Determine the minimum value in the distance set, and determine the feature vector of the filtering result corresponding to the minimum value; Based on the feature vector of the filtering results, the target image corresponding to the image to be queried in the image library is determined.

2. The method for re-identifying a person changing clothes based on isolating interfering factors, as described in claim 1, is characterized in that... The training loss function of the clothing-changing model is expressed as: ； in, This represents the first loss function of the dual-stream identity feature learning framework; Represents the cross-entropy loss function; Represents the allocation loss function; and All are constants.

3. The method for re-identifying a person changing clothes based on isolating interfering factors, as described in claim 1, is characterized in that... The original image is processed using a dual-stream identity feature learning framework. Perform raw image stream processing to obtain the raw stream processing results, including: Using a shared image encoder and identity feature extractor For the original image The process is performed to obtain the first feature vector. And by pedestrian classifier Using the first feature vector Generate the first 3D probability vector , to the feature vector and the first 3D probability vector As the result of the original stream processing.

4. The method for re-identifying a person changing clothes based on isolating interfering factors, as described in claim 1, is characterized in that... The original image is processed using a dual-stream identity feature learning framework. The clothing wiping flow is processed to obtain the clothing wiping flow processing results, including: Generate a binary mask using the SCHP model. ; Using the binary mask And the image erasure formula, to obtain the original image Corresponding erased image ; Using a shared image encoder and identity feature extractor For the erased image Feature extraction is performed to obtain the second feature vector. And by pedestrian classifier Using the second feature vector , generate the second dimensional probability vector , the second feature vector and the second dimensional probability vector As a result of the clothing wiping process.

5. The method for re-identifying a person changing clothes based on isolating interfering factors according to claim 1, characterized in that, The first loss function of the dual-stream identity feature learning framework is calculated using the original stream processing result and the clothing erasure stream processing result. ,include: According to the first 3D probability vector and the second 3D probability vector Calculate the loss function of the pedestrian classifier in the dual-stream identity feature learning framework. ; Determine the first feature vector and the second feature vector Computational Identity Feature Extractor loss function ; Construct with the first feature vector The corresponding first prototype model Construct the second feature vector The corresponding second prototype model And the first prototype model was analyzed using the exponential moving average method. and the second prototype model The updated first prototype model is obtained by performing an update. and the updated second prototype model ; According to the updated first prototype model and the updated second prototype model Calculate the cross-flow matching loss function of the prototype model. ; Calculate the original image The minimum pedestrian interior distance is calculated, and this minimum pedestrian interior distance is used to pass the interior distance constraint function. Constrain the first feature vector and the second feature vector The distance between them; wherein, the minimum pedestrian distance is: the minimum distance between different images of pedestrians with the same identity; According to the weight calculation formula, the result is calculated to be the same as the second... dimensional probability vector corresponding weighting coefficients ; according to , , and the aforementioned interior distance constraint function Determine the first loss function of the dual-stream identity feature learning framework. .

6. The method for re-identifying a person changing clothes based on isolating interfering factors according to claim 1, characterized in that, The interference factor identification branch in the Generative Adversarial Factor Decoupling Network is used to analyze the original image. Interference features are extracted to obtain the characteristics of interfering factors. ,include: Using a shared image encoder For the original image The process is performed to obtain the initial processing result; Utilizing the interference factor identification branch for specific feature extractors Feature extraction is performed on the initial processing results to obtain the interference factor features. .

Images