Method for realizing re-identification of clothes changing personnel based on interference factor isolation

Abstract

The invention discloses a method for realizing re-identification of a clothes changing person based on interference factor isolation, and relates to the technical field of intelligent image processing, and the method comprises the steps: obtaining a to-be-queried image and an image library image set; inputting a to-be-queried image into the trained clothes changing model to obtain a to-be-queried feature vector, and inputting the image library image set into the trained clothes changing model to obtain a screening result feature vector set; respectively calculating the distance between the to-be-queried feature vector and each screening result feature vector in the screening result feature vector set to obtain a plurality of distance sets; determining a minimum value in the plurality of distance sets, and determining a screening result feature vector corresponding to the minimum value; determining an image library image set image corresponding to the to-be-queried image according to the screening result feature vector; the extracted features can neglect the difference between the clothes and the camera, so that whether the two images belong to the same pedestrian or not is effectively distinguished.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence image processing technology, and in particular to a method for re-identifying a person changing clothes based on isolating interference factors. Background Art

[0002] The pedestrian re-identification (ReID) task aims to train a deep model to recognize the same pedestrian in short-term scenarios with limited temporal and spatial spans. The clothing-changing pedestrian re-identification (CC-ReID) task focuses on more challenging and realistic long-term surveillance scenarios, retrieving images of pedestrians whose clothes may change. Obviously, in addition to general challenges such as moderate view / pose changes, occlusions, and low resolution, CC-ReID faces more severe challenges such as clothing changes caused by long-term use and drastic field of view changes. Therefore, pedestrians' clothing and camera views are often associated with identity information, which interferes with the learning of robust identity features. Separating identity features from these interference factors is the key to solving CC-ReID.

[0003] Existing CC-ReID methods mainly focus on learning feature representations related to stable identities rather than distractors such as cloth and environmental features. These methods can be roughly divided into unimodal and multimodal methods. Unimodal methods only learn cloth-independent identity features from RGB images, which is not enough to eliminate the influence of distractors. Multimodal methods improve the performance by exploring other synthetic data modalities such as shape, gait motion, clothing-erased images, or cloth-swapped images to capture cloth-independent identity features. These methods mainly transform the distractor disentanglement problem into a cross-modal feature alignment problem from a data-driven perspective. However, methods using clothing-erased images or cloth-swapped images lack intrinsic consideration of the impact of clothing features. Summary of the invention

[0004] The present invention provides a method for re-identifying a person who has changed clothes by isolating interference factors, thereby achieving the goal of extracting features that ignores differences in clothes and cameras, thereby effectively distinguishing whether two images belong to the same person.

[0005] The present invention provides a method for re-identifying a person changing clothes based on isolating interference factors, the method comprising:

[0006] Get the image to be queried and the gallery image set;

[0007] The image to be queried is input into the trained clothing-changing model to obtain the feature vector to be queried, and the image set of the gallery is input into the trained clothing-changing model to obtain the feature vector set of the screening result; wherein the clothing-changing model comprises: a two-stream identity feature learning framework and a generative adversarial interference factor decoupling network, wherein the two-stream identity feature learning framework is used to extract identity features that are unrelated to clothing fabrics; and the generative adversarial interference factor decoupling network is used to offset the interference of interference factors on the identity features;

[0008] respectively calculating the distance between the feature vector to be queried and each of the feature vectors of the screening result in the feature vector set, and obtaining a distance set consisting of a plurality of distances;

[0009] Determine a minimum value in the distance set, and determine a screening result feature vector corresponding to the minimum value;

[0010] According to the screening result feature vector, a target image in the gallery image set corresponding to the image to be queried is determined.

[0011] In a possible implementation, during the training of the clothing-changing model, the first loss function L of the two-stream identity feature learning framework is calculated. dual Adjust the parameters of the two-stream identity feature learning framework; wherein the first loss function L of the two-stream identity feature learning framework in training is calculated dual ,include:

[0012] The original image X is trained using a two-stream identity feature learning framework. i Process the original image stream to obtain the original stream processing result; wherein the original stream processing result includes: the first feature vector f i and the first M ped dimensional probability vector p i ; Among them, M ped Indicates the number of pedestrian categories in the training set;

[0013] The original image X is trained using a two-stream identity feature learning framework. i Performing clothes erasing stream processing to obtain clothes erasing stream processing results; wherein the clothes erasing stream processing results include: the second feature vector and the second M ped dimensional probability vector

[0014] The first loss function L of the dual-stream identity feature learning framework is calculated using the original stream processing results and the clothing erasure stream processing results. dual .

[0015] In a possible implementation, during the training of the clothing-changing model, the cross entropy loss function L of the anti-interference factor decoupling network is calculated. int-ce And the distribution loss function L dise , adjust the parameters of the generated adversarial interference factor decoupling network, wherein the cross entropy loss function L of the adversarial interference factor decoupling network is calculated int-ce And the distribution loss function L dise ,include:

[0016] The interference factor recognition branch in the generative adversarial interference factor decoupling network is used to identify the original image X i Extract interference features and obtain interference factor features f i int ; Wherein, the generation of the interference factor decoupling network against interference factors includes an interference factor identification branch, a feature detangling module and an interference factor decoupling target;

[0017] Using the interference factor feature f i int Calculate the cross entropy loss function L of the interference factor identification branch int-ce And the allocation loss function L of the interference factor decoupling target dise .

[0018] In a possible implementation, the training loss function of the clothing changing model is expressed as:

[0019] L=L dual +αL int-ce +ηL dise ;

[0020] Among them, L dual represents the loss function of the two-stream identity feature learning framework; L int-ce represents the cross entropy loss function; L dise Denotes the allocation loss function L dise ; α and η are both constants.

[0021] In a possible implementation, the two-stream identity feature learning framework is used to learn the original image X i Perform raw image stream processing to obtain raw stream processing results, including:

[0022] Leveraging a shared image encoder and identity feature extractor For the original image X i After processing, the first eigenvector f is obtained i ;

[0023] Pedestrian Classifier Using the first eigenvector f iGenerate the first M ped dimensional probability vector p i , the feature vector f i and the first M ped dimensional probability vector p i Process the result as a raw stream.

[0024] In a possible implementation, the two-stream identity feature learning framework is used to learn the original image X i Perform clothes erasing stream processing to obtain clothes erasing stream processing results, including:

[0025] Generate a binary mask S through the SCHP model i ;

[0026] Using the binary mask S i And erase the image formula to get the original image X i The corresponding erased image X i ;

[0027] Leveraging a shared image encoder and identity feature extractor For the erased image X i Perform feature extraction to obtain the second feature vector

[0028] Pedestrian Classifier Using the second eigenvector Generate the second M ped dimensional probability vector The second eigenvector and the second M ped dimensional probability vector Process the result as a clothes erase stream.

[0029] In a possible implementation, the first loss function L of the dual-stream identity feature learning framework is calculated using the original stream processing result and the clothing erasure stream processing result. dual ,include:

[0030] According to the first M ped dimensional probability vector p i and the second M ped dimensional probability vector Calculate the loss function of the pedestrian classifier in the two-stream identity feature learning framework to obtain the pedestrian classifier loss function L ce ;

[0031] Determine the first eigenvector f i and the second eigenvector Compute identity feature extractor The loss function of the identity feature extractor is obtained by tri ;

[0032] Construct the first eigenvector f i The corresponding first prototype model m , construct the second eigenvector with The corresponding second prototype model And the first prototype model is analyzed by exponential moving average method. m And the second prototype model Update to obtain an updated first prototype model. m and an updated second prototype model

[0033] According to the updated first prototype model. m and the updated second prototype model Calculate the prototype model cross-flow matching loss function L mm ;

[0034] Calculate the original image X i The minimum pedestrian inner distance is calculated by using the inner distance constraint function L cicl Constrain the first eigenvector f i and the second eigenvector The distance between them; wherein the minimum pedestrian distance is: the minimum distance between different images of pedestrians with the same identity;

[0035] According to the weight calculation formula, the second M ped dimensional probability vector The corresponding weighting coefficient ω i ;

[0036] According to the pedestrian classifier loss function L ce , identity feature extractor loss function L tri , the prototype model cross-flow matching loss function L mm and the inner distance constraint function L cicl , determine the first loss function L of the two-stream identity feature learning framework dual .

[0037] In a possible implementation, the interference factor identification branch in the generative adversarial interference factor decoupling network is used to identify the original image X i Extract interference features and obtain interference factor features f i int ,include:

[0038] Leveraging a shared image encoder For the original image X i Perform processing to obtain an initial processing result;

[0039] Utilize the interference factor to identify the specific feature extractor in the branch Perform feature extraction on the initial processing result to obtain interference factor feature f i int .

[0040] In a possible implementation, the interference factor feature f i int Calculate the cross entropy loss function L of the interference factor identification branch int-ce And the allocation loss function L of the interference factor decoupling target dise ,include:

[0041] The interference factor identification branch uses the first classifier and the second classifier to classify the interference factor feature f i int Perform feature extraction and obtain clothing p i clo and camera number p i cam ;

[0042] The interference factor identification branch is based on the clothing p i clo and the camera number p i cam Calculate the loss function in the decoupling network of the generated adversarial interference factor to obtain the cross entropy loss function L int-ce ;

[0043] Using the trained generative adversarial interference factor decoupling network, the interference factor feature f i int Supplement to the first eigenvector f i In the equation, we get the intermediate eigenvector;

[0044] Determine a pedestrian classifier according to the intermediate feature vector The allocation loss function L dise .

[0045] In one possible implementation, training a generative adversarial interference factor decoupling network includes:

[0046] The feature disentanglement module uses the first feature generator The interference factor feature f i int Perform domain transfer to obtain first transfer information, and use the second feature generator For the first eigenvector f i Performing domain transfer and obtaining second transfer information;

[0047] Using the first feature generator The corresponding first discriminator The authenticity of the first transfer information is judged to obtain a first judgment result; using the second feature generator The corresponding second discriminator Determining the authenticity of the second transfer information to obtain a second determination result;

[0048] Performing domain recovery on the first transfer information according to the first discrimination result to obtain first recovery information, and performing domain recovery on the second transfer information according to the second discrimination result to obtain second recovery information;

[0049] According to the interference factor characteristics f i int and the first restored information, and the first eigenvector f i With the second recovered information, determine the cycle consistency loss function

[0050] According to the first discrimination result and the second discrimination result, determine the adversarial loss function

[0051] According to the first discrimination result and the second discrimination result, determine the generator loss function L disc ;

[0052] According to the cycle consistency loss function The adversarial loss function and the generator loss function L disc , and obtain the trained generative adversarial interference factor decoupling network.

[0053] One or more technical solutions provided in the present invention have at least the following technical effects or advantages:

[0054] The present invention separates identity features from interference factors, and enhances the robustness of identity features by suppressing the intra-person differences caused by clothing textures and reducing the dependence on interference factors in the process of discriminative learning of identification features. Firstly, a two-stream identity feature learning framework is proposed, which consists of an original image stream and a wipe stream. The wipe stream is implemented based on the image parsing method SCHP, thereby realizing the extraction of identity features that are independent of clothing. The purpose of this two-stream framework is to suppress the fluctuation of different cloth textures in the identity feature space, so as to obtain a cloth-independent identity feature representation. A cloth-independent contrastive learning objective is introduced, which constrains the distance between the features extracted from the original image and the clothing-erased stream to the minimum intra-person distance. This helps to build a more compact intra-person relationship structure between the identity features of pedestrian images of different clothing. In addition, in order to reduce the negative impact of inaccurate analysis results of clothing-erased images, it is further proposed to adaptively reweight the cloth-independent contrastive learning objective according to the distinguishability of clothing-erased pedestrian images.

[0055] In order to more thoroughly mitigate the impact of interference factors, a generative adversarial interference factor decoupling network is also proposed. Specifically, a interference factor identification branch is first designed to capture interference factor-oriented features. At the same time, a feature disentanglement module is designed to extract identity-related information from interference factor-oriented features. This module is learned in a cycle-consistent adversarial manner to ensure the homogeneity of the generated features and identity features. Finally, the discriminative learning process of identity features is made independent of interference factors. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A flowchart of the method steps for re-identifying a person changing clothes based on isolating interference factors provided by an embodiment of the present invention;

[0057] Figure 2 A framework diagram of a method for re-identifying a person changing clothes based on isolating interference factors provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] Problem definition: Given a clothing-changing pedestrian re-identification dataset, denoted as Contains from Μ ped N images of pedestrians, these images are collected by M camIn real scenes, the same pedestrian may appear in different clothes. Total number of clothes in Μ clo .here, represents the i-th pedestrian image, where H, W, and C represent the height, width, and number of channels, respectively. i ∈[1,Μ ped ], and Respectively represent X i The goal of the present invention is to learn a feature extractor so that the extracted features can ignore the differences in clothes and cameras, thereby effectively distinguishing whether two images belong to the same pedestrian.

[0060] The present invention provides a method for re-identifying a person who changes clothes based on isolating interference factors. Figure 1 and Figure 2 As shown, the method includes the following steps S101 to S104.

[0061] S101, obtaining a query image and a gallery image set.

[0062] S102, inputting the image to be queried into the trained clothing-changing model to obtain a feature vector to be queried, and inputting the gallery image set into the trained clothing-changing model to obtain a screening result feature vector set.

[0063] Among them, the clothing changing model includes: a two-stream identity feature learning framework and a generative adversarial interference factor decoupling network. The two-stream identity feature learning framework is used to extract identity features that are unrelated to clothing fabrics; the generative adversarial interference factor decoupling network is used to offset the interference of interference factors on the identity features.

[0064] Specifically, in step S102, the first loss function L of the dual-stream identity feature learning framework is calculated dual , including the following steps S1021 to S1023.

[0065] S1021, using the two-stream identity feature learning framework to learn the original image X i Process the original image stream to obtain the original stream processing result; wherein the original stream processing result includes: the first feature vector f i and the first M ped dimensional probability vector p i ; The two-stream identity feature learning framework includes: shared image encoder Identity Feature Extractor and pedestrian classifier

[0066] Specifically, in step S1021, the original image X is trained using a dual-stream identity feature learning framework. i Perform raw image stream processing to obtain raw stream processing results, including:

[0067] (1) Using a shared image encoder and identity feature extractor For the original image X i After processing, the first eigenvector f is obtained i ;

[0068] (2) Pedestrian Classifier Using the first eigenvector f i Generate the first M ped dimensional probability vector p i Among them, M ped Indicates the number of pedestrian categories in the training set;

[0069] Exemplarily, the clothes-changing model consists of a shared image encoder consisting of the first four layers of blocks (denoted as ) and the identity feature extractor consisting of the fifth layer of blocks (denoted as ).exist Then add a pedestrian classifier, denoted as Used to identify people in input images.

[0070] The calculation process of the original image flow is as follows: given an input image X i , can be Generate a first eigenvector f i . The pedestrian classifier predicts the first M ped dimensional probability vector, denoted as p i ,Right now

[0071] S1022, using the two-stream identity feature learning framework to learn the original image X i Perform clothing erasure flow processing to obtain clothing erasure flow processing results; wherein the clothing erasure flow processing results include: the second feature vector and the second M ped dimensional probability vector

[0072] Specifically, in step S1022, the original image X is trained using a dual-stream identity feature learning framework. i Perform clothes erasing stream processing to obtain clothes erasing stream processing results, including:

[0073] (1) Generate a binary mask S through the SCHP model i ;

[0074] (2) Using binary mask S iand erased image formula, we get the same image as the original image X i The corresponding erased image X i ;

[0075] (3) Using a shared image encoder and identity feature extractor To erase image X i Perform feature extraction to obtain the second feature vector

[0076] (4) Pedestrian Classifier Using the second eigenvector Generate the second M ped dimensional probability vector

[0077] For example, the clothing area is extracted using the SCHP model, which can generate a binary mask S i Then, by Get X i The erased image (expressed as ), where ⊙ represents the element-wise product. Such a masking operation can remove the clothing texture. The clothing-erased image is extracted by sharing the image encoder and the identity feature extractor. The second eigenvector of Right now Finally, we input the pedestrian classifier Generate a second M ped dimensional probability vector Right now

[0078] S1023, using the original stream processing results and the clothes erasing stream processing results, calculate the first loss function L of the dual-stream identity feature learning framework dual .

[0079] Specifically, in step S1023, the first loss function L of the dual-stream identity feature learning framework is calculated using the original stream processing results and the clothes erasing stream processing results. dual ,include:

[0080] (1) According to the first M ped dimensional probability vector p i and the second M ped dimensional probability vector Calculate the loss function of the pedestrian classifier in the two-stream identity feature learning framework and obtain the pedestrian classifier loss function L ce Specific:

[0081]

[0082] Among them, pi [m] and Respectively represent p i and The mth element.

[0083] (2) Determine the first eigenvector f i and the second eigenvector Compute identity feature extractor The loss function of the identity feature extractor is obtained by tri ; Here, the pedestrian classifier loss function L ce The clothing-changing model can be made to focus on extracting features that can distinguish pedestrian identities. In addition, the triplet loss function is expressed as L tri To regularize the output of the identity feature extractor to improve the distinguishability between pedestrians:

[0084]

[0085] In the formula, Indicates that X i The image index set with the same pedestrian identity, Indicates that X i The image index set with different pedestrian identities. dist(f i ,f j ) Calculate f i and f j The cosine distance between i ,f j )=1-cos(f i ,f j ),∈ is a constant.

[0086] Compared with the original images, the clothes-erased images have a clear feature distribution. This affects the representation learning of the shared image encoder and identity feature extractor. To address this issue, a cross-stream matching objective based on a prototype model is designed to approximate the feature distribution of the original images and clothes-erased images.

[0087] (3) Construct the first eigenvector f i The corresponding first prototype model m , construct the second eigenvector The corresponding second prototype model The first prototype model is analyzed by exponential moving average method. m and the second prototype model Update to obtain an updated first prototype model. m and an updated second prototype model

[0088] Here, firstly, in order to estimate the feature distribution of the m-th pedestrian, two prototype models are constructed by accumulating the features of the original image and the clothes-erased image, namely, m and Updated by exponential moving average method. m and

[0089]

[0090] Represents the indicator function. The second term of the formula is to calculate the average value of each pedestrian identity feature extracted by the current model, and use the average identity feature to iteratively update the pedestrian prototype model. γ is a hyperparameter that controls the update rate of the pedestrian prototype model.

[0091] (4) Based on the updated first prototype model m and an updated second prototype model Calculate the prototype model cross-flow matching loss function L mm ; Here, the prototype model cross-flow matching loss function L mm It is expressed as:

[0092]

[0093] Here, “·” is the inner product operation. The above loss function attracts the identity features of one stream to the prototype model of the identity feature estimation of another stream, thereby achieving the goal of reducing the feature distribution gap between the original image and the clothes-erased image.

[0094] To alleviate the discrepancy caused by clothing texture, an adaptive clothing-independent contrastive learning loss function objective is proposed, which helps to cultivate identity features with more compact internal relation structures of pedestrians.

[0095] (5) Calculate the original image X i The minimum pedestrian inner distance is used to constrain the inner distance function L cicl Constrain the first eigenvector f i and the second eigenvector Here, first, the minimum intra-person distance for each original image is estimated in the identity feature space. Subsequently, it is used to constrain the distance between the identity features of the original image and the clothing-erased image:

[0096]

[0097] Among them, sg(·) is the stop gradient operation.

[0098] (6) According to the weight calculation formula, the second M ped dimensional probability vector The corresponding weighting coefficient ω i ; Here, ω is a measure of X i The weighted coefficient of identity discernibility is given by the pedestrian classifier, i.e. The prediction results are calculated using the following formula:

[0099]

[0100] τ Temperature coefficient. L cicl The goal is: given an image where a pedestrian is wearing an arbitrary piece of clothing, the feature distance between this image and the corresponding clothing-erased image is constrained to be less than the minimum feature distance of the pedestrian's internal image. This means that under the same pedestrian recognition conditions, the difference in identity features brought by clothing texture will not exceed the normal feature fluctuations caused by other factors such as posture changes. Therefore, this loss function is conducive to cultivating identity features that are independent of clothing texture changes. The motivation for introducing weighting coefficients is that the segmented clothing area may be inaccurate, resulting in serious damage to the identity discriminability of clothing-erased images.

[0101] (7) According to the pedestrian classifier loss function L ce , identity feature extractor loss function L tri , prototype model cross-flow matching loss function L mm and the inner distance constraint function L cicl , determine the first loss function L of the two-stream identity feature learning framework dual Specifically, the overall goal of forming a two-stream identity feature learning framework is L dual :

[0102] L dual =L ce +L tri +β(L mm +L cicl );

[0103] where β is a constant. This framework ensures that the learned identity features are distinguishable between different pedestrians, regardless of different clothing textures. In the next step, the identity features will be more thoroughly decoupled from interference factors.

[0104] Generate an adversarial interference factor decoupling network,Pedestrian recognition depends on clothing and camera parameters to a certain extent. For example, a person may often wear the same clothes or appear in the same camera view for a period of time. Pedestrian classifiers may inadvertently rely on these factors to complete pedestrian recognition tasks, which hinders the exploration of robust identity features. In order to effectively offset the influence of interference factors such as clothing and camera perspective changes on identity features, a generative adversarial interference factor decoupling network is proposed, which consists of an interference factor identification branch, a generative adversarial feature decoupling module and an interference factor decoupling target.

[0105] Specifically, in step S102, the interference factor identification branch in the generative adversarial interference factor decoupling network is used to analyze the original image X i Extract interference features and obtain interference factor features f i int ,include:

[0106] (1) Using a shared image encoder For the original image X i Perform processing to obtain an initial processing result;

[0107] (2) Using interference factors to identify specific feature extractors in the branch Perform feature extraction on the initial processing results to obtain the interference factor feature f i int .

[0108] Specifically, in step S102, during the training of the clothing-changing model, the first loss function L of the dual-stream identity feature learning framework is calculated. dual Adjust the parameters of the two-stream identity feature learning framework; wherein the first loss function L of the two-stream identity feature learning framework in training is calculated dual ,include:

[0109] (1) The interference factor identification branch uses the first classifier and the second classifier to classify the interference factor feature f i int Perform feature extraction and obtain clothing p i clo and camera number p i cam ; Here, a specific feature extractor is placed after the shared image encoder, denoted as This configuration is conducive to extracting features specifically for identifying interference factors. i Extracted interference factor features f i int ,Right now Subsequently, two specialized classifiers are used to extract the i intDetermine the clothing and camera numbers in the output p i clo and p i cam .

[0110] The training of the classifiers is governed by a cross entropy loss function, which aims to fine-tune their accuracy in predicting clothing and camera numbers.

[0111] (2) Interference factor identification branch based on clothing p i clo and camera number p i cam Calculate the loss function in the decoupling network of the generated adversarial interference factor and obtain the cross entropy loss function L int-ce ; Cross entropy loss function L of interference factor identification branch int-ce It is expressed as:

[0112]

[0113] (3) Using the trained generative adversarial interference factor decoupling network, the interference factor feature f i int Add to the first eigenvector f i In the equation, we get the intermediate eigenvector;

[0114] (4) Determine the pedestrian classifier based on the intermediate feature vector The allocation loss function L dise Here, the adversarial feature disentanglement module is generated. The training of the generative adversarial interference factor decoupling network includes:

[0115] The feature disentanglement module uses the first feature generator Characteristics of interference factors i int Perform domain transfer, obtain first transfer information, and use the second feature generator For the first eigenvector f i Performing domain transfer and obtaining second transfer information;

[0116] Using the first feature generator The corresponding first discriminator The authenticity of the first transfer information is judged to obtain a first judgment result; using the second feature generator The corresponding second discriminator Determining the authenticity of the second transfer information to obtain a second determination result;

[0117] Performing domain recovery on the first transfer information according to the first discrimination result to obtain first recovery information, and performing domain recovery on the second transfer information according to the second discrimination result to obtain second recovery information;

[0118] According to the characteristics of interference factors f i int With the first restored information, and the first eigenvector f i With the second recovery information, determine the cycle consistency loss function

[0119] According to the first discrimination result and the second discrimination result, determine the adversarial loss function

[0120] According to the first discrimination result and the second discrimination result, determine the generator loss function L disc ;

[0121] According to the cycle consistency loss function Adversarial Loss Function And the generator loss function L disc , and obtain the trained generative adversarial interference factor decoupling network.

[0122] In the process of training the clothing-changing model, the cross entropy loss function L of the adversarial interference factor decoupling network is calculated. int-ce And the distribution loss function L dise , adjust the parameters of the generated adversarial interference factor decoupling network, wherein the cross entropy loss function L of the adversarial interference factor decoupling network is calculated int-ce And the distribution loss function L dise ,include:

[0123] (1) Generate adversarial interference factor decoupling network to identify interference factors in the original image X i Extract interference features and obtain interference factor features f i int ; Wherein, the generated interference factor decoupling network includes an interference factor identification branch, a feature detangling module and an interference factor decoupling target.

[0124] (2) Using the interference factor feature f i int Calculate the cross entropy loss function L of the interference factor identification branch int-ce And the allocation loss function L of the interference factor decoupling target dise .

[0125] Exemplarily, the training process includes: In order to extract identity-related information from the above-mentioned interference factor-oriented features, a generative adversarial feature disentanglement module is introduced. This involves constructing a feature generator It extracts identity-related information from distractor-oriented features, and another feature generator Aims to capture features related to interference factors from identity features. and The two discriminators evaluate the authenticity of features in the identity feature domain and distractor-oriented feature domain respectively.

[0126] The discriminator is optimized by an objective function that enhances the ability to distinguish features generated by the feature generator from features of the corresponding domain:

[0127]

[0128] In the above objective function, the first two terms constrain The latter two constraints of learning.

[0129] The constraint loss function (L gen ) is represented by an adversarial loss function and a cycle consistency loss function composition:

[0130]

[0131] Among them, the adversarial loss function It effectively promotes feature domain transformation, while the cycle consistency loss function The cohesive relationships between the features facing the interference factors were maintained.

[0132] Distraction factor decoupling objective. In order to eliminate the influence of distractors, the output probability of the pedestrian classifier is redistributed. Then, the following objective is used to constrain the redistributed output probability:

[0133]

[0134] in This loss function ensures that the classification decision process is not affected by interference factors, thereby guiding the feature extractor to ignore features related to these factors.

[0135] Combining all the formulas in the above formula, we can form the optimized network parameters (excluding the generative adversarial feature disentanglement module), and the training loss function of the clothing changing model is expressed as:

[0136] L=L dual +αL int-ce +ηL dise ;

[0137] Among them, L dualrepresents the loss function of the two-stream identity feature learning framework; L int-ce represents the cross entropy loss function; L dise Denotes the allocation loss function L dise ; α and η are both constants.

[0138] S103, respectively calculating the distance between the feature vector to be queried and each of the screening result feature vectors in the screening result feature vector set to obtain a plurality of distance sets.

[0139] Exemplarily, a set of gallery images (denoted as ) and a set of query images (denoted as ) to find the correct match between them.

[0140] This is done by computing the distance between all gallery image and query image pairs. i and Q j Input feature extractor to generate feature vectors g i and q j Subsequently, using the camera classifier trained as part of the interference factor disentanglement module, we represent and The camera tag is assigned to g i and q j Considering that images taken by the same camera usually have higher visual similarity than images taken by different cameras, we use these predicted camera labels to adjust g i and q j The cosine distance between them. Formulate the calculation of this distance as d ij .as follows:

[0141] S104, determining a minimum value among the multiple distance sets, and determining a screening result feature vector corresponding to the minimum value.

[0142] Here, if the inferred camera number and Consistent, g is set by a constant threshold i and q j The cosine distance between them is reduced to δ. With the above distance measurement, it is easy to find the gallery image that is most similar to each query image to complete the pedestrian re-identification task.

[0143] S105, determining the image in the gallery image set corresponding to the image to be queried according to the feature vector of the screening result.

[0144] This paper proposes a method to achieve clothing-changing person re-identification based on isolating interference factors to obtain discriminative clothing-independent identity feature representation. Specifically, an adaptive clothing-independent contrastive learning objective is introduced to suppress the fluctuation of clothing texture in the identity feature space. To further enhance the robustness of identity features, a generative adversarial interference factor decoupling network is introduced, which can alleviate the dependence on interference factor-related identity information in the identity feature discriminative learning process. Extensive experimental results greatly demonstrate the effectiveness of the proposed method in learning CC-ReID discriminative identity features, and its performance is better than the state-of-the-art methods.

[0145] The effect of the present invention can be further illustrated by the following simulation experiment.

[0146] Dataset: The following three commonly used CC-ReID datasets (PRCC, VC-Clothes, and LTCC) are adopted. The PRCC dataset consists of 33,689 images from 221 identities. Everyone in shots A and B wears the same clothes. For the C camera, people wear different clothes and the images are taken at different times. VC-Clothes is a synthetic dataset containing 19,060 images rendered by the GTA5 game engine. These images are captured from 512 pedestrian identities under 4 cameras. Each person has 1 to 3 sets of clothes. LTCC is an indoor CC-ReID dataset that captures 17,138 images of 152 identities from 12 camera views, with 478 different costumes.

[0147] Evaluation criteria: In the test phase, standard rank-1 accuracy and mean average precision (mAP) are used for evaluation. According to previous studies, the following cloth-changing setting is adopted: the gallery setting only contains cloth-changing samples. For the three datasets, the rank-1 accuracy and mAP under the cloth-changing setting are reported.

[0148] Table 1 Experimental results under the clothing changing setting

[0149]

[0150]

[0151] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. All or part of the present invention can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, mobile communication terminals, multi-processor systems, microprocessor-based systems, programmable electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.

[0152] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the present invention.

Claims

1. A method for re-identifying a person who has changed clothes based on isolating interference factors, characterized in that: include: Get the image to be queried and the gallery image set; The image to be queried is input into the trained clothing-changing model to obtain the feature vector to be queried, and the image set of the gallery is input into the trained clothing-changing model to obtain the feature vector set of the screening result; wherein the clothing-changing model comprises: a two-stream identity feature learning framework and a generative adversarial interference factor decoupling network, wherein the two-stream identity feature learning framework is used to extract identity features that are unrelated to clothing fabrics; and the generative adversarial interference factor decoupling network is used to offset the interference of interference factors on the identity features; respectively calculating the distance between the feature vector to be queried and each of the feature vectors of the screening result in the feature vector set, and obtaining a distance set consisting of a plurality of distances; Determine a minimum value in the distance set, and determine a screening result feature vector corresponding to the minimum value; According to the screening result feature vector, a target image in the gallery image set corresponding to the image to be queried is determined.

2. The method for re-identifying a person changing clothes by isolating interference factors according to claim 1, characterized in that: In the process of training the clothing-changing model, the first loss function L of the two-stream identity feature learning framework is calculated dual Adjust the parameters of the two-stream identity feature learning framework; wherein the first loss function L of the two-stream identity feature learning framework in training is calculated dual ,include: The original image X is trained using a two-stream identity feature learning framework. i Process the original image stream to obtain the original stream processing result; wherein the original stream processing result includes: the first feature vector f i and the first M ped dimensional probability vector p i ; Among them, M ped Indicates the number of pedestrian categories in the training set; The original image X is trained using a two-stream identity feature learning framework. i Performing clothes erasing stream processing to obtain clothes erasing stream processing results; wherein the clothes erasing stream processing results include: the second feature vector and the second M ped dimensional probability vector The first loss function L of the dual-stream identity feature learning framework is calculated using the original stream processing result and the clothing erasure stream processing result. dual .

3. The method for re-identifying a person changing clothes by isolating interference factors according to claim 2, characterized in that: In the process of training the clothing-changing model, the cross entropy loss function L of the anti-interference factor decoupling network is calculated. int-ce And the distribution loss function L dise , adjust the parameters of the generated adversarial interference factor decoupling network, wherein the cross entropy loss function L of the adversarial interference factor decoupling network is calculated int-ce And the distribution loss function L dise ,include: The interference factor recognition branch in the generative adversarial interference factor decoupling network is used to identify the original image X i Extract interference features and obtain interference factor features f i int ; Using the interference factor feature f i int Calculate the cross entropy loss function L of the interference factor identification branch int-ce And the allocation loss function L of the interference factor decoupling target dise .

4. The method for re-identifying a person changing clothes by isolating interference factors according to claim 1, characterized in that: The training loss function of the clothing changing model is expressed as: L=L dual +αL int-ce +ηL dise ; Among them, L dual represents the first loss function of the two-stream identity feature learning framework; L int-ce represents the cross entropy loss function; L dise represents the allocation loss function; α and η are both constants.

5. The method for re-identifying a person changing clothes by isolating interference factors according to claim 2, characterized in that: The two-stream identity feature learning framework is used to learn the original image X i Perform raw image stream processing to obtain raw stream processing results, including: Leveraging a shared image encoder and identity feature extractor For the original image X i After processing, the first eigenvector f is obtained i , and by the pedestrian classifier Using the first eigenvector f i Generate the first M ped dimensional probability vector p i , the feature vector f i and the first M ped dimensional probability vector p i As the original stream processing result.

6. The method for re-identifying a person who changes clothes by isolating interference factors according to claim 2, characterized in that: The two-stream identity feature learning framework is used to learn the original image X i Perform clothes erasing stream processing to obtain clothes erasing stream processing results, including: Generate a binary mask S through the SCHP model i ; Using the binary mask S i And erase the image formula to get the original image X i The corresponding erased image X i ; Leveraging a shared image encoder and identity feature extractor For the erased image X i Perform feature extraction to obtain the second feature vector And by the pedestrian classifier Using the second eigenvector Generate the second M ped dimensional probability vector The second eigenvector and the second M ped dimensional probability vector Process the result as a clothes erase stream.

7. The method for re-identifying a person who changes clothes by isolating interference factors according to claim 2, characterized in that: The first loss function L of the dual-stream identity feature learning framework is calculated by using the original stream processing result and the clothing erasure stream processing result. dual ,include: According to the first M ped dimensional probability vector p i and the second M ped dimensional probability vector Calculate the loss function L of the pedestrian classifier in the two-stream identity feature learning framework ce ; Determine the first eigenvector f i and the second eigenvector Compute identity feature extractor The loss function L tri ; Construct the first eigenvector f i The corresponding first prototype model m , construct the second eigenvector with The corresponding second prototype model And the first prototype model is analyzed by exponential moving average method. m And the second prototype model Update to obtain an updated first prototype model. m and an updated second prototype model According to the updated first prototype model. m and the updated second prototype model Calculate the prototype model cross-flow matching loss function L mm ; Calculate the original image X i The minimum pedestrian inner distance is calculated by using the inner distance constraint function L cicl Constrain the first eigenvector f i and the second eigenvector The distance between them; wherein the minimum pedestrian distance is: the minimum distance between different images of pedestrians with the same identity; According to the weight calculation formula, the second M ped dimensional probability vector The corresponding weighting coefficient ω i ; According to L ce , L tri , L mm and the inner distance constraint function L cicl , determine the first loss function L of the two-stream identity feature learning framework dual .

8. The method for re-identifying a person who changes clothes by isolating interference factors according to claim 3, characterized in that: The interference factor identification branch in the generative adversarial interference factor decoupling network is used to generate the original image X i Extract interference features and obtain interference factor features f i int ,include: Leveraging a shared image encoder For the original image X i Perform processing to obtain an initial processing result; Utilize the interference factor to identify the specific feature extractor in the branch Perform feature extraction on the initial processing result to obtain interference factor feature f i int .

9. The method for re-identifying a person changing clothes by isolating interference factors according to claim 3, characterized in that: The use of the interference factor feature f i int Calculate the cross entropy loss function L of the interference factor identification branch int-ce And the allocation loss function L of the interference factor decoupling target dise ,include: The interference factor identification branch uses the first classifier and the second classifier to classify the interference factor feature f i int Perform feature extraction and obtain clothing p i clo and camera number p i cam ; The interference factor identification branch is based on the clothing p i clo and the camera number p i cam Calculate the cross entropy loss function L of the generated adversarial interference factor decoupling network int-ce ; The interference factor feature f i int Supplement to the first eigenvector f i In the equation, we get the intermediate eigenvector; Determine a pedestrian classifier according to the intermediate feature vector The allocation loss function L dise .

10. The method for re-identifying a person who changes clothes by isolating interference factors according to claim 9, characterized in that: The training method of generating a decoupling network against interference factors includes: The feature disentanglement module uses a first feature generator The interference factor feature f i int Perform domain transfer, obtain first transfer information, and use the second feature generator For the first eigenvector f i Performing domain transfer and obtaining second transfer information; Using the first feature generator The corresponding first discriminator The authenticity of the first transfer information is judged to obtain a first judgment result; using the second feature generator The corresponding second discriminator Determining the authenticity of the second transfer information to obtain a second determination result; Performing domain recovery on the first transfer information according to the first discrimination result to obtain first recovery information, and performing domain recovery on the second transfer information according to the second discrimination result to obtain second recovery information; According to the interference factor characteristics f i int and the first restored information, and the first eigenvector f i With the second recovered information, determine the cycle consistency loss function According to the first discrimination result and the second discrimination result, determine the adversarial loss function According to the first discrimination result and the second discrimination result, the generator loss function L is determined disc ; According to the cycle consistency loss function The adversarial loss function and the generator loss function L disc , the generative adversarial interference factor decoupling network is trained.

Images