Clothes changing pedestrian re-identification method, system and device and storage medium

Abstract

The invention discloses a clothes changing pedestrian re-identification method, system and device and a storage medium, which are corresponding schemes, and in the scheme, the representation distribution of clothes in a data set is learned, and is stored in a mixed dictionary to support the implementation of subsequent causal intervention; meanwhile, a novel and effective method is adopted for modeling a backdoor adjustment formula in the causal theory; in addition, modeling of clothes in a hybrid dictionary is improved by introducing decoupling regularization, so that the effectiveness of an intervention module is further improved; according to the method, causal intervention in the training process is jointly realized, better clothes invariant representation is provided for clothes changing pedestrian re-identification, the problem that an existing method is interfered by clothes and false identities is solved, and the clothes changing pedestrian re-identification effect is improved.

Description

Technical Field

[0001] This invention relates to the field of pedestrian re-identification technology, and in particular to a method, system, device, and storage medium for pedestrian re-identification when changing clothes. Background Technology

[0002] Pedestrian re-identification (ReID) aims to retrieve specific pedestrians using different cameras and is widely used in urban surveillance, smart security, and related fields. It is a highly challenging task due to frequent occlusion, background interference, and changes in lighting. Standard pedestrian re-identification assumes that people do not change their clothes throughout the retrieval process, which has limitations in long-term scenarios as pedestrians change their clothing over time. To meet the long-term retrieval requirements of real-world scenarios, Clothes-changing Person Re-identification (CC-ReID) considers the additional challenges of clothing changes. In the clothing-changing person re-identification task, learning clothing-invariant features is crucial because it requires discriminative features robust to clothing changes. However, learning clothing-invariant features is challenging because there is a spurious correlation between clothing and pedestrian identity. This spurious correlation arises because everyone's clothing is different and it is rare for two people to wear the same clothes, creating a clothing shortcut to identify people by recognizing what they are wearing. However, this clothing shortcut is an unreliable relationship. When such a model is deployed in a real-world application, it still attempts to identify people using the aforementioned shortcuts when faced with previously unseen clothing and unknown pedestrians. Therefore, pedestrians with similar clothing may be incorrectly identified, leading to poor re-identification results.

[0003] Despite the remarkable achievements of existing methods for re-identifying pedestrians changing clothes, they remain susceptible to interference from clothing shortcuts. Some methods utilize additional clothing-independent modal data to guide model training. Reference 1 (Yang et al., Person re-identification by contour sketch under moderate clothing change, TPAMI 2019) uses pure contour sketches for discriminative feature learning. Reference 2 (Chen et al., Learning 3d shapefeature for texture-insensitive person re-identification, CVPR 2021) extracts texture-insensitive 3D shape embeddings directly from 2D images by adding 3D body reconstruction as an auxiliary task. Reference 3 (Qian et al., Long-term cloth-changing person re-identification, ACCV 2020) introduces keypoint-assisted retrieval. Reference 4 (Jin et al., Cloth-changing person re-identification from a single image with gait prediction and regularization, CVPR 2022) uses gait to aid in robust feature learning for clothing changes.

[0004] Other methods use only the original RGB images to address pedestrian re-identification by changing clothes. For example, Reference 5 (Huang et al., Clothing status awareness for long-term person re-identification, ICCV 2021) proposes a clothing status awareness method to improve the robustness of retrieving people with unknown clothing status; Reference 6 (Gu et al., Clothes-changing person re-identification with RGB modality only, CVPR 2022) introduces adversarial loss to decouple clothing-irrelevant features from RGB modality. These methods are essentially based on likelihood. Likelihood probabilities model all relationships between images and identities. When combined with clothing-aware modality data (such as RGB images), they are inevitably affected by false correlations related to clothing identity, failing to meet the retrieval needs of real-world monitoring systems for pedestrians changing clothes over long periods.

[0005] In view of this, the present invention is hereby proposed. Summary of the Invention

[0006] The purpose of this invention is to provide a method, system, device, and storage medium for re-identifying pedestrians changing clothes, which can achieve discriminative learning of clothing-invariant features and improve the effect of pedestrian re-identification.

[0007] The objective of this invention is achieved through the following technical solution: A method for re-identifying pedestrians changing clothes includes: A framework for re-identifying pedestrians changing clothes is constructed, including: a two-stream network, a heterogeneous dictionary, a causal intervention module, and a decoupled regularization module; Training the pedestrian re-identification framework for changing clothes includes: extracting image features and clothing features from training images using a two-stream network; determining the clothing representation distribution using the clothing features of each training image and storing it in a mixed dictionary; using a causal intervention module to combine image features and the clothing representation distribution stored in the mixed dictionary to predict the probability distribution of identity labels when intervening in the corresponding training images, and then constructing an intervention loss; using a decoupling regularization module to predict identity and clothing based on the correspondence between image features and clothing features, constructing an orthogonal loss for clothing identity probability and a KL divergence loss based on the predicted probabilities, and constructing a metric learning loss based on the distance between features; and training the pedestrian re-identification framework for changing clothes end-to-end by combining all losses. After training, a two-stream network is used to extract image features from the input image, and the extracted images are used for retrieval to obtain the re-identification results of pedestrians changing clothes.

[0008] A pedestrian re-identification system for changing clothes, used to implement the aforementioned method, includes: The framework building unit is used to build a framework for re-identifying pedestrians changing clothes, including: a two-stream network, a hybrid dictionary, a causal intervention module, and a decoupling regularization module; The framework training unit, used to train the pedestrian re-identification framework for changing clothes, includes: extracting image features and clothing features from training images using a two-stream network; determining the representation distribution of clothing using the clothing features of each training image and storing it in a heterogeneous dictionary; using a causal intervention module to combine image features and the clothing representation distribution stored in the heterogeneous dictionary to predict the probability distribution of identity labels when intervening in the corresponding training images, and then constructing an intervention loss; using a decoupling regularization module to predict identity and clothing based on the corresponding image features and clothing features, constructing an orthogonal loss for clothing identity probability and a KL divergence loss based on the predicted probabilities, and constructing a metric learning loss based on the distance between features; and training the pedestrian re-identification framework for changing clothes end-to-end by combining all losses. The pedestrian re-identification unit for changing clothes is used to extract image features from the input image using a two-stream network after training, and then to retrieve the re-identification results of the pedestrian changing clothes using the extracted images.

[0009] A processing device includes: one or more processors; and a memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method described above.

[0010] A readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned method.

[0011] As can be seen from the technical solution provided by the present invention, by modeling the probability of causal intervention, the backdoor path brought by clothing is cut off and a more stable causal pattern is captured, thereby providing a better clothing-invariant representation for the re-identification of pedestrians who have changed clothes, and solving the problem that existing methods are interfered with by clothing and identity fraud. Attached Figure Description

[0012] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 This is a flowchart of a pedestrian re-identification method for changing clothes, provided as an embodiment of the present invention.

[0014] Figure 2 This is a schematic diagram of a mixed dictionary provided in an embodiment of the present invention.

[0015] Figure 3 This is a schematic diagram of the causal intervention module provided in an embodiment of the present invention.

[0016] Figure 4 This is a schematic diagram of decoupling regularization provided in an embodiment of the present invention.

[0017] Figure 5 This is a schematic diagram of a pedestrian re-identification system for changing clothes, provided as an embodiment of the present invention.

[0018] Figure 6 This is a schematic diagram of a processing device provided in an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of the present invention.

[0020] First, the following explanations are provided for the terms that may be used in this article: The terms "comprising," "including," "containing," "having," or other similar semantic descriptions should be interpreted as non-exclusive inclusion. For example, including a technical feature element (such as raw material, component, ingredient, carrier, dosage form, material, size, part, component, mechanism, device, step, process, method, reaction conditions, processing conditions, parameter, algorithm, signal, data, product or article of manufacture, etc.) should be interpreted as including not only the expressly listed technical feature element, but also other technical feature elements that are not expressly listed and are well-known in the art.

[0021] The following is a detailed description of a method, system, device, and storage medium for re-identifying pedestrians changing clothes provided by the present invention. Contents not described in detail in the embodiments of the present invention are prior art known to those skilled in the art. Where specific conditions are not specified in the embodiments of the present invention, conventional conditions in the art or conditions recommended by the manufacturer shall apply. Where the manufacturers of the instruments used in the embodiments of the present invention are not specified, they are all conventional products that can be purchased commercially.

[0022] Example 1 This invention provides a method for re-identifying pedestrians changing clothes, such as... Figure 1 As shown, it mainly includes the following steps: Step 1: Construct a framework for re-identifying pedestrians changing clothes.

[0023] In this embodiment of the invention, a framework for re-identifying pedestrians changing clothes is constructed, which mainly includes: a two-stream network, a hybrid dictionary, a causal intervention module, and a decoupling regularization module.

[0024] Step 2: Train the pedestrian re-identification framework for changing clothes.

[0025] In this embodiment of the invention, training the pedestrian re-identification framework for changing clothes includes: extracting image features and clothing features of training images using a two-stream network; determining the representation distribution of clothing using the clothing features of each training image and storing it in a mixed dictionary; using a causal intervention module to combine image features and the clothing representation distribution stored in the mixed dictionary to predict the probability distribution of identity labels when intervening in the corresponding training images, and then constructing an intervention loss; using a decoupling regularization module to predict identity and clothing based on the corresponding image features and clothing features, constructing an orthogonal loss for clothing identity probability and a KL divergence loss based on the predicted probabilities, and constructing a metric learning loss based on the distance between features; and training the pedestrian re-identification framework for changing clothes end-to-end by combining all losses.

[0026] (1) Use the clothing features of each training image to determine the representation distribution of clothing and store it in a mixed dictionary.

[0027] In this embodiment of the invention, the heterogeneous dictionary stores the representation distribution of clothing, as well as clothing tags and probabilities, represented as follows: ; Where D represents a mixed dictionary. Indicates the number of clothing categories. Representing the Labels for clothing Statistics from the training set The probability of; and The two types of representations for clothing are based on clothing feature diagrams. Calculated.

[0028] and The calculation method is as follows: Aggregate clothing feature maps using pooling operations. The channel and spatial information are represented as: ; ; in, and These represent pooling operations applied along the channel and spatial axes, respectively. , This refers to the aggregated information.

[0029] In conjunction with clothing labels, relevant information is updated using an exponential moving average method. and , represented as: ; ;

[0030] in, Represents the memory coefficient; symbol This indicates assignment, and its right side... and The information on the left side is the information before the update. and For the updated information; for example, the initial information (i.e., before the first update). and All are vectors containing only zeros.

[0031] (2) By combining the causal intervention module with image features and the representation distribution of clothing stored in the mixed dictionary, the probability distribution of identity labels when intervening in the corresponding training images is predicted, and then the intervention loss is constructed.

[0032] In this embodiment of the invention, the probability distribution of identity labels when intervening in the corresponding training images is predicted by using a causal intervention module that combines image features with the representation distribution of clothing stored in a mixed dictionary, as shown below: ; in, This indicates that intervention is performed on the training image X to make X the i-th image. ; Indicates the given condition is The probability distribution of identity label Y at that time; The label representing clothing category j. Given conditions When, the probability distribution of identity label Y, for The probability of; To extract from the image The image features extracted from them Let j-th type of clothing be represented by two classes; function Used for To generate the identity probability under the condition, that is, to output the identity probability corrected by causal intervention, Cls represents the causal classifier, which outputs the identity probability without intervention.

[0033] In this embodiment of the invention, the function It includes channel attention, spatial attention, and offset terms, and is represented as follows: ; in, This indicates a pooling operation applied along the spatial axis. This indicates element-wise multiplication. , and There are three different linear layers.

[0034] In this embodiment of the invention, the intervention loss is expressed as: ; Among them, L int Let E be the expected value of the intervention loss. For image identity tags Indicates identity label , Indicates the given condition is At that time, identity tags The probability value.

[0035] (3) Use the decoupling regularization module to predict identity and clothing based on the corresponding image features and clothing features. Combine the predicted probabilities to construct clothing identity probability orthogonal loss and KL divergence loss, and combine the distance between features to construct metric learning loss.

[0036] In this embodiment of the invention, the method of predicting identity and clothing based on the correspondence between image features and clothing features using a decoupling regularization module includes: constructing a clothing identity classifier, which has a total of The categories, among which, Indicates the number of identity categories. This indicates the number of clothing categories; the image features and clothing features are input into the clothing identity classifier to obtain the corresponding probabilities, represented as: ; ; in, and The corresponding values ​​represent the weights of the identity classifier and the clothing classifier, and softmax is the normalization exponential function. For image features, The predicted probability of identity. As a feature of clothing, This represents the predicted probability of the clothing.

[0037] In the above formula, when predicting a certain probability, the corresponding weight is placed at the front end. In this case, even if the two types of labels overlap (for example, the label for clothing is 0~100 and the label for identity is 0~50), there is no need to modify the labels.

[0038] In this embodiment of the invention, the step of constructing an orthogonal loss for clothing identity probability by combining the predicted probability distribution with the KL divergence loss, and constructing a metric learning loss by combining the distance between features, includes: The orthogonal loss of clothing identity probability is represented as: ; Where E is the expected value. For identity tags, For clothing labels, The loss is an orthogonal loss function for clothing identity probability.

[0039] The KL divergence loss is expressed as: ; in, Indicates the KL divergence. For KL divergence loss; This refers to the average identity probability of half (a random half, as will be the case later) of the training images in the current batch, each with the same identity label. The average identity probability of the other half of the training images is... ; This refers to the average clothing probability of half of the training images with the same clothing label in the current batch, while the average clothing probability of the other half of the training images is... That is, each identity tag and clothing tag has a corresponding average probability. and , and .

[0040] The metric learning loss, constructed by combining the distances between features, is expressed as follows: ; in, To measure the learning loss, Euclidean distance; function max means to take the maximum value, that is, the function It is a function that sets the value in parentheses to 0 when the value in the parentheses is less than 0. For margin parameters, ,when hour, This represents the image feature centers of half of the training images with the same identity label in the current batch. hour, This represents the clothing feature centers of half of the training images with the same clothing label in the current batch. The feature center represents anchor sample a, which is the currently selected image. Images with the same label as anchor sample a are positive samples, and their feature centers are... Images with different labels from anchor sample a are negative samples, and their feature centers are... Depending on the value of m, the feature center represents the corresponding image feature center or clothing feature center.

[0041] Finally, the total loss is constructed based on the above four loss parameters, and end-to-end training is performed.

[0042] Step 3: Re-identify pedestrians changing clothes.

[0043] In this embodiment of the invention, after training is completed, a two-stream network is used to extract image features from the input image, and the extracted image is used for retrieval to obtain the re-identification result of the person changing clothes.

[0044] The above-described scheme provided by this invention learns the representation distribution of clothing in the dataset and stores it in a heterogeneous dictionary to support subsequent causal intervention. Simultaneously, a novel and effective method is employed to model the backdoor adjustment formula in causal theory. Furthermore, decoupling regularization is introduced to improve the modeling of clothing in the heterogeneous dictionary, thereby further enhancing the effectiveness of the intervention module. Through these methods, causal intervention during the training process is achieved, providing a better clothing-invariant representation for pedestrian re-identification after clothing changes, solving the problem of interference from clothing and identity fraud in existing methods, and thus improving the performance of pedestrian re-identification after clothing changes.

[0045] To more clearly demonstrate the technical solution and its effects provided by the present invention, the method provided by the embodiments of the present invention will be described in detail below with reference to specific examples.

[0046] I. Overall Overview of the Plan

[0047] This invention proposes a causal clothing-invariant learning technique that focuses on learning causal relationships. To model causal intervention within a pedestrian re-identification framework involving clothing changes, three novel modules are designed: First, the representation distribution of clothing in the dataset is learned and stored in a heterogeneous dictionary to support subsequent causal intervention implementation; second, the causal intervention module employs a novel and effective method to model the backdoor adjustment formula in causal theory; finally, decoupling regularization improves the modeling of clothing in the heterogeneous dictionary, thereby further enhancing the effectiveness of the intervention module. These three modules collectively implement causal intervention during the training process, thus achieving discriminative clothing-invariant feature learning.

[0048] II. Detailed introduction of the plan.

[0049] This section provides a detailed introduction to the three new modules in the framework, the loss during training, and training details. Other modules in the framework can be implemented using conventional techniques.

[0050] 1. Mixed dictionary.

[0051] like Figure 2 As shown, v represents the implementation flow of the heterogeneous dictionary. During training, a given image is fed into a two-stream network to obtain image features and clothing features. A heterogeneous dictionary D is constructed to represent all clothing in the training set, supporting subsequent causal interventions. The heterogeneous dictionary stores clothing labels. probability And two representations for each outfit and A mixed dictionary is defined as follows: ; Among them, probability It can be obtained statistically from the training set. and From clothing feature diagram calculate.

[0052] Specifically, pooling operations are used to aggregate the channel and spatial information of the garment feature map: ; ; in, and These represent the pooling operations applied along the channel and spatial axes, respectively. Through these two pooling operations, the hybrid dictionary contains information about two different garment representations.

[0053] Subsequently, based on clothing tags, relevant information was updated using an exponential moving average scheme. and : ; ; in, Represents the memory coefficient, for example, which can be set. The value is set to 0.9 to achieve a balance between update speed and stability.

[0054] 2. Causal intervention module.

[0055] To eliminate the interference of clothing shortcuts, we construct the intervention loss by maximizing the intervention probability. : ; Here, variable X represents the image, and Y represents the identity label. The intervention reflects the causal relationship between variables X and Y, independent of confounding factors, which is consistent with the goal of keeping the clothing unchanged.

[0056] To achieve the intervention optimization in the above equation, a causal intervention module was designed to be well implemented within the framework of pedestrian re-identification while changing clothes. .like Figure 3 As shown, the intervention module uses images Image feature map The module takes a mixed dictionary as input and then calculates the intervention probability based on backdoor adjustment theory. Using this module, the intervention probability is calculated as follows: ; in, and Implemented in a mixed dictionary. It is a classifier. Defined as a linear layer, softmax is a normalized exponential function. Functions are used for conditional Probability of generating a human identity.

[0057] In this embodiment of the invention, a system is designed that includes a channel attention term, a spatial attention term, and an offset term. The function is represented as follows: ; in, This represents element-wise multiplication. ϕ is a linear layer with a sigmoid activation function and a reshape operation. and These are two independent linear layers. (Function) Spatial and channel information of confounding factors can be combined simultaneously to better estimate conditional probability This invention utilizes a normalized weighted geometric mean approximation to remove the classifier from the accumulation operation: ; By simplifying multiple classifications into a single operation using the above method, the probability of intervention is reduced. The calculation. In summary, the causal intervention module can eliminate the influence of clothing shortcuts by simulating causal intervention, ensuring the learning of clothing invariant features.

[0058] 3. Decoupling regularization.

[0059] In order to achieve the probability of intervention through deep learning Modeling is crucial to ensuring the effectiveness of interventions as much as possible. Therefore, as... Figure 4 As shown, a decoupling regularization was designed to ensure that clothing representations are discriminative and do not contain identity cues. This allows causal interventions to accurately cut off clothing shortcuts without affecting the extraction of identity features.

[0060] In this embodiment of the invention, a clothing-identity probability orthogonal loss is proposed. To replace the original classification loss. Specifically, a clothing-identity classifier was constructed, which has a total of The categories, of which, Represents the number of human identity categories. This represents the number of clothing categories. Subsequently, image features... and clothing characteristics Input the data into the clothing-identity classifier to obtain the predicted probabilities: ; ; in, and The corresponding values ​​represent the weights of the identity classifier and the clothing classifier, and softmax is the normalization exponential function. For image features, The predicted probability of identity. As a feature of clothing, For the predicted probability of clothing, Indicates splicing.

[0061] Clothing Identity Probability Orthogonal Loss Minimize the cross-entropy between the predicted probability and the identity tag or clothing tag: .

[0062] This strategy results in clothing and images becoming negative categories of each other, with their features belonging to different spaces. Therefore, clothing features contain information different from image features, contributing to better decoupling.

[0063] To further separate clothing from identity, an additional KL divergence loss was used. : ; in, This represents the KL divergence. It is based on the identity probability of half of the samples with the same identity label. It is calculated by averaging, and It refers to the average identity probability of the other half of the samples in the current small batch. and They have similar meanings and calculation methods. Therefore, this loss term implies that the classification probabilities between positive samples should be as similar as possible, and that probability averaging is used to reduce the interference of noisy samples.

[0064] In addition, a method for measuring learning loss was designed. It directly constrains the Euclidean metric between features: ; in, It is Euclidean distance. , This is the margin parameter, which can be set to 0.6. , Image / clothing feature centers representing half of the samples with the same identity / clothing label in the current mini-batch. yes The feature centers of positive samples It is the feature center of the negative sample.

[0065] Specifically, measuring learning loss The aim is to cluster positive features tightly together and maintain Euclidean distances between negative features. This is consistent with the approach proposed in this invention. Similarly, measure the learning loss. Feature learning is performed in a shared metric space, where clothing and images are treated as negative categories to emphasize their distinction. This strategy further reinforces the idea that clothing features should possess different information than image features, leading to better decoupling.

[0066] 4. Total loss function.

[0067] The entire framework is trained end-to-end, with a total loss. Defined as: .

[0068] This invention uniformly assigns a weight of 1 to all losses, eliminating the need for hyperparameter search, which has already yielded good results.

[0069] 5. Training details.

[0070] In this embodiment of the invention, the two-stream network can be implemented using existing network structures as needed. For example, a ResNet-50 model (a residual network with 50 weight layers) pre-trained on ImageNet can be used as the backbone of the two-stream network. The first bottleneck of the ResNet-50 is designated as a shared module, while other bottlenecks are configured to be branch-specific.

[0071] For example: The entire framework was trained for 120 iterations using the Adam (Adaptive Moment Estimation) optimizer. In the first 10 epochs, the learning rate was linearly increased from 3.5e-5 to 3.5e-4. Afterward, the learning rate decayed to 0 according to a cosine decay rule. The batch size was set to 64, with 8 identities. The image size was resized to 384×192 and enhanced during training through random horizontal flipping, padding, random cropping, and random erasing. The relevant training procedures involved can be referenced from conventional techniques, and will not be elaborated upon in this invention.

[0072] During the reasoning phase, only the identity feature extraction stream is activated, such as... Figure 2 As shown in the lower left corner, this stage only extracts image features and directly measures the cosine similarity between image features to obtain retrieval results (re-identification results of pedestrians changing clothes).

[0073] Through the above description of the embodiments, those skilled in the art can clearly understand that the above embodiments can be implemented by software, or by using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions of the above embodiments can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, mobile hard drive, etc.), including several instructions to cause a computer device (such as a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0074] Example 2 This invention also provides a pedestrian re-identification system for changing clothes, which is mainly used to implement the methods provided in the foregoing embodiments, such as... Figure 5 As shown, the system mainly includes: The framework building unit is used to build a framework for re-identifying pedestrians changing clothes, including: a two-stream network, a hybrid dictionary, a causal intervention module, and a decoupling regularization module; The framework training unit, used to train the pedestrian re-identification framework for changing clothes, includes: extracting image features and clothing features from training images using a two-stream network; determining the representation distribution of clothing using the clothing features of each training image and storing it in a heterogeneous dictionary; using a causal intervention module to combine image features and the clothing representation distribution stored in the heterogeneous dictionary to predict the probability distribution of identity labels when intervening in the corresponding training images, and then constructing an intervention loss; using a decoupling regularization module to predict identity and clothing based on the corresponding image features and clothing features, constructing an orthogonal loss for clothing identity probability and a KL divergence loss based on the predicted probabilities, and constructing a metric learning loss based on the distance between features; and training the pedestrian re-identification framework for changing clothes end-to-end by combining all losses. The pedestrian re-identification unit for changing clothes is used to extract image features from the input image using a two-stream network after training, and then to retrieve the re-identification results of the pedestrian changing clothes using the extracted images.

[0075] Since the main technical details of this system have been described in detail in previous embodiments, they will not be repeated here.

[0076] Those skilled in the art will understand that, for the sake of convenience and brevity, the above-described division of functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above.

[0077] Example 3 The present invention also provides a processing device, such as Figure 6As shown, it mainly includes: one or more processors; a memory for storing one or more programs; wherein, when the one or more programs are executed by the one or more processors, the one or more processors implement the method provided in the foregoing embodiments.

[0078] Furthermore, the processing device also includes at least one input device and at least one output device; in the processing device, the processor, memory, input device, and output device are connected via a bus.

[0079] In this embodiment of the invention, the specific types of the memory, input device, and output device are not limited; for example: Input devices can be touchscreens, image acquisition devices, physical buttons, or mice, etc. The output device can be a display terminal; The memory can be random access memory (RAM) or non-volatile memory, such as disk storage.

[0080] Example 4 The present invention also provides a readable storage medium storing a computer program that, when executed by a processor, implements the method provided in the foregoing embodiments.

[0081] In this embodiment of the invention, the readable storage medium is a computer-readable storage medium and can be disposed in the aforementioned processing device, for example, as a memory in the processing device. Furthermore, the readable storage medium can also be any medium capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), magnetic disk, or optical disk.

[0082] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims. The information disclosed in the background section is intended only to enhance the understanding of the overall background technology of the present invention and should not be construed as an admission or implication in any way that such information constitutes prior art known to those skilled in the art.

Claims

1. A method for re-identifying pedestrians changing clothes, characterized in that, include: A framework for re-identifying pedestrians changing clothes is constructed, including: a two-stream network, a heterogeneous dictionary, a causal intervention module, and a decoupled regularization module; Training the pedestrian re-identification framework for changing clothes includes: extracting image features and clothing features from training images using a two-stream network; determining the representation distribution of clothing using the clothing features of each training image and storing it in a mixed dictionary; using a causal intervention module to combine image features and the clothing representation distribution stored in the mixed dictionary to predict the probability distribution of identity labels when intervening in the corresponding training images, and then constructing an intervention loss; using a decoupling regularization module to predict identity and clothing based on the correspondence between image features and clothing features, constructing clothing identity probability orthogonal loss and KL divergence loss based on the predicted probabilities, and constructing a metric learning loss based on the distance between features; and training the pedestrian re-identification framework for changing clothes end-to-end by combining all losses. After training, a two-stream network is used to extract image features from the input image, and the extracted images are used for retrieval to obtain the re-identification results of pedestrians changing clothes.

2. The method for re-identifying pedestrians changing clothes according to claim 1, characterized in that, The step of determining the representation distribution of clothing using clothing features from each training image and storing it in a heterogeneous dictionary includes: The heterogeneous dictionary stores the distribution of clothing representations, as well as clothing tags and probabilities, expressed as follows: ； Where D represents a mixed dictionary. Indicates the number of clothing categories. Representing the Labels for clothing Statistics from the training set The probability of; and The two types of representations for clothing are based on clothing feature diagrams. The calculation is as follows: Aggregate clothing feature maps using pooling operations. The channel and spatial information are represented as: ； ； in, and These represent pooling operations applied along the channel and spatial axes, respectively. , The aggregated information; In conjunction with clothing labels, relevant information is updated using an exponential moving average method. and , represented as: ； ； in, Represents the memory coefficient; symbol This indicates assignment, and its right side... and The information on the left side is the information before the update. and This is the updated information.

3. The method for re-identifying pedestrians changing clothes according to claim 2, characterized in that, By utilizing the causal intervention module, combined with image features and the clothing representation distribution stored in a mixed dictionary, the probability distribution of identity labels when intervening in the corresponding training images is predicted, as follows: ； in, This indicates that intervention is performed on the training image X to make X the i-th image. ; Indicates the given condition is The probability distribution of identity label Y at that time; The label representing clothing category j. Given conditions When, the probability distribution of identity label Y, for The probability of; To extract from the image The image features extracted from them Let j-th type of clothing be represented by two classes; function Used for Cls represents a causal classifier, which generates identity probabilities based on conditions.

4. The method for re-identifying pedestrians changing clothes according to claim 3, characterized in that, The function Represented as: ； in, This indicates a pooling operation applied along the spatial axis. This indicates element-wise multiplication. , and There are three different linear layers.

5. A method for re-identifying pedestrians changing clothes according to claim 3 or 4, characterized in that, The intervention loss is expressed as: ； Among them, L int Let E be the expected value of the intervention loss. For image identity tags Indicates identity label , Indicates the given condition is At that time, identity tags The probability value.

6. The method for re-identifying pedestrians changing clothes according to claim 1, characterized in that, The method of using the decoupling regularization module to predict identity and clothing based on the correspondence between image features and clothing features includes: Build a clothing identity classifier with a total of The categories, among which, Indicates the number of identity categories. Indicates the number of clothing categories; The image features and clothing features are input into the clothing identity classifier to obtain the corresponding probabilities, represented as follows: ； ； in, and The corresponding values ​​represent the weights of the identity classifier and the clothing classifier, and softmax is the normalization exponential function. For image features, The predicted probability of identity. As a feature of clothing, This represents the predicted probability of the clothing.

7. The method for re-identifying pedestrians changing clothes according to claim 6, characterized in that, The method of constructing an orthogonal loss for clothing identity probability by combining the predicted probabilities with the KL divergence loss, and constructing a metric learning loss by combining the distance between features, includes: The orthogonal loss of clothing identity probability is represented as: ； Where E is the expected value. For identity tags, For clothing labels, Orthogonal loss for clothing identity probability; The KL divergence loss is expressed as: ； in, Indicates the KL divergence. KL divergence loss; This refers to the average identity probability of half of the training images with the same identity label in the current batch, while the average identity probability of the other half of the training images is... ; This refers to the average clothing probability of half of the training images with the same clothing label in the current batch, while the average clothing probability of the other half of the training images is... ; The metric learning loss, constructed by combining the distances between features, is expressed as follows: ； in, To measure the learning loss, For Euclidean distance, the function , max means take the maximum value. For margin parameters, ,when hour, This represents the image feature centers of half of the training images with the same identity label in the current batch. hour, This represents the clothing feature centers of half of the training images with the same clothing label in the current batch. The feature center represents anchor sample a, which is the currently selected image. Images with the same label as anchor sample a are positive samples, and their feature centers are... Images with different labels from anchor sample a are negative samples, and their feature centers are... Depending on the value of m, the feature center represents the corresponding image feature center or clothing feature center.

8. A pedestrian re-identification system for changing clothes, characterized in that, To implement the method according to any one of claims 1 to 7, comprising: The framework building unit is used to build a framework for re-identifying pedestrians changing clothes, including: a two-stream network, a hybrid dictionary, a causal intervention module, and a decoupling regularization module; The framework training unit, used to train the pedestrian re-identification framework for changing clothes, includes: extracting image features and clothing features from training images using a two-stream network; determining the representation distribution of clothing using the clothing features of each training image and storing it in a heterogeneous dictionary; using a causal intervention module to combine image features and the clothing representation distribution stored in the heterogeneous dictionary to predict the probability distribution of identity labels when intervening in the corresponding training images, and then constructing an intervention loss; using a decoupling regularization module to predict identity and clothing based on the corresponding image features and clothing features, constructing an orthogonal loss for clothing identity probability and a KL divergence loss based on the predicted probabilities, and constructing a metric learning loss based on the distance between features; and training the pedestrian re-identification framework for changing clothes end-to-end by combining all losses. The pedestrian re-identification unit for changing clothes is used to extract image features from the input image using a two-stream network after training, and then to retrieve the re-identification results of the pedestrian changing clothes using the extracted images.

9. A processing device, characterized in that, include: One or more processors; Memory, used to store one or more programs; Wherein, when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the method as described in any one of claims 1 to 7.

10. A readable storage medium storing a computer program, characterized in that, When a computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images