A target re-identification method and system based on feature disentangled representation

Abstract

The application discloses a target re-identification method and system based on feature disentangled representation, which comprises the following steps: step 1: using a multi-granularity feature extraction network module to extract multi-granularity features of an image to be identified, and classifying the features into identification target irrelevant features and identification target relevant features; step 2: inputting the identification target irrelevant features and the identification target relevant features into a generator in a generative adversarial network module to perform semantic feature reconstruction, disentangling from single feature representation, and generating a new image; step 3: inputting the generated new image into a discriminator in the generative adversarial network module to identify the authenticity and category of the image, and outputting a final identification result; the application proposes a new feature learning framework based on feature disentangling, does not need to provide sample label information additionally, realizes the robustness of feature representation of an identification target to a view angle change, enhances network generalization, and simultaneously improves the accuracy of target re-identification.

Description

Technical Field

[0001] This invention belongs to the field of target re-identification, and specifically relates to a target re-identification method and system based on feature unwrapped representation. Background Technology

[0002] With the further development of deep learning, object re-identification tasks have seen continuous innovation and progress in recent years, achieving nearly satisfactory results. Most current object re-identification methods involve adding new layer structures to convolutional neural networks to extract target image features or calculating attention maps of local regions to improve the model's representational capabilities. However, current methods of adding new structures to convolutional networks have limitations in handling intra-class discrepancies such as image occlusion. Meanwhile, attention enhancement or horizontal segmentation methods offer advantages in improving the network's ability to extract subtle features and exhibit strong robustness against occlusion. However, this approach, by overemphasizing the extraction of discriminative visual information, is prone to intra-class matching contradictions. This is because local details are often visible only from specific viewpoints, especially in complex scenes where the viewpoint is greatly limited, resulting in significant viewpoint differences. This makes it difficult for the model to directly learn pure identity-related information.

[0003] In recent years, unlike discriminative models, generative adversarial training (GANs) has seen the emergence of new design ideas. These GANs achieve robust feature representations against specific intra-class variations through a competitive game between the generator and the discriminator. Compared to the discriminative learning stage, the generator algorithm is independent. With the continuous evolution of GAN models, it is now possible to directly extract identity-related or pose-related features from input target images while removing interfering target re-identification information, thus eliminating the need for additional target image annotations.

[0004] While existing methods have made good progress, they still have limitations, as most rely heavily on the number of samples. However, it is difficult to obtain a large number of high-quality sample images in real-world scenarios, making it difficult for the aforementioned methods to effectively perform target re-identification tasks in complex scenes. Summary of the Invention

[0005] To address the aforementioned problems, this invention proposes a target re-identification method and system based on feature unwrapped representation. Employing the proposed GPS-GAN framework, it eliminates the need for any auxiliary constraint signals. Two sets of branches are cascaded within a shared backbone network to decompose target-related and target-independent features, making the entire target re-identification network more concise. Furthermore, it can directly unwrap all variable factors and remove target-independent features, effectively overcoming the shortcomings of existing technologies in target re-identification tasks.

[0006] On the one hand, a target re-identification method based on feature unwrapped representation includes:

[0007] Step 1: Use the multi-granularity feature extraction network module to extract multi-granularity features from the image to be identified and classify them into features irrelevant to the identification target and features relevant to the identification target;

[0008] Step 2: Input the target-irrelevant features and target-related features into the generator in the generative adversarial network module for semantic feature reconstruction, deentangle the features from the single feature representation, and generate a new image;

[0009] Step 3: Input the newly generated image into the discriminator in the generative adversarial network module to identify the authenticity and category of the image, and output the final recognition result;

[0010] A target re-identification network based on feature untangling is constructed by sequentially cascading multi-granularity feature extraction network modules and generative adversarial network modules. During the training process of the target re-identification network based on feature untangling, the losses of each step from step 1 to step 3 are accumulated, and the minimum accumulated loss is taken as the training objective.

[0011] The technical solution of this invention proposes a GPS-GAN framework, which combines adversarial generative training without the need for additional annotation information. By unwrapping target features, irrelevant features of the target are removed, while target-related features are mined and enhanced, thereby making accurate judgments about the target.

[0012] Furthermore, the deentanglement from a single feature representation refers to:

[0013] By using a generative adversarial network module to perform feature unwrapping and synthesize a new image through the same target transformation, based on global-level and local-level feature exchange strategies;

[0014] Global-level feature swapping strategies involve swapping different images x of the same target. a x p Target-related feature encoding ensures that the new image generated by the generative adversarial network module still possesses the visual features matched by the original target, thus generating a different image x of the same target. a x p Synthetic images;

[0015] Because x a x p Although they belong to the same target, they present different images due to different viewpoints, poses, lighting, and backgrounds. Therefore, a global-level feature exchange strategy will force the identification of the target-related branch R. θ Extract features robust to intra-class variations, further mining and enhancing image x. a and image xp The common features between them, while forcibly identifying target-irrelevant branches U θ To extract the unique variable factors within a single image.

[0016] The local-level feature exchange strategy divides the complete feature map into multiple local blocks, performs feature deentanglement and target transformation at the local level, and randomly selects local features of the target from the target feature-related feature embedding function of any one of the two images of the same target, and exchanges them with the local features at the corresponding positions in the target feature-related feature embedding function of the other image at the same position.

[0017] Furthermore, the expression for global-level feature exchange is:

[0018] G(f R (a)⊕f U (p))

[0019] G(f R (p)⊕f U (a))

[0020] Among them, f R (a), f U (a) Represents the images x from the same input target. a The target-related features and target-irrelevant features extracted from the data, f R (p), f U (p) represent images x from the same input target. p The features extracted are those relevant to the target and those irrelevant to the target. ⊕ represents the concatenation of the two types of features, and G() represents the feature exchange function.

[0021] Generating adversarial network modules requires starting from G(f) R (a)⊕f U Image x is generated in (p)) p At the same time from G(f) R (p)⊕f U (a) generates the image x p Image x of the same target a .

[0022] Furthermore, global-level feature exchange loss Defined as:

[0023]

[0024] Local-level feature exchange loss Defined as:

[0025]

[0026] Where, |||1 represents the absolute value of L1 loss, E represents the generative adversarial network module, and x i Images i and f representing the same target R (i), f U (i) represent the images x from the same input target. i The features extracted are those related to the target and those unrelated to the target. The values ​​of i and j are a or p, and S represents a local region transformation operator.

[0027] Furthermore, loss for identifying target-irrelevant features. as follows:

[0028]

[0029] in, p represents the true probability distribution, and q represents the approximate probability distribution. This represents the target label c and feature f. R Associated classifier parameters; It is the index of the target label c. It takes the value 1 when the target predicted by the three branches in the multi-granularity feature extraction network module is the same as the target label, and 0 otherwise. f represents the embedding function for extracting target-related features, which yields k target-related features of the input image encoded using the R-th encoding method, where k = 1, 2…K, and K represents the total number of input images containing the same target; R D represents the global features and local features of different scales extracted by a multi-granularity feature network using the R-th encoding method, where R = 1, 2…8; KL It represents the KL divergence of a feature that is independent of the constraint target and approximates a normal distribution N(0,1).

[0030] Furthermore, the multi-granularity feature extraction network adopts the MGN multi-granularity learning structure and is composed of multiple layers of shared convolutional layers stacked together; the backbone in the multi-granularity feature extraction network is a ResNet50 loaded with ImageNet pre-trained weights.

[0031] A multi-granularity feature extraction network was trained using Cross-Entropy, and a target classification loss was set. as follows:

[0032]

[0033] Among them, f R This represents the global features and local features of different scales extracted by a multi-granularity feature network using the R-th encoding method, where R = 1, 2, ..., 8; This represents the target label c and feature f. R Associated classifier parameters; It is the index of the target label c. It takes the value 1 when the target predicted by the three branches in the multi-granularity feature extraction network module is the same as the target label, and 0 otherwise. C represents the total number of target labels. The embedding function is used to extract target-related features, which yields k target-related features of the input image encoded using the R-th encoding method, where k = 1, 2, ..., K, and K represents the total number of input images of the same target.

[0034] Representing feature f R The corresponding target probability is obtained by applying Softmax.

[0035] Furthermore, the process of inputting target-independent and target-related features into the generative adversarial network module for semantic feature reconstruction, untangling the features from a single feature representation, and generating a new image, is the loss L of the image reconstruction process. r The L1 loss function is used, as follows:

[0036]

[0037] Where |||1 represents the absolute value of L1 loss, E represents the generative adversarial network module, and x i Images i and f representing the same target R (i), f U (i) represent the images x from the same input target. i The features extracted are those related to the target and those unrelated to the target, where i can take the value a or p.

[0038] Furthermore, in the aforementioned step, the discriminator includes a domain-discriminator D. d And class-discriminator D c Domain-Discriminator D d The class-discriminator D distinguishes between real and synthetic images. c The obtained target image and the input target image will be used for target classification; the loss of the domain-discriminator is defined as follows:

[0039]

[0040] The loss of the class-discriminator is defined as follows:

[0041]

[0042] Where, x i Images i and f representing the same target R (i), fU (i) represent the images x from the same input target. i The features extracted are those related to the target and those unrelated to the target. The value of i is either a or p, and S represents a local region transformation operator.

[0043] Furthermore, the accumulated loss L total Specifically as follows:

[0044]

[0045] Where, λ R , λ U , λ r , λ gs , λ ps , λ d , λ c The target classification loss is represented in sequence. KL divergence loss, L1 loss, global-level feature exchange loss, local-level feature exchange loss, domain-discriminator loss, and the scaling factor of domain-discriminator loss.

[0046] On the other hand, a target re-identification system based on feature unwrapped representation includes:

[0047] The multi-granularity feature extraction network module extracts multi-granularity features from the image to be identified and classifies them into features irrelevant to the target and features relevant to the target.

[0048] Generative Adversarial Network (GAN) module: Inputting target-irrelevant features and target-related features into the generator in the GAN module for semantic feature reconstruction, untangling the single feature representation, and generating a new image; inputting the generated new image into the discriminator in the GAN module to identify the authenticity and category of the image, and outputting the final recognition result;

[0049] The network construction module consists of a multi-granularity feature extraction network module and a generative adversarial network module, which are cascaded together to build a target re-identification network based on feature untangling.

[0050] In the training module, during the training process of the target re-identification network based on feature untangling, the losses executed in the multi-granularity feature extraction network module and the generative adversarial network module are accumulated, and the minimum of the accumulated loss is taken as the training objective.

[0051] The multi-granularity feature extraction network module, generative adversarial network module, and training module use the aforementioned target re-identification method based on feature unwrapped representation for image data processing.

[0052] Beneficial effects

[0053] Compared with the prior art, the technical solution provided by this invention has the following advantages:

[0054] (1) The technical solution of the present invention proposes a novel feature unwrapping representation learning GPS-GAN framework for target re-identification tasks, which unwrappes the target-related features and the target-irrelevant features without providing additional annotation information. This framework combines adversarial generative training, which removes the target-irrelevant features, thereby making an accurate judgment on the target and realizing the robustness of the target feature representation to changes in viewpoint.

[0055] (2) Based on the multi-granularity feature network learning mode, a global-level and local-level target-related feature exchange strategy is proposed on the basis of feature untangling learning, which further strengthens the model’s ability to represent the target identity, especially in the case of occlusion and target viewpoint changes.

[0056] (3) By learning to identify target-related and target-independent feature codes, the robustness of target class-intraclass variation and reasonable feature matching are achieved, which reflects the representational advantages of the GPS-GAN framework, enhances the network generalization, and improves the accuracy of target re-identification. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 This is an overall framework diagram of the target re-identification network described in the example of the present invention, wherein, E enc E represents the multi-granularity feature extraction network module. gen E represents the generator module. dis Represents the discriminator module, R θ U represents the branch related to the objective. θ f represents a target-independent branch. R f represents the target-related features. U Indicates target-irrelevant features;

[0059] Figure 2 This is a diagram of the multi-granularity feature extraction network framework described in this invention example;

[0060] Figure 3 This is the global-level feature exchange strategy diagram described in the example of this invention;

[0061] Figure 4This is a local-level feature exchange strategy diagram as described in the example of this invention. Detailed Implementation

[0062] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] Example 1:

[0064] like Figures 1-4 As shown, a target re-identification method based on feature unwrapped representation includes:

[0065] Step 1: Use the multi-granularity feature extraction network module to extract multi-granularity features from the image to be identified and classify them into features irrelevant to the identification target and features relevant to the identification target;

[0066] The multi-granularity feature extraction network adopts the MGN multi-granularity learning structure and is composed of multiple layers of shared convolutional layers stacked together; the backbone in the multi-granularity feature extraction network is a ResNet50 loaded with ImageNet pre-trained weights.

[0067] A multi-granularity feature extraction network was trained using Cross-Entropy, and a target classification loss was set. as follows:

[0068]

[0069] Among them, f R This represents the global features and local features of different scales extracted by a multi-granularity feature network using the R-th encoding method, where R = 1, 2, ..., 8; This represents the target label c and feature f. R Associated classifier parameters; It is the index of the target label c. It takes the value 1 when the target predicted by the three branches in the multi-granularity feature extraction network module is the same as the target label, and 0 otherwise. C represents the total number of target labels. The embedding function is used to extract target-related features, which yields k target-related features of the input image encoded using the R-th encoding method, where k = 1, 2, ..., K, and K represents the total number of input images of the same target.

[0070] Representing feature f R The corresponding target probability is obtained by applying Softmax.

[0071] During training, a set of images X taken from different angles containing the target to be identified is given. k (k = 1, 2, ..., K) are input into the multi-granularity feature extraction network module E. enc (x) Extracting the appearance features of the target, the first three stages of ResNet50 are responsible for extracting the basic features of the input target image. Then, in the higher-level stages, it separates into three independent branches: the first branch, the second branch, and the third branch. Each branch has a similar structure, but the number of channels and network weight parameters differ. These branches consist of two convolutional layers, a global max-pooling layer, and a bottleneck layer, respectively, and output feature maps with tensor sizes of 1×1×p, 1×1×3p, and 1×1×4p. Here, p is the number of channels in the feature map; when the feature map represents target-related features, p is 256, and when it represents target-independent features, p is 64. The branches divide the feature map output by the feature extraction network into one, two, and three horizontal regions on an equal horizontal axis. The first branch obtains a single horizontal region, the second branch obtains two horizontal regions bisected horizontally, and the third branch obtains three horizontal regions trisected horizontally. The first branch is responsible for extracting the global features of the entire image containing the target. Other branches are responsible for extracting global and local features of the target portion in the image. Local features are extracted from the corresponding horizontal regions. Taking the third branch as an example, the third branch will output three local features and one global feature. Therefore, the multi-granularity feature extraction network E enc (x) will extract eight sets of feature codes. The number of groups can be variable, and more scales can be considered.

[0072] To train the multi-granularity feature extraction network module Eenc(x) and improve its performance, an Adam optimizer with β1 = 0.9 was used during training, and only the target-related branch R of the multi-granularity feature extraction model was used. θ and loss function The training data was trained for 300 epochs with a learning rate of 2e-4.

[0073] It should also be noted that global pooling is used for all global features, while max pooling is used for all local features. Furthermore, to reduce the number of model parameters and improve model speed, all local features are uniformly reduced to 256 dimensions using Reduction before being input into the classifier for unified object detection.

[0074] First, target annotations are used only to decompose and identify features related to and unrelated to the target. Although training two sets of branch modules enables them to extract these features separately, they originate from the same feature extractor E. enc (x) causes generator E gen(·) This approach tends to rely heavily on identifying target-independent features to synthesize new target images, thus neglecting the role of identifying target-related features and diverting the feature de-entanglement process. To avoid this problem, we restrict the identification of target-independent features to approximate a normal distribution N(0; 1), with the mean and unit variance approaching 1 and 0, respectively. The loss of identifying target-independent features is then addressed. as follows:

[0075]

[0076] in, p represents the true probability distribution, and q represents the approximate probability distribution. This represents the target label c and feature f. R Associated classifier parameters; It is the index of the target label c. It takes the value 1 when the target predicted by the three branches in the multi-granularity feature extraction network module is the same as the target label, and 0 otherwise. f represents the embedding function for extracting target-related features, which yields k target-related features of the input image encoded using the R-th encoding method, where k = 1, 2…K, and K represents the total number of input images containing the same target; R D represents the global features and local features of different scales extracted by a multi-granularity feature network using the R-th encoding method, where R = 1, 2…8; KL It represents the KL divergence of a feature that is independent of the constraint target and approximates a normal distribution N(0,1).

[0077] Step 2: Input the target-irrelevant features and target-related features into the generator in the generative adversarial network module for semantic feature reconstruction, deentangle the features from the single feature representation, and generate a new image;

[0078] Loss L in image reconstruction process r The L1 loss function is used, as follows:

[0079]

[0080] Where |||1 represents the absolute value of L1 loss, E represents the generative adversarial network module, and x i Images i and f representing the same target R (i), f U (i) represent the images x from the same input target. i The extracted features are those relevant to the target and those irrelevant to the target, where i can take the value of a or p. R and f U This is the result of concatenating eight sets of feature codes;

[0081] The image generation and recognition stages will freeze the parameters of the multi-granularity feature extraction network and train the target-independent branch U. θ Generative Adversarial Network Module E gen And the discriminator E in the generative adversarial network module dis And obtain the corresponding loss function L. r This phase iterates for 200 epochs at a learning rate of 2e-4. The generative adversarial network module E is trained. gen The Adam optimizer with β2 = 0.999 was used, and the discriminator E... dis The SGD optimizer with a momentum of 0.9 was then used.

[0082] The so-called unentanglement from a single feature representation means:

[0083] By using a generative adversarial network module to perform feature unwrapping and synthesize a new image through the same target transformation, based on global-level and local-level feature exchange strategies;

[0084] Global-level feature swapping strategies involve swapping different images x of the same target. a x p Target-related feature encoding ensures that the new image generated by the generative adversarial network module still possesses the visual features matched by the original target, thus generating a different image x of the same target. a x p Synthetic images;

[0085] Because x a x p Although they belong to the same target, they present different images due to different viewpoints, poses, lighting, and backgrounds. Therefore, a global-level feature exchange strategy will force the identification of the target-related branch R. θ Extract features robust to intra-class variations, further mining and enhancing image x. a and image x p The common features between them, while forcibly identifying target-irrelevant branches U θ To extract the unique variable factors within a single image.

[0086] The local-level feature exchange strategy divides the complete feature map into multiple local blocks, performs feature deentanglement and target transformation at the local level, and randomly selects local features of the target from the target-related feature embedding function of any one of the two images of the same target, and exchanges them at the same position with the local features at the corresponding position of the target-related feature embedding function of the other image.

[0087] The expression for global-level feature exchange is:

[0088] G(fR (a)⊕f U (p))

[0089] G(f R (p)⊕f U (a))

[0090] Among them, f R (a), f U (a) Represents the images x from the same input target. a The target-related features and target-irrelevant features extracted from the data, f R (p), f U (p) represent images x from the same input target. p The features extracted are those relevant to the target and those irrelevant to the target. ⊕ represents the concatenation of the two types of features, and G() represents the feature exchange function.

[0091] Generating adversarial network modules requires starting from G(f) R (a)⊕f U Image x is generated in (p)) p At the same time from G(f) R (p)⊕f U (a) generates the image x p Image x of the same target a .

[0092] Global-level feature exchange loss Defined as:

[0093]

[0094] Local-level feature exchange loss Defined as:

[0095]

[0096] Where |||1 represents the absolute value of L1 loss, E represents the generative adversarial network module, and x i Images i and f representing the same target R (i), f U (i) represent the images x from the same input target. i The features extracted are those related to the target and those unrelated to the target. The values ​​of i and j are a or p, and S represents a local region transformation operator.

[0097] Local-level target transformation has the following advantages: 1) This transformation enables the network model to see various combinations of target-related features of individual local parts, further regularizing the feature de-entanglement process; 2) It promotes feature consistency between corresponding parts of the image.

[0098] Step 3: Input the newly generated image into the discriminator in the generative adversarial network module to identify the authenticity and category of the image, and output the final recognition result;

[0099] In step 3, the discriminator includes a domain-discriminator D. d And class-discriminator D c Domain-Discriminator D d The class-discriminator D distinguishes between real and synthetic images. c The obtained target image and the input target image will be classified.

[0100] The loss of the domain discriminator is defined as follows:

[0101]

[0102] The loss of the class-discriminator is defined as follows:

[0103]

[0104] Where, x i Images i and f representing the same target R (i), f U (i) represent the images x from the same input target. i The features extracted are those related to the target and those unrelated to the target. The value of i is either a or p, and S represents a local region transformation operator.

[0105] When a generator synthesizes an image that is difficult to distinguish, if there is a lack of sufficient target-related information for judgment, the class-discriminator will incorrectly predict the target label of the generated image. Therefore, in adversarial learning, the generator will accurately identify target-related features in order to synthesize specific target images.

[0106] The generator has six identical convolutional blocks, each consisting of a transposed convolutional layer, Batch Normalization (BN), Leaky ReLU, and Dropout. The generator takes as input target-related and target-independent feature information, a random noise vector, and one-hot encoded target labels.

[0107] The domain-discriminator and class-discriminator share five convolutional blocks, each consisting of a convolutional layer with a stride of 2, instance normalization, and Leaky ReLU. In the domain-discriminator, two additional blocks are added, outputting a 12×4 feature map which is then fed into PatchGAN. In the class-discriminator, one additional block is added, followed by a fully connected layer.

[0108] A target re-identification network based on feature untangling is constructed by sequentially cascading multi-granularity feature extraction network modules and generative adversarial network modules. During the training process of the target re-identification network based on feature untangling, the losses of each step from step 1 to step 3 are accumulated, and the minimum accumulated loss is taken as the training objective.

[0109] The accumulated loss L total Specifically as follows:

[0110]

[0111] Where, λ R , λ U , λ r , λ gs , λ ps , λ d , λ c The target classification loss is represented in sequence. KL divergence loss, L1 loss, global-level feature exchange loss, local-level feature exchange loss, domain-discriminator loss, and the scaling factor of domain-discriminator loss.

[0112] During training, the input image size was 256×256. Data augmentation was performed on the input image using horizontal flipping and random erasure, but random erasure was only used in the multi-granularity feature extraction network module. The batch size was set to 32, with each batch selecting 8 different target objects, and 4 images were sampled for each target.

[0113] The global and local target-related feature exchange phases will train the entire network end-to-end at a learning rate of 2e-5, with a training time of 100 epochs.

[0114] Due to λ U Larger values ​​are detrimental to training; therefore, the KL divergence loss function should be used. The proportionality coefficient λ U The value is 0.01, used to regularize the entanglement process of features, and the L1 loss function is taken. r The proportionality coefficient λ r Global-level feature exchange loss function The proportionality coefficient λ gs and the local-level feature exchange loss function The proportionality coefficient λ ps The domain-discriminative loss function L is 10. D The proportionality coefficient λ d The target classification loss function is 1. The proportionality coefficient λ R The domain-discriminator loss function L is 20. CThe proportionality coefficient λ c The value is 2.

[0115] The technical solution of this invention proposes a GPS-GAN framework, which combines adversarial generative training without the need for additional annotation information. By unwrapping target features, irrelevant features of the target are removed, while target-related features are mined and enhanced, thereby making accurate judgments about the target.

[0116] The present invention provides some experimental results. Table 1 shows the performance comparison between GPS-GAN (the target re-identification method described in the present invention examples) and other re-identification methods, including two local segmentation methods, PCB and MGN, and three feature unwrapping methods, FD-GAN, D... 2 The AE and DG-Net datasets were compared, and the retrieval accuracy (mAP), Rank 1, and Rank 5 of GPS-GAN were tested on the VeRi-776 and VehicleID vehicle datasets. GPS-GAN uses the same multi-granularity feature extraction network and network parameters as MGN, and the experimental results in Table 1 show that the network performance of GPS-GAN is significantly higher than that of MGN.

[0117] Table 1. Performance comparison of different methods on the VeRi-776 and VehicleID datasets.

[0118]

[0119] Similar to GPS-GAN, FD-GAN uses additional target labels to extract target-related and target-independent features. However, FD-GAN does not consider that other intra-class variable factors also need to be separated. GPS-GAN, on the other hand, can extract other target-independent features such as pose, size, and background from a single feature representation using only the target label, thus exhibiting greater robustness to intra-class variations.

[0120] DG-Net also uses feature unentanglement technology, but the target appearance and structural features in DG-Net differ from the target-related and target-independent features in GPS-GAN. Furthermore, DG-Net's training method is more unique than GPS-GAN; it simultaneously unentangles features from a pair of triplet samples. To achieve the reconstruction constraint after exchanging appearance features between negative pairs, it uses an encoding-level reconstruction loss. This requires the image synthesized by the generator module to be input again into the feature extractor to obtain target appearance and structural feature codes for comparison, thus increasing the training difficulty.

[0121] Table 2 Performance comparison of different loss combinations in GPS-GAN on the VeRi-776 dataset

[0122]

[0123] Table 2 presents ablation analysis of different loss combinations in GPS-GAN on the VeRi-776 dataset. The first row represents common discriminative methods, while the second row onwards represents complete feature untangling training methods, constraining target-irrelevant features and feature extraction. Comparing these two datasets, we can conclude that using feature untangling techniques to decompose target-related and irrelevant features from a single feature representation helps the surrogate network learn better and achieves better untangling results. The comparison in the third, fourth, and fifth rows illustrates that incorporating global and local-level target-related feature transformations can better uncover and strengthen common features between positive pairs. While strengthening the common features of positive pairs, it forces the target-irrelevant branch to capture the unique variable factors of a single image, thereby promoting the image generation effect of the generative module and further improving the performance of target re-identification. The last three rows of data incorporate discriminative constraints on the discriminator, reflecting the complementary characteristics of the domain-discriminator and class-discriminator. This forces the generator module to generate more realistic synthetic images, which will better guide the network to distinguish invalid intra-class variable features and regularize the untangling process.

[0124] Table 3 compares the impact of the number of local features in the multi-granularity feature extractor on the performance of GPS-GAN, and further compares the differences in retrieval performance between using local-level target feature exchange strategies and not using them. It is important to note that the network architecture based on global feature representation does not include local exchange. Performance comparisons confirm that, compared to limited global feature representation, the addition of a branch structure of Part 1, Part 2, and Part 3 significantly improves the performance of GPA-GAN, increasing Rank-1 / mAP from 88.72% / 63.27% to 94.3% / 73.1% (+5.58% / 9.83%). From the perspective of local target features, the first three rows of Table 3 compare the performance differences of three single networks. Part 2 and Part 3 combine fine-grained global and local feature information, thus supplementing the discriminative feature representation of local regions. On the other hand, from the perspective of global features, the three combinations of networks under the corresponding branch settings are compared. The features learned by the three branches of MGN achieve further performance improvements compared to Part 1-2 and Part 1-3. As theoretically envisioned, the interaction between the branches fills the blind spots in their respective learning processes, collaboratively supplementing the common backbone with low-level recognition information, which is the main reason for its performance improvement.

[0125] Table 3 Performance comparison of different local combination settings in GPS-GAN on the VeRi-776 dataset

[0126]

[0127] From a theoretical perspective, the local-level target feature exchange strategy assumes that the horizontal region in a single target image contains enough discriminative features to distinguish the target. Thus, it can be seen that it exhibits good performance generalization on different local combination structures. As the number of branches increases, the performance of GPS-GAN further improves, which indirectly confirms the regularization constraint effect of the local-level target feature exchange strategy on generative models.

[0128] Example 2:

[0129] A target re-identification system based on feature unwrapped representation, comprising:

[0130] The multi-granularity feature extraction network module extracts multi-granularity features from the image to be identified and classifies them into features irrelevant to the target and features relevant to the target.

[0131] Generative Adversarial Network (GAN) module: Inputting target-irrelevant features and target-related features into the generator in the GAN module for semantic feature reconstruction, untangling the single feature representation, and generating a new image; inputting the generated new image into the discriminator in the GAN module to identify the authenticity and category of the image, and outputting the final recognition result;

[0132] The network construction module consists of a multi-granularity feature extraction network module and a generative adversarial network module, which are cascaded together to build a target re-identification network based on feature untangling.

[0133] In the training module, during the training process of the target re-identification network based on feature untangling, the losses executed in the multi-granularity feature extraction network module and the generative adversarial network module are accumulated, and the minimum of the accumulated loss is taken as the training objective.

[0134] The multi-granularity feature extraction network module, generative adversarial network module, and training module use the aforementioned target re-identification method based on feature unwrapped representation for image data processing.

[0135] For details on the implementation process of each module, please refer to the methods described above; they will not be repeated here. It should be understood that the above division of functional modules is merely a logical functional division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the integrated units described above can be implemented in hardware or as software functional units.

[0136] Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A target re-identification method based on feature unwrapped representation, characterized in that, include: Step 1: Use the multi-granularity feature extraction network module to extract multi-granularity features from the image to be identified and classify them into features irrelevant to the identification target and features relevant to the identification target; Step 2: Input the target-irrelevant features and target-related features into the generator in the generative adversarial network module for semantic feature reconstruction, deentangle the features from the single feature representation, and generate a new image; Step 3: Input the newly generated image into the discriminator in the generative adversarial network module to identify the authenticity and category of the image, and output the final recognition result; A target re-identification network based on feature untangling is constructed by using a cascaded multi-granularity feature extraction network module and a generative adversarial network module. During the training process of the target re-identification network based on feature untangling, the losses of each step 1 to step 3 are accumulated, and the minimum accumulated loss is taken as the training objective. The so-called unentanglement from a single feature representation means: By using a generative adversarial network module to perform feature unwrapping and synthesize a new image through the same target transformation, based on global-level and local-level feature exchange strategies; Global-level feature swapping strategies involve swapping different images of the same target. , Target-related feature encoding ensures that the new image generated by the generative adversarial network module still possesses the visual features matched by the original target, thus generating different images of the same target. , Synthetic images; The local-level feature exchange strategy divides the complete feature map into multiple local blocks, performs feature deentanglement and target transformation at the local level, and randomly selects local features of the target from the target feature-related feature embedding function of any one of the two images of the same target, and exchanges them at the same position with the local features at the corresponding position of the target feature-related feature embedding function of the other image. The expression for global-level feature exchange is: ； ； in, , These represent images of the same target from the input. The features relevant to the target and the features irrelevant to the target extracted from the data are: , These represent images of the same target from the input. The features relevant to the target and the features irrelevant to the target extracted from the data are: This indicates the concatenation of two features. This represents the feature exchange function.

2. The method according to claim 1, characterized in that, Global-level feature exchange loss Defined as: ； Local-level feature exchange loss Defined as: ； in, Let E represent the absolute value of the L1 loss, and let E represent the generative adversarial network module. Image i representing the same target, , These represent images of the same target from the input. The features extracted are those related to the target and those unrelated to the target. The value of i is either a or p, and S represents a local region transformation operator.

3. The method according to claim 2, characterized in that, Loss for identifying target-irrelevant features as follows: ； in, ; For the true probability distribution, It is an approximate probability distribution. , This represents the target label c and features. Associated classifier parameters; It is the index of the target label c. It takes the value 1 when the target predicted by the three branches in the multi-granularity feature extraction network module is the same as the target label, and 0 otherwise. The embedding function is used to extract target-related features, which yields k target-related features of the input image encoded using the R-th encoding method, where k = 1, 2, ..., K, and K represents the total number of input images of the same target. This represents the global features and local features of different scales extracted by a multi-granularity feature network using the R-th encoding method. ; Features that are independent of the constraint target approximate a normal distribution KL divergence under [condition].

4. The method according to claim 1, characterized in that, The multi-granularity feature extraction network adopts the MGN multi-granularity learning structure and is composed of multiple layers of shared convolutional layers stacked together; the backbone in the multi-granularity feature extraction network is a ResNet50 loaded with ImageNet pre-trained weights. A multi-granularity feature extraction network was trained using Cross-Entropy, and a target classification loss was set. as follows: ； in, This represents the global features and local features of different scales extracted by a multi-granularity feature network using the R-th encoding method. ; This represents the target label c and features. Associated classifier parameters; It is the index of the target label c. It takes the value 1 when the target predicted by the three branches in the multi-granularity feature extraction network module is the same as the target label, and 0 otherwise. C represents the total number of target labels. The embedding function is used to extract target-related features, which yields k target-related features of the input image encoded using the R-th encoding method, where k = 1, 2, ..., K, and K represents the total number of input images of the same target. Representation of features The corresponding target probability passes The result after that, .

5. The method according to claim 1, characterized in that, The process of inputting target-independent and target-related features into a generative adversarial network module for semantic feature reconstruction, and de-entanglement from a single feature representation to generate a new image, is the loss in the image reconstruction process. The L1 loss function is used, as follows: ； in, Let E represent the absolute value of the L1 loss, and let E represent the generative adversarial network module. Image i representing the same target, , These represent images of the same target from the input. The features extracted are those related to the target and those unrelated to the target, where i can take the value a or p.

6. The method according to claim 1, characterized in that, In step 3, the discriminator includes a domain-discriminator. And class-discriminator Domain-Discriminator Class-discriminator distinguishes between real and synthetic images. The obtained target image and the input target image will be classified. The loss of the domain discriminator is defined as follows: ； The loss of the class-discriminator is defined as follows: ； in, Image i representing the same target, , These represent images of the same target from the input. The features extracted are those related to the target and those unrelated to the target. The value of i is either a or p, and S represents a local region transformation operator.

7. The method according to claim 1, characterized in that, The accumulated loss Specifically as follows: ； in, , , , , , , The target classification loss is represented in sequence. The loss includes KL divergence loss, L1 loss, global-level feature exchange loss, local-level feature exchange loss, domain-discriminator loss, and the scaling factor of domain-discriminator loss.

8. A target re-identification system based on feature unwrapped representation, characterized in that, include: The multi-granularity feature extraction network module extracts multi-granularity features from the image to be identified and classifies them into features irrelevant to the target and features relevant to the target. Generative Adversarial Network (GAN) module: Inputting target-irrelevant features and target-related features into the generator in the GAN module for semantic feature reconstruction, untangling the single feature representation, and generating a new image; inputting the generated new image into the discriminator in the GAN module to identify the authenticity and category of the image, and outputting the final recognition result; The network construction module consists of a multi-granularity feature extraction network module and a generative adversarial network module, which are cascaded together to build a target re-identification network based on feature untangling. In the training module, during the training process of the target re-identification network based on feature untangling, the losses executed in the multi-granularity feature extraction network module and the generative adversarial network module are accumulated, and the minimum of the accumulated loss is taken as the training objective. The multi-granularity feature extraction network module, generative adversarial network module, and training module use the target re-identification method based on feature unwrapped representation as described in any one of claims 1-7 to process image data.

Images