A target re-identification method and device based on a frequency domain guidance prompt and a medium

Abstract

The application belongs to the technical field of computer vision and artificial intelligence, and discloses a target re-identification method based on frequency domain guidance prompt, equipment and medium, the method comprises the following steps: obtaining a target image to be processed; performing frequency domain decomposition on the target image to extract and fuse high-frequency components to obtain a high-frequency detail image; determining a plurality of high-frequency image blocks containing fine-grained texture information in the image according to the high-frequency detail image, and encoding to generate instance tokens; obtaining a visual prototype corresponding to the image identity from a prototype library, and generating a prototype token; embedding the instance token and the prototype token into a text template to generate an image-specific personalized text prompt; finally, constructing a composite image-text sample pair containing difficult positive and negative samples based on the generated personalized text prompt, and training a visual-language pre-training model using the cross-modal contrast loss mined by the composite sample. The application generates a personalized text prompt for each image and effectively improves the performance of target re-identification.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, and in particular to a target re-identification method, device and medium based on frequency domain guidance and prompting. Background Technology

[0002] Object re-identification (Re-ID) is an important research direction in computer vision. Its core task is to retrieve and match objects with the same identity, such as pedestrians or vehicles, in non-overlapping cross-camera networks. Traditional object re-identification methods mostly focus on single-modal visual information processing, using complex convolutional neural networks (CNNs) or visual transformer (ViT) architectures to extract robust visual appearance features from RGB images and perform metric learning to achieve object matching and retrieval. While these methods have made some progress, their reliance solely on visual representations to describe semantically rich object instances limits their performance improvement.

[0003] In recent years, with the emergence of large-scale vision-language pre-trained models (such as CLIP), their powerful cross-modal alignment and reasoning capabilities have brought breakthroughs to many downstream vision tasks. Researchers have begun to try to apply CLIP models to the field of object re-identification. For example, pioneering works such as CLIP-ReID have achieved remarkable results by generating learnable text prompts for each identity category (e.g., "A photo of [X1][X2][X3][X4]person"), transforming the unimodal re-identification task into an image-text cross-modal matching task.

[0004] However, existing methods of this kind have significant technical drawbacks. They typically generate a uniform text cue for all image instances under the same identity (category), forming a "many-to-one" mapping. This design ignores subtle differences (such as pose, lighting, occlusion, etc.) between different image instances under the same identity, resulting in a lack of diversity in the generated image-text sample pairs. Furthermore, the sampling of positive and negative sample pairs during training is usually limited to a small batch of data, making it difficult to discover truly challenging "hard samples," which greatly limits the realization of the CLIP model's powerful cross-modal inference potential.

[0005] Therefore, how to generate more personalized text descriptions for each image instance and construct more challenging image-text training sample pairs to fully explore the cross-modal representation capabilities of visual-language models is an urgent problem to be solved in the current field of object re-identification. Summary of the Invention

[0006] The purpose of this invention is to provide a target re-identification method, device, and medium based on frequency domain guidance prompts, so as to solve the problems of insufficient diversity of image-text sample pairs and insufficient exploitation of the cross-modal reasoning ability of the model caused by the use of category-level unified text prompts in the existing CLIP-based re-identification method.

[0007] In a first aspect, the present invention provides a target re-identification method based on frequency domain guidance and prompting, comprising the following steps:

[0008] Obtain the target image to be processed;

[0009] The target image to be processed is decomposed in the frequency domain, and the high-frequency components of the target image are extracted and fused to obtain a high-frequency detail map;

[0010] Based on the high-frequency detail map, multiple high-frequency image blocks containing fine-grained texture information are determined in the target image;

[0011] The multiple high-frequency image blocks are input into a self-attention encoder for feature encoding, and the encoded feature vectors are extracted.

[0012] The feature vector is projected onto the text space to generate an instance token representing the instance-level information of the target image.

[0013] Obtain the visual prototype corresponding to the identity identifier of the target image from the preset prototype library;

[0014] The visual prototype is projected onto the text space to generate a prototype token that represents the prototype-level information of the target image.

[0015] The instance token and the prototype token are embedded into a preset text prompt template to generate a personalized text prompt for frequency domain guidance specific to the target image;

[0016] The target image and the personalized text prompt are respectively input into the image encoder and text encoder of the vision-language pre-trained model to obtain image features and text features;

[0017] Based on the image features and the text features, the vision-language pre-trained model is trained using a preset loss function to complete the target re-identification task.

[0018] As an optional implementation of the first aspect of this application, the step of performing frequency domain decomposition on the target image to be processed, extracting and fusing the high-frequency components of the target image to obtain a high-frequency detail map specifically includes: performing second-order decomposition on the target image using discrete Haar wavelet transform to obtain diagonal detail sub-bands, horizontal detail sub-bands and vertical detail sub-bands; performing pixel-level summation on the diagonal detail sub-bands, horizontal detail sub-bands and vertical detail sub-bands to obtain fused high-frequency components; and performing resampling and upsampling processing on the fused high-frequency components using bilinear interpolation to generate a high-frequency detail map with the same size as the target image.

[0019] As an optional implementation of the first aspect of this application, the step of determining multiple high-frequency image blocks containing fine-grained texture information in the target image based on the high-frequency detail map specifically includes: dividing the target image and the high-frequency detail map into an image block sequence and a high-frequency image block sequence of the same length, respectively; performing a pixel-level averaging operation on each high-frequency image block in the high-frequency image block sequence to obtain a mean value used to characterize the detail information of the corresponding image block; retaining the image block index of the high-frequency component with a preset ratio based on the mean value; and mapping the multiple high-frequency image blocks from the image block sequence using the image block index.

[0020] As an optional implementation of the first aspect of this application, the method further includes: the preset prototype library is a key-value pair structure used to store visual prototypes and text prototypes of all categories in the training set; the visual prototype is the feature center extracted by the image encoder for all images in each category, and is updated after each training round; the text prototype is the feature center extracted by the text encoder for all frequency domain-guided personalized text prompts in each category, and is updated synchronously with the visual prototype.

[0021] As an optional implementation of the first aspect of this application, before training the visual-language pre-trained model using a preset loss function, the method further includes constructing composite image-text sample pairs. Specifically, this includes: sampling anchor images and their corresponding frequency-domain guided personalized text prompts across the entire training set; sorting candidate text prompts with the same identity as the anchor image in ascending order based on the inner product similarity with the personalized text prompts of the anchor image, and sampling to obtain a set of difficult-to-correct texts; sorting candidate text prompts with different identity as the anchor image in descending order based on the inner product similarity with the personalized text prompts of the anchor image, and sampling to obtain a set of difficult-to-negative texts; and constructing composite image-text sample pairs containing ordinary image-text pairs, difficult-to-correct image-text pairs, and difficult-to-negative image-text pairs based on the anchor image, the set of difficult-to-correct texts, and the set of difficult-to-negative texts.

[0022] As an optional implementation of the first aspect of this application, the method further includes, in which the preset loss function includes a cross-modal contrastive loss for composite sample mining, used to drive the model to learn the cross-modal correlation of the composite image-text sample pair, specifically including: the cross-modal contrastive loss for composite sample mining includes image-text contrastive loss and text-image contrastive loss; the image-text contrastive loss is used to narrow the distance between the cross-modal fusion features of the anchor image and the most difficult positive text features, while widening the distance between them and multiple difficult negative text features sampled within the batch; the text-image contrastive loss is used to narrow the distance between the text features of the anchor text and the most difficult positive cross-modal fusion features, while widening the distance between them and multiple difficult negative image features sampled within the batch.

[0023] As an optional implementation of the first aspect of this application, the method further includes, wherein the preset loss function is a total objective loss function, to jointly supervise the training process of the vision-language pre-trained model, and the total objective loss function further includes: cross-entropy loss for basic identity classification; triplet loss for metric learning; and image-text cross-entropy loss.

[0024] As an optional implementation of the first aspect of this application, the image-text cross-entropy loss is used to pass the identity supervision information learned by the frequency domain-guided personalized text prompts to the image encoder, specifically including: obtaining the classification prediction result output by the image encoder for the target image; obtaining the text prototype corresponding to the real identity label of the target image from the prototype library, and using it as a soft label distribution; calculating the cross-entropy between the classification prediction result and the soft label distribution to obtain the image-text cross-entropy loss.

[0025] Thirdly, embodiments of this application provide an electronic device, which includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the steps of the methods described in the first and second aspects.

[0026] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the methods described in the first and second aspects.

[0027] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0028] 1. Enhanced diversity and personalization of image-text pairs: The Frequency Domain Guided Personalized Prompts (FIP) designed in this invention generates a unique text description for each image by combining instance tokens representing image details and prototype tokens representing commonalities, achieving a one-to-one image-text mapping. This greatly enriches the diversity of training data and overcomes the representational limitations caused by the many-to-one mapping in existing methods.

[0029] 2. Deeply explore the model's cross-modal reasoning capabilities: By introducing a hard-cue sampler, the most challenging hard-positive and hard-negative sample pairs are mined across the entire dataset, and a cross-modal contrastive loss for composite sample mining is designed for targeted learning. This enables the model to learn cross-modal attitude quantities in a more complex semantic space, thereby more fully stimulating and utilizing the potential reasoning capabilities of visual-language models such as CLIP.

[0030] 3. Improved accuracy and robustness of target re-identification: Instance tokens in FIP utilize high-frequency image information, enabling text prompts to focus on the fine-grained texture of the target, enhancing the model's ability to identify subtle differences. Simultaneously, prototype tokens ensure the stability of identity information in personalized expressions. The combination of these two aspects, along with effective learning from difficult samples, results in a re-identification model with higher accuracy and stronger robustness on mainstream benchmarks.

[0031] 4. Optimized training paradigm: This invention adopts a single-stage end-to-end training method, extracts information from images in real time to generate prompts, and dynamically updates the prototype library. Compared with some multi-stage training methods, the process is simpler, the optimization goals are more consistent, and it helps to achieve a better balance between efficiency and accuracy. Attached Figure Description

[0032] Figure 1 This is a schematic diagram of the overall framework of a target re-identification method based on frequency domain guidance and prompting (FIPCIP-ReID) proposed in an embodiment of the present invention;

[0033] Figure 2 This is a schematic diagram illustrating the generation of frequency domain guidance personalized prompts (FIP) in an embodiment of the present invention;

[0034] Figure 3 This is a visualization example of the correlation between high-frequency components in an embodiment of the present invention;

[0035] Figure 4 This is a visual comparison of the attention maps of FIPCIP-ReID proposed in this embodiment of the invention and the existing method CLIP-ReID;

[0036] Figure 5This is a t-SNE visualization comparison chart of the visual features extracted by FIPCIP-ReID proposed in this embodiment of the invention and the existing method CLIP-ReID. Detailed Implementation

[0037] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0038] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0039] Example

[0040] Reference Figure 1 This invention provides a target re-identification method based on frequency domain-guided cues, the overall framework of which is called FIPCIP-ReID. The core of this method lies in: Step 1, frequency domain-guided personalized cues (FIP) generation; Step 2, construction of composite image-text sample pairs; and Step 3, model training and optimization. Assuming a training set... ,in This represents the number of target images in the training set. First, input the target images and their corresponding FIP text, where the training set... FIP text T corresponding to each target image in the file Embed an instance token[ ] and a prototype token[ It is worth emphasizing that a prototype library of key-value pair structures was introduced during the training process. It was used to memorize the feature centers of the two modalities for each class, among which and Let these represent the visual prototype and text prototype of the k-th category, respectively. This indicates the number of categories in the training set. Then, each target image is fed into the hard cue sampler. The corresponding sets of difficult positive and difficult negative texts are generated by sampling. Ordinary sample pairs, difficult positive sample pairs, and difficult negative sample pairs are constructed from the target image and the difficult text sets, respectively, and simultaneously input into the image encoder of the vision-language pre-trained model (CLIP). and text encoder In the middle. Finally, cross-entropy loss is used. Triple loss Cross-modal contrastive loss for composite sample mining and image-text cross-entropy loss Conduct joint supervision training.

[0041] Specifically, the steps of this method will be explained in detail below.

[0042] Step 1: Generation of Frequency Domain Guided Personalized Prompts (FIP)

[0043] To construct more diverse image-text pair combinations to mine the cross-modal capabilities of CLIPs, this invention proposes a frequency-domain guided personalized cue, called FIP. For example... Figure 2 As shown, taking pedestrian targets as an example, FIP combines instance-level and prototype-level semantic information to generate personalized text cues for each image. Given a target image... The corresponding FIP text It contains an instance token[ ] and a prototype token[ The template content is "A photo of [ ] [ For vehicles, the template content is "A photo of [person."] ] [ The instance token represents personalized information about the target image, while the prototype token reflects the identity-level information contained in the target image. In other words, by embedding a combination of instance tokens and prototype tokens in the FIP text, a one-to-one image-text mapping relationship is constructed for each target image.

[0044] For instance token[ This invention preserves the high-frequency components of the target image to express the individuality of the FIP (Frequency Injection Process). First, the target image is processed using Discrete Haar Wavelet Transform (DHWT). Perform a second-order decomposition to obtain four sub-bands. , , and Based on existing experience, Represents coarse-grained information about the image, while , and These represent the diagonal, horizontal, and vertical detail subbands of the image, respectively. These are high-frequency components that preserve the fine-grained texture of the image. Therefore, performing pixel-level summation on these three wavelet subbands yields a fused high-frequency component. It contains detailed texture information of the original image in different directions. Subsequently, a bilinear interpolation method is used to fuse the high-frequency components. Resampling and upsampling are performed to obtain a result similar to the target image. High-frequency detail images of the same size At the same time, the target image and high-frequency detail diagram They were divided into two image block sequences of the same length. and ,in This indicates the number of image patches. Then, Each image patch in A pixel-level averaging operation is performed to obtain the mean of each image patch. to indicate The detailed information corresponds to the image patch index. More importantly, this invention preserves... Center front Image patch indexes of high-frequency components at a % ratio are used to map and obtain The corresponding high-frequency image block sequence Where z represents the length of the sequence, that is... , This represents the z-th token. Through self-attention encoding ( After that, the token at the very beginning of the output sequence is extracted as a feature vector representing high-frequency components, and then projected onto the text space of the FIP template through a multilayer perceptron (MLP) to generate instance tokens. As shown in formula (1):

[0045] [ ]

[0046] For the prototype token[ In this invention, the central features of all target images within each category are considered the prototype of that category. Therefore, this invention maintains a prototype library. This is used to store the visual and textual prototypes of all classes in the training set. Taking the k-th class as an example, the image encoder of the Visual-Language Pre-trained Model (CLIP) is used in the initial training phase. Initialize a visual feature center for the k-th class as the initial visual prototype. It is in the image encoder It is updated after each fine-tuning of the training rounds. From Take out Then, through MLP mapping, [ This is then embedded into the FIP template. This approach ensures that the prototype token can stably and accurately reflect the characteristics of the k classes. Similarly, text prototypes... At the end of each training epoch, record the FIP feature centers of k classes and follow the pattern of k classes. The same update strategy.

[0047] The design of FIP (Framework Indicator) enables text data to include both instance-level and prototype-level semantics. The main reasons can be summarized as follows: (1) Instance-level semantics accurately express the fine-grained texture information of images by preserving high-frequency components, thereby enabling personalized text prompts for each image; (2) Prototype-level semantics represent the central representative of each identity, which can retain complete identity information while personalizing text prompts. In short, the combination of the two can achieve a one-to-one mapping between images and text.

[0048] Step 2: Construction of composite image-text sample pairs

[0049] Existing CLIP-based re-identification work samples positive and negative image-text pairs within a small batch of data to optimize the CLIP image encoder. Perform a parameter update once. This limited range of simple graph-text pair combinations severely restricts CLIP's cross-modal reasoning capabilities. To address this issue, this invention proposes a hard cue sampler. To construct composite image-text sample pairs, the sampler samples across the entire training set, acquiring challenging sets of hard-to-positive and hard-to-negative texts, thus providing the model with suitable image-text pair combinations. Given an anchor image... And corresponding personalized text prompts for frequency domain guidance ,pass In the entire training set Sampling the corresponding difficult text set within the range and difficult text collection More specifically, The sampling process follows two settings: (1) for In terms of ensuring prototype With the entire training set Difficult text of candidate text prototype Under the premise of equality, calculate and Extracted text features and The inner product between them. Then sort them according to the ascending order. Placed element by element (2) On the contrary, and candidate text Under the condition that the prototypes are not equal, place them according to the descending order sorting rule. The entire sampling process can be written as formula (2):

[0050]

[0051] in, and Representing anchor images respectively The set of difficult positive texts and the set of difficult negative texts. Representing vectors sum vector The inner product, (·)and (·) represents the sorting functions for ascending and descending order, respectively.

[0052] It is worth emphasizing that, compared to the original mini-batch sampling strategy, the hard cue sampler of this invention decouples the dependence on the sampling range of a single mini-batch, focusing on sampling within the entire dataset. and For an anchor image From respectively and Extract the elements from the set, namely the difficult-to-positive text and the difficult-to-negative text, and quickly target the anchor image. Composite image-text pairs are constructed, namely ordinary image-text pairs, hard positive image-text pairs, and hard negative image-text pairs, which will provide more diverse image-text sample selections for subsequent CLIP fine-tuning.

[0053] Step 3: Model Training and Optimization

[0054] To dynamically enable FIP to accurately describe each instance of each identity, this invention designs a cross-modal contrastive loss for composite sample mining. = + Metric learning for cross-modal applications includes a graph-text contrastive loss. Loss compared with text-image They are defined as formulas (3) and (4) respectively:

[0055]

[0056]

[0057] in and Representing anchor images respectively The corresponding cross-modal fusion features and the most difficult text features. and These represent anchor text. The corresponding text features and the most difficult positive cross-modal fusion features. and These represent personalized text prompts. and personalized text prompts Textual features. and Representing images respectively and images Cross-modal fusion features. and These represent a sample pair and temperature parameter in a small batch, respectively. express The sampling range of difficult-to-bear texts with a small batch size in the middle. Conversely, Represents anchor text The sampling range of the corresponding first small batch number of difficult-to-bear images. This represents the cosine similarity function.

[0058] Finally, similar to CLIP-ReID, a graph-text cross-entropy loss is introduced. The identity supervision information learned by FIP is passed to the image encoder of CLIP. For example, in formula (5):

[0059]

[0060] in This represents soft labels in the true label distribution. and These are self-prototype libraries The middle label is k and The prototype, also known as the text prototype.

[0061] Unlike the two-stage training CLIP-ReID, the FIPCIP-ReID proposed in this invention uses single-stage training to fine-tune the CLIP image encoder. This is used as the re-identification model. This design is reasonable because the learnable tokens in FIP are derived from high-frequency information extracted in real time during training and from a dynamically updated prototype library. In summary, the FIPCIP-ReID framework suffers from cross-entropy loss during model training. Triple loss Cross-modal contrastive loss for composite sample mining and image-text cross-entropy loss Under joint supervision, the overall objective loss function is defined by formula (6):

[0062]

[0063] After implementing this method, the performance of the method was verified through experimental analysis, specifically as follows:

[0064] 1. Dataset and Evaluation Metrics

[0065] To verify the effectiveness of the proposed method, two pedestrian re-identification datasets, Market-1501 and MSMT17, and one vehicle re-identification dataset, VeRi-776, were used to perform extensive experiments. Consistent with mainstream re-identification evaluation methods, this invention uses mean average accuracy (mAP) and Rank-1 accuracy in cumulative matching features (CMC) as metrics to evaluate the performance of the re-identification model.

[0066] 2. Implementation details

[0067] This invention employs a CLIP-based VIT image encoder containing 12 Transformer blocks as the re-recognition backbone for fine-tuning. During the training phase, the batch size is set to 64, the number of training epochs is set to 60, and the initial learning rate is set to 3.5 × 10⁻⁶. The model was updated using Adam as the optimizer. The input pedestrian and vehicle images were resized to 256×156 and 256×128 respectively, and then subjected to data augmentation operations including random horizontal flipping, padding, cropping, and erasing. Notably, a penalty term was used... It was empirically set to -1.4. Temperature parameters in It equals 0.07. This experiment was conducted using the Ubuntu 22.04 operating system and PyTorch environment, and was completed with two NVIDIA RTX 5880 Ada GPUs.

[0068] 3. Comparison with existing best methods

[0069] Table 1 reports the performance of our method and other state-of-the-art methods on pedestrian and vehicle re-identification datasets. It is evident that our method achieves the best re-identification performance across all four datasets. Notably, FIPCIP-ReID still offers a performance improvement compared to CLIP-ReID, which has the same backbone. This is primarily because FIPCIP-ReID effectively exploits the cross-modal diversity of difficult image-text pairs across the entire dataset.

[0070] Table 1: Comparison with state-of-the-art methods on pedestrian and vehicle re-identification datasets.

[0071]

[0072] 4. Ablation test

[0073] Comparison of different ablation modules. This invention evaluates the contributions of different ablation modules in FIPCIP-ReID, and the relevant experimental results are summarized in Table 2. "Base" indicates the image encoder using CLIP. As the backbone, "FIP" represents frequency-domain guided personalized cues, while "VP" and "HP" represent cross-modal contrastive loss based on ordinary sample pairs and hard sample pairs, respectively. Two key observations are clearly evident: (1) Compared to "Base" without cues, "Base+FIP+VP" achieved a 1.4% Rank-1 and 0.7% mAP improvement on Market-1501, and a 1.2% Rank-1 and 1.8% mAP improvement on VeRi-776. This is mainly attributed to the fact that the cues of this invention can provide additional identity-specific supervisory information to the CLIP image encoder, enhancing its representation learning ability; (2) The use of ordinary and hard sample pairs, i.e., composite sample pair mining, on the basis of FIP, further significantly improves the re-identification performance. This encouraging result well explains why the FIPCIP-REID method can synergistically learn diverse cross-modal information. In summary, these findings reveal that all individual modules contribute to FIPCIP-ReID.

[0074] Table 2: Ablation experiments of different modules on the Mark-1501 and VeRi-776 datasets

[0075]

[0076] The impact of different prompt settings. Table 3 explores the effectiveness of embedding different tokens in the prompt template. When using only VP, "BP" represents the basic prompt, with the template content being "A photo of [cls].", where [cls] represents pedestrians and vehicles respectively in the pedestrian and vehicle re-identification datasets. "w / o DHWT" indicates that DHWT is not used for high-frequency component extraction. Through analysis of […] Whether or not DHWT ablation experiments are used can confirm that retaining some high-frequency components can better express semantic information that is discriminative in terms of identity. On the other hand, when […] ]and[ When both are embedded in the template "A photo of person / vehicle.", the precision of the FIP of this invention will exceed the hint settings using either token alone. Therefore, the FIP can only fully extract learnable identity information from the hint when it combines both the instance token and the prototype token.

[0077] Table 3: Ablation experiments for different prompts

[0078]

[0079] The impact of different selection ratios of high-frequency components. Table 4 reports the impact of using different proportions of high-frequency components in the FIP on the model's re-identification performance. It can be clearly observed that when the top 25% of high-frequency components are selected to generate the [ When the high-frequency components are retained (50% or 75%), the model achieves optimal performance; conversely, when 50% or 75% of the high-frequency components are retained as text input, the re-recognition performance decreases to some extent. The above results clearly demonstrate that a 25% high-frequency proportion neither lacks key image information nor is excessively redundant, enabling more accurate characterization of image details and maximizing the utilization of the FIP template. ] instance-level semantic information.

[0080] Table 4: FIP extraction before different Comparison of high-frequency components

[0081]

[0082] 4. Visualization Experiments

[0083] Correlation visualization of high-frequency components. Figure 3 Two cases visually demonstrate whether the high-frequency components contain detailed information about the image. The color bars represent the similarity scores between image patches and global features; darker colors indicate higher correlation. The high-frequency components and all image patches are embedded into the corresponding re-identification models in the FIP (Frequency Indicator Image). The cosine similarity between each image patch feature and the global features is calculated sequentially and normalized to obtain the corresponding similarity score. Compared to "FIP (w / o DHWT)," it can be observed that "FIP" has higher similarity in areas where vehicles have discriminative features, such as car logos and sunroofs, and in areas where pedestrians have logos on their bags and clothing. This indicates that the high-frequency components extracted using DHWT contain rich instance details and are more representative of the image's personalized features.

[0084] Attention map visualization analysis. To further verify the superiority of FIPCIP-ReID over CLIP-ReID in target recognition tasks from an intuitive and interpretable perspective, in Figure 4The paper presents multiple sets of pedestrian and vehicle images sampled from the Market-1501 and VeRi-776 datasets, along with their corresponding attention heatmaps. It can be observed that, when faced with interference factors such as viewpoint differences, pose changes, and occlusion, FIPCIP-ReID can accurately activate more discriminative fine-grained semantic regions, while CLIP-ReID's response distribution is relatively diffuse. Specifically, FIPCIP-ReID exhibits high responses to key cues such as car window stickers, car door logos, clothing logos, rooftop advertisements, and backpacks, indicating its stronger fine-grained localization ability and cross-viewpoint robustness. In summary, FIPCIP-ReID outperforms CLIP-ReID to some extent in capturing details in difficult samples and learning discriminative representations.

[0085] t-SNE Visualization Analysis. To evaluate the ability of FIPCIP-REID and CLIP-ReID to distinguish between positive and negative samples, t-SNE technology was used to visualize the feature distribution of 20 randomly selected categories in the Market-1501 dataset. The results are as follows: Figure 5 As shown, different colors represent instances of different classes. Compared to CLIP-ReID, the FIPCIP-REID proposed in this invention makes the feature distribution of images of the same class more compact, while enhancing the distinguishability of feature distributions between different classes. Specifically, in the feature space obtained by FIPCIP-REID, instances of classes represented by clusters such as black, gray, and green are tightly clustered and maintain a relatively large distance from negative samples of other classes; while CLIP-ReID does not have this obvious feature, and the intra-class feature distribution of multiple classes is still relatively loose. The above results show that FIPCIP-REID is better at capturing the similarity between positive samples, while also being more effective at distinguishing the differences between negative samples.

[0086] In summary, this invention proposes an effective architecture to fully exploit the cross-modal diversity of image-text sample pairs for the target image re-identification task (FIPCIP-ReID). A personalized cue named FIP is designed for each image by exploring instance-level and prototype-level cross-modal semantic information in the high-frequency components of the image. Composite image-text sample pairs are constructed for the FIP using a hard cue sampler, and a cross-modal contrastive loss based on composite sample mining is used to fully learn the cross-modal information of a wide variety of image-text pairs. Significant performance on multiple public datasets validates that FIPCIP-ReID can effectively exploit the diversity of cross-modal information in the re-identification model.

[0087] Optionally, embodiments of this application also provide an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the various processes of the above-described target re-identification method embodiment based on frequency domain guidance and cues, and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0088] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described target re-identification method based on frequency domain guidance and prompting, and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0089] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0090] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0091] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0092] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A target re-identification method based on frequency domain guidance and cues, characterized in that, Includes the following steps: Obtain the target image to be processed; The target image to be processed is decomposed in the frequency domain to extract and fuse the high-frequency components of the target image to obtain a high-frequency detail map. Specifically, this includes: performing a second-order decomposition of the target image using discrete Haar wavelet transform to obtain diagonal detail sub-bands, horizontal detail sub-bands, and vertical detail sub-bands; performing pixel-level summation on the diagonal detail sub-bands, horizontal detail sub-bands, and vertical detail sub-bands to obtain fused high-frequency components; and using bilinear interpolation to resample and upsample the fused high-frequency components to generate a high-frequency detail map with the same size as the target image. Based on the high-frequency detail map, multiple high-frequency image blocks containing fine-grained texture information are determined in the target image; The multiple high-frequency image blocks are input into a self-attention encoder for feature encoding, and the encoded feature vectors are extracted. The feature vector is projected onto the text space to generate an instance token representing the instance-level information of the target image. Obtain the visual prototype corresponding to the identity identifier of the target image from the preset prototype library; The visual prototype is projected onto the text space to generate a prototype token that represents the prototype-level information of the target image. The instance token and the prototype token are embedded into a preset text prompt template to generate a personalized text prompt for frequency domain guidance specific to the target image; The target image and the personalized text prompt are respectively input into the image encoder and text encoder of the vision-language pre-trained model to obtain image features and text features; Based on the image features and the text features, the vision-language pre-trained model is trained using a preset loss function to complete the target re-identification task.

2. The method according to claim 1, characterized in that, The step of determining multiple high-frequency image patches containing fine-grained texture information in the target image based on the high-frequency detail map specifically includes: The target image and the high-frequency detail map are respectively divided into image block sequences and high-frequency image block sequences of the same length; Perform a pixel-level averaging operation on each high-frequency image block in the high-frequency image block sequence to obtain the mean value used to characterize the detailed information of the corresponding image block; Based on the mean, retain the image patch index of high-frequency components at a preset ratio; The multiple high-frequency image blocks are obtained by mapping from the image block sequence using the image block index.

3. The method according to claim 1, characterized in that, The method further includes that the preset prototype library is a key-value pair structure used to store visual prototypes and text prototypes of all categories in the training set. The visual prototype is the feature center extracted by the image encoder for all images in each category, and is updated after each training round. The text prototype is a personalized text cue for all frequency domain guidance in each category, with feature centers extracted by the text encoder and updated synchronously with the visual prototype.

4. The method according to claim 1, characterized in that, Before training the vision-language pre-trained model using a preset loss function, the method further includes constructing composite image-text sample pairs, specifically including: Personalized text prompts for anchor images and their corresponding frequency domain guidance are sampled across the entire training set; Among the candidate text prompts that have the same identity as the anchor image, they are sorted in ascending order according to the inner product similarity with the personalized text prompts of the anchor image, and a set of difficult-to-correct texts is obtained by sampling. Among the candidate text prompts with different identities from the anchor image, they are sorted in descending order according to the inner product similarity with the personalized text prompts of the anchor image, and a set of difficult-to-bear texts is obtained by sampling. Based on the anchor image, the set of difficult positive texts, and the set of difficult negative texts, construct composite image-text sample pairs that include ordinary image-text pairs, difficult positive image-text pairs, and difficult negative image-text pairs.

5. The method according to claim 4, characterized in that, The method further includes, in which the preset loss function includes a cross-modal contrastive loss for composite sample mining, used to drive the model to learn the cross-modal correlation of the composite image-text sample pairs, specifically including: The cross-modal contrast loss for composite sample mining includes image-text contrast loss and text-image contrast loss; The image-text contrast loss is used to bring the cross-modal fusion features of the anchor image closer to the most difficult positive text features, while pushing them further away from the multiple difficult negative text features sampled within the batch. The text-image contrast loss is used to bring the text features of the anchor text closer to the most difficult positive cross-modal fusion features, while pushing them further away from multiple difficult negative image features sampled within the batch.

6. The method according to claim 5, characterized in that, The method further includes, wherein the preset loss function is a total objective loss function, to jointly supervise the training process of the vision-language pre-trained model, and the total objective loss function further includes: Cross-entropy loss used for basic identity classification; Triple loss used to measure learning; And the image-text cross-entropy loss.

7. The method according to claim 6, characterized in that, The image-text cross-entropy loss is used to pass the identity supervision information learned from the frequency-domain guided personalized text prompts to the image encoder, specifically including: Obtain the classification prediction result output by the image encoder for the target image; From the prototype library, obtain the text prototype corresponding to the real identity label of the target image, and distribute it as a soft label; The cross-entropy between the classification prediction result and the soft label distribution is calculated to obtain the image-text cross-entropy loss.

8. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein when the program or instructions are executed by the processor, they implement the steps of a target re-identification method based on frequency domain guidance cues as described in any one of claims 1-7.

9. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions, which, when executed by a processor, implement the steps of a target re-identification method based on frequency domain guidance and cues as described in any one of claims 1-7.

Images