Cross-modal pedestrian re-identification method based on relationship-aware adaptive learning optimization

Abstract

This invention specifically relates to a cross-modal person re-identification method based on relation-aware adaptive learning optimization. This method uses a pre-trained visual-language large model as the feature extraction backbone. Addressing the problems of semantic gaps within modalities and insufficient negative sample coverage in existing technologies, it designs a semantic correspondence correction module to mine inter-modal relationships and optimize cross-modal alignment training objectives. It also designs an adaptive negative sample enhancement module to synthesize difficult negative samples based on inter-modal relationships and adaptively allocate weights according to the difficulty level to strengthen the learning of difficult samples. The method integrates semantic correspondence correction loss, adaptive negative sample enhancement loss, and the basic loss function, and fine-tunes the model using the Adam algorithm to achieve multi-objective collaborative optimization. This invention comprehensively mines multi-level sample relationships within and between modalities, optimizes the model learning process, and significantly improves the retrieval accuracy and model robustness of cross-modal person re-identification, possessing significant application value in fields such as security monitoring.

Description

Technical Field

[0001] This invention relates to computer vision, natural language processing, and cross-modal retrieval, and particularly to a natural language cross-modal person re-identification method based on relation-aware adaptive learning optimization. Background Technology

[0002] Natural language cross-modal person re-identification technology aims to retrieve images of target persons from image databases based on text descriptions, and it has wide applications in security monitoring, intelligent video analysis, and other fields. As a fine-grained cross-modal retrieval technology, related research mainly focuses on overcoming the modal heterogeneity between images and text. However, previous methods still suffer from two problems: in-modal semantic loss and insufficient negative sample coverage. In-modal semantic loss refers to the over-reliance on hard label supervision of image-text ground truth pairs, which causes the model to forcibly separate negative samples with partial semantic similarity, making it difficult to learn fine-grained hierarchical semantic structures. Insufficient negative sample coverage means that the combinations of negative samples accessible during training cannot fully cover the diversity space of pedestrian appearance and language descriptions, resulting in insufficient learning.

[0003] To address the issue of semantic gaps within modalities, a semantic correspondence correction module was designed. This module mines inter-sample relationships within modalities and utilizes these relationships to improve the alignment training objective across modalities, resulting in a more comprehensive and accurate cross-modal training objective optimization learning process. Furthermore, to enhance the coverage of negative samples during model training, an adaptive negative sample enhancement module was designed. This module uses inter-sample relationships between modalities to synthesize more difficult negative samples, increasing their diversity. Based on this, weights are adaptively assigned to each negative sample according to its difficulty, enhancing the model's focus on difficult samples during learning and thus improving the model's learning performance. Finally, the proposed relationship-aware adaptive learning optimization-based cross-modal person re-identification method corrects semantic correspondences during training and synthesizes high-quality difficult negative samples for focused learning, comprehensively improving the learning effect and robustness of the cross-modal person re-identification model, resulting in a significant improvement in the model's retrieval performance after training.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of the present invention, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] To address the issues of missing intra-modal semantics and insufficient negative sample coverage that are overlooked in existing natural language cross-modal person re-identification technologies, this invention provides a natural language cross-modal person re-identification method based on relation-aware adaptive learning optimization.

[0006] Other features and advantages of the invention will become apparent from the following detailed description, or may be learned in part by practice of the invention.

[0007] According to a first aspect of the present invention, a natural language cross-modal person re-identification method based on relation-aware adaptive learning optimization is provided, the method comprising: The training set images are preprocessed, and the global features of the images are extracted using the visual encoder of a pre-trained vision-language large model. ; The text description is word-vector encoded, and the text encoder of the aforementioned vision-language big data model is used to extract global text features. ; global features of the image and text global features Cosine similarity is calculated by mapping to a unified encoding space to obtain the cross-modal similarity between image and text. Text-to-image cross-modal similarity Align training objectives with ground truth. Construct cross-modal contrastive loss KL divergence loss Simultaneously, pedestrian classification loss is constructed by combining pedestrian identity tags. ; Build a cache to store image features from recent training rounds. and text features Calculate intra-modal similarity of images from batch to cache level. Intramodal similarity of text Image-to-text cross-modal similarity Text-to-image cross-modal similarity ;use , The ground truth alignment training objectives for both image and text directions are improved separately, resulting in semantic correspondence between images and text that corrects the training objectives. Correcting the training objective based on the semantic correspondence between text and image ; based on , , and Construct semantic correspondences to correct the loss. ; according to Calculate the synthesis weight of the negative samples corresponding to the text, and synthesize the hard negative samples of the text modality at the feature level based on the weight. ;according to Calculate the negative sample synthesis weights corresponding to the image, and synthesize hard negative samples of the image modality at the feature level based on these weights. ; Calculate the cross-modal similarity between the original image features and the synthesized text features, and then compare it with... The concatenation yields a text difficulty metric matrix, and the difficulty weight of each text feature is calculated using the standard deviation as the difficulty metric. ; Calculate the cross-modal similarity between the original text features and the synthesized image features, and then compare it with... The concatenation yields an image difficulty metric matrix, and the difficulty weight of each image feature is calculated using the standard deviation as the difficulty metric. ; Will , The weights are applied to the corresponding hardness metric matrices to construct adaptive negative sample augmentation losses for the image-to-text and text-to-image directions, respectively. The average of these values ​​is then used to obtain the adaptive negative sample augmentation loss. ; Joint optimization , , , and Fine-tuning of the pre-trained model; During the testing phase, the fine-tuned model was used to extract image and text features and calculate the text-to-image similarity. The search results are obtained by sorting them according to similarity.

[0008] In some exemplary embodiments, the preprocessing of the training set images specifically includes: The training set images are normalized to a uniform size, and data augmentation is performed through image flipping and random cropping.

[0009] In some exemplary embodiments, the construction of cross-modal contrast loss KL divergence loss Specifically:

[0010]

[0011] in, This indicates a comparison loss operation. express operate, This indicates that the training objective is aligned with the truth value.

[0012] In some exemplary embodiments, the method of constructing pedestrian classification loss by combining pedestrian identity tags... Specifically:

[0013]

[0014]

[0015] in, and These are pedestrian identification tags corresponding to the images and text, respectively. Indicates the first The pedestrian identity category number to which each image belongs. Indicates the first Each text description corresponds to a pedestrian identity category number. To calculate the total number of all pedestrian identity categories in the training set, Represents cross-entropy loss, A classifier that shares parameters across different modalities. This indicates the size of the training batch. In some exemplary embodiments, the semantic correspondence between the image and text corrects the training objective. Correcting the training objective based on the semantic correspondence between text and image Specifically:

[0016]

[0017] in, This indicates that the ground truth of the image is aligned with the training objective. This indicates that the truth values ​​of the text are aligned with the training objective. This is the fusion factor.

[0018] In some exemplary embodiments, the semantic correspondence correction loss Specifically: .

[0019] In some exemplary embodiments, the adaptive negative sample augmentation loss Specifically:

[0020]

[0021]

[0022] in, and These are adaptive negative sample enhancement losses for the image-to-text direction and the text-to-image direction, respectively.

[0023] According to a second aspect of the present invention, a storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the natural language cross-modal person re-identification method based on relation-aware adaptive learning optimization described in the first aspect.

[0024] According to a third aspect of the present invention, a computer program product is provided, on which a computer program is stored, wherein when the computer program is executed by a processor, it implements the natural language cross-modal person re-identification method based on relation-aware adaptive learning optimization described in the first aspect above.

[0025] According to a fourth aspect of the present invention, an electronic device is provided, comprising: Processor; and Memory for storing the executable instructions of the processor; The processor is configured to implement the natural language cross-modal person re-identification method based on relation-aware adaptive learning optimization described in the first aspect above by executing the executable instructions.

[0026] This invention proposes a cross-modal person re-identification method based on relation-aware adaptive learning optimization. Addressing the core pain points of existing natural language cross-modal person re-identification technologies, such as intra-modal semantic gaps and insufficient negative sample coverage, this method optimizes the model learning process by comprehensively exploring multi-level inter-sample relationships within and between modalities. It achieves technological breakthroughs in multiple dimensions, including training target correction, negative sample enhancement, and model learning efficiency and robustness. Compared to existing methods, it significantly improves retrieval accuracy and model generalization ability. Specific beneficial effects are as follows: 1. Addressing the issue of missing semantics within modalities and achieving precise optimization of cross-modal training objectives. The semantic correspondence correction module designed in this invention overcomes the limitations of existing methods that overly rely on hard label supervision of image-text ground truth pairs. By mining semantic relationships between intra-modal samples at the batch to cache level, this relationship is fused with the original ground truth alignment training objective, constructing a corrected training objective that more closely reflects actual semantic associations. This approach avoids the model forcibly separating negative samples with partial semantic similarity, enabling the model to learn fine-grained hierarchical semantic structures, achieving comprehensive and accurate optimization of cross-modal alignment training objectives, and significantly improving the learning accuracy of semantic correspondences between image and text modalities.

[0027] 2. Overcoming the deficiency of insufficient negative sample coverage and enhancing the model's learning ability for difficult samples. The adaptive negative sample enhancement module proposed in this invention fully explores the relationships between samples between modalities through two core steps: online negative sample synthesis and adaptive weight allocation. On the one hand, it adaptively synthesizes difficult negative samples at the feature level, effectively enriching the diversity of negative samples and allowing the model training process to cover a wider range of pedestrian appearance and language description diversity spaces, solving the problem of insufficient learning depth caused by insufficient negative sample combination coverage in traditional methods. On the other hand, it dynamically allocates weights to different negative samples using standard deviation as a difficulty metric, allowing the model to automatically focus on more difficult negative samples during training, achieving "focused learning of difficult samples" and significantly improving the model's ability to distinguish similar samples.

[0028] 3. Fully leverage the general knowledge of multimodal large models to enhance the discriminative power of feature extraction. This invention uses a pre-trained visual-language large model containing external general knowledge as the backbone feature extraction network. By leveraging its cross-modal feature representation capabilities learned from massive amounts of data, the extracted image and text features have stronger semantic discriminative power and cross-modal compatibility. At the same time, the large model is fine-tuned through the Adam optimization algorithm, allowing the pre-trained general knowledge to be deeply adapted to the fine-grained retrieval task of pedestrian re-identification, effectively reducing the difficulty of cross-modal feature mapping and laying a high-quality feature foundation for subsequent similarity calculation and model training.

[0029] 4. Collaborative Optimization of Multiple Loss Functions to Enhance Model Learning Performance and Robustness: This invention integrates semantic correspondence correction loss, adaptive negative sample enhancement loss, and basic cross-modal contrastive loss, KL divergence loss, and pedestrian classification loss into a total loss function. This achieves multi-objective collaborative optimization of cross-modal semantic alignment, hard sample reinforcement learning, and pedestrian identity feature constraints. This loss function design allows the model to firmly anchor pedestrian identity features while learning cross-modal semantic associations, effectively improving the model's training stability and robustness, enabling it to maintain stable retrieval performance even in complex scenarios.

[0030] 5. Significantly Improved Retrieval Accuracy and Practical Engineering Application Value: The method of this invention optimizes the cross-modal pedestrian re-identification model from the entire process, including training objectives, negative samples, feature extraction, and loss functions, comprehensively solving the core defects of existing technologies and ultimately achieving a significant improvement in model retrieval accuracy. This method can be directly applied to pedestrian retrieval tasks in real-world scenarios such as security monitoring, intelligent video analysis, and smart parks. It can quickly and accurately retrieve target pedestrian images from image databases through text descriptions, possessing strong practical engineering application value and promising prospects for widespread adoption.

[0031] 6. The method is highly versatile and easy to extend and adapt. The core modules of this invention (semantic correspondence correction module and adaptive negative sample enhancement module) are designed based on cross-modal sample relationship mining, do not depend on a specific multimodal large model architecture, and can be flexibly adapted to mainstream vision-language pre-trained large models; at the same time, the module parameters (such as fusion factor) The k-value of topk can be fine-tuned according to different datasets and application scenarios, and has good versatility and scalability, and can be transferred to other cross-modal retrieval tasks.

[0032] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0033] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention. It is obvious that the drawings described below are merely some embodiments of the invention, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0034] Figure 1 This is a schematic diagram of the framework of the cross-modal person re-identification method based on relation-aware adaptive learning optimization in an exemplary embodiment of the present invention; Figure 2 This is a flowchart of a natural language cross-modal person re-identification method based on relation-aware adaptive learning optimization in an exemplary embodiment of the present invention. Detailed Implementation

[0035] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that the invention will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0036] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0037] To address the shortcomings and deficiencies of existing technologies, this example implementation provides a natural language cross-modal person re-identification method based on relation-aware adaptive learning optimization. For natural language cross-modal person re-identification technology, a multimodal large model is used as the backbone feature extraction network. Based on its contained knowledge, the model comprehensively mines the inter-sample relationships within and between modalities to optimize model learning. First, a semantic correspondence correction module is designed to mine the inter-sample relationships within a modality and use these relationships to improve the cross-modal alignment training objective, achieving accurate and comprehensive cross-modal training objective optimization learning. Then, an adaptive negative sample enhancement module is designed, consisting of two steps: online negative sample synthesis and adaptive weight allocation. The online negative sample synthesis step utilizes the inter-sample relationships between modalities to synthesize more difficult negative samples, enhancing negative sample diversity. Then, the adaptive weight allocation step adaptively assigns weights to each negative sample based on its difficulty level, enhancing the model's attention to difficult negative samples during learning, thereby improving the model's learning performance. The proposed method comprehensively explores the inter-sample relationships within and between modalities, and based on this, modifies the objective function and constructs more difficult negative samples, thereby comprehensively improving the learning effect and robustness of the cross-modal person re-identification technology model, and ultimately achieving a significant improvement in search accuracy.

[0038] refer to Figure 2 As shown, the specific steps may include: Step S0: Obtain global image features using a pre-trained model; Step S2: Obtain global text features using a pre-trained model; Step S3: Obtain the basic loss function using contrastive loss, KL divergence loss, and pedestrian classification loss; Step S4: Correct the training objective through semantic correspondence and obtain the semantic correspondence correction loss function; Step S5: Obtain difficult negative samples of the text modality by mining the relationship between samples between image and text modalities; Step S6: Obtain difficult negative samples of the image modality by mining the relationship between samples from text to image modalities; Step S7: Obtain the hard weight corresponding to each text by calculating the cross-modal similarity from image to text; Step S8: Obtain the hard weight corresponding to each image by calculating the cross-modal similarity from text to image; Step S9: Obtain the adaptive negative sample augmentation loss function by using the hard weights corresponding to each sample; Step S10: Train the model using the Adam algorithm based on the complete loss function until convergence, and obtain the fine-tuned model; Step S11: Calculate the cross-modal similarity between text and image using the fine-tuned model to obtain the retrieval sequence.

[0039] The steps in this exemplary embodiment will now be described in more detail with reference to the accompanying drawings and embodiments.

[0040] Step S0: For the image side, firstly, the training set images are normalized to a uniform size, and data augmentation is performed using strategies such as image flipping and random cropping. Then, image features are extracted using a visual encoder of a pre-trained visual-language large model that incorporates external general knowledge. .Will As global features of an image , As a local feature of an image ,in This represents the number of local regions in an image.

[0041] Step S1: For the text side, words are encoded into word vectors using word embedding methods, and then text features are obtained through the text encoder of a vision-language pre-trained large model. .Will As a global feature of text , As a local feature of text ,in The number of tokens in each text.

[0042] Step S2, extract global features of the image and text global features Cosine similarity is calculated by mapping to a unified encoding space. This indicates the size of the training batch and yields the cross-modal similarity from image to text. Text-to-image cross-modal similarity Then, cross-modal contrastive loss was used. and KL divergence loss Align the training objective with the ground truth. Perform initial alignment:

[0043]

[0044]

[0045] in, This indicates a comparison loss operation. express Operation. Simultaneously, to leverage pedestrian identity constraints, global features of the image and text are mapped from the feature space to the identity space using a pedestrian classification loss. study:

[0046]

[0047]

[0048] in, and These are pedestrian identification tags corresponding to the images and text, respectively. Indicates the first The pedestrian identity category number to which each image belongs. Indicates the first Each text description corresponds to a pedestrian identity category number. This represents the total number of all pedestrian identity categories in the training set. Represents cross-entropy loss, A classifier that shares parameters across different modalities. Step S3: To construct a more comprehensive and accurate cross-modal training objective, a semantic correspondence correction module is proposed. This module mines the relationships between samples within a modality and uses these relationships to improve the ground truth alignment training objective across modalities, achieving more comprehensive and accurate cross-modal training objective optimization learning. First, a module of size [size missing] is constructed. The cache is used to store image features from the most recent training epochs. and text features And calculate the intra-modal similarity of images from the current batch to the cache level. Intramodal similarity of text Image-to-text cross-modal similarity Text-to-image cross-modal similarity ; and utilize and To improve the ground value alignment training objective across modalities Thus, the semantic correspondence is obtained to correct the training objective. Specifically, in the image-to-text direction, the corresponding semantic correspondence is obtained to correct the training objective. :

[0049] in, This indicates that the ground truth of the image is aligned with the training objective. This is a fusion factor used to adjust the intra-modal similarity of images from batch to cache level. The contribution weights to the overall training objective control the intensity of the correction to cross-modal semantic correspondences. Similarly, in the text-to-image direction, the corresponding semantic correspondences are obtained to correct the training objective. :

[0050] in, This indicates that the truth values ​​of the text are aligned with the training objective. Consistent with the above formula.

[0051] Step S4: Use the semantic correspondences in the two directions obtained in step S3 to correct the training objective and construct the semantic correspondence correction loss. To obtain more comprehensive and accurate cross-modal training objectives:

[0052] Step S5 proposes an adaptive negative sample enhancement module. This module utilizes the inter-modal sample relationships to adaptively construct difficult negative samples during training and dynamically focus on even more difficult negative samples, thereby improving model learning performance. This module consists of two steps: online negative sample synthesis and adaptive weight allocation. The online negative sample synthesis step adaptively selects several of the most difficult samples based on the inter-modal sample relationships during training and synthesizes them into a new difficult negative sample at the feature level. Specifically, to obtain synthetic difficult negative samples for the text modality... First, it is necessary to calculate the composite weight of the negative samples corresponding to the text. :

[0053] in, For standardized operation, This means retaining the largest one. One value, The image-to-text cross-modal similarity is calculated; then, based on the negative sample weights corresponding to the text... Obtaining synthetically difficult negative samples of text modalities :

[0054] Step S6, referring to step S5, involves obtaining difficult negative samples for image modality synthesis, similar to the method for obtaining difficult negative samples for text modality synthesis. First, the synthesis weights of the negative samples corresponding to the image are calculated. Subsequently, according to Obtaining image modalities is difficult due to the synthesis of negative samples. The calculation process is as follows:

[0055]

[0056] in, This represents the cross-modal similarity between text and image.

[0057] Step S7: To fully utilize the synthesized negative samples, an adaptive weight allocation step is designed. This step constructs an adaptive negative sample enhancement loss to help the model dynamically focus on more difficult negative samples during training. For the image-to-text direction, the cross-modal similarity between the original image features and the synthesized text features is first calculated. Compare it with image cross-modal similarity Concatenate to form a text difficulty metric matrix :

[0058] in, This indicates a feature concatenation operation. Each row represents the similarity between an original image feature, an original text feature, and a synthesized text feature; the standard deviation is used as a difficulty measure, and the difficulty weight corresponding to each text feature is obtained by taking the standard deviation column by column. :

[0059] in, This indicates the standard deviation operation.

[0060] Step S8, referring to step S7, first calculate the cross-modal similarity between the original text features and the synthesized image features. Compare it with text cross-modal similarity The concatenation yields the image difficulty metric matrix in the text-to-image direction. Then, the standard deviation is taken column-wise to obtain the hard weights corresponding to each image feature. The calculation process is as follows:

[0061] . Step S9: By weighting and fusing the hardness weights into the hardness metric matrix, the adaptive negative sample augmentation loss for the image-to-text direction and the text-to-image direction can be obtained respectively. and The complete adaptive negative sample augmentation loss is obtained by averaging the two directions. The calculation process is as follows:

[0062]

[0063]

[0064] Step S10, respectively , , , and The loss function used for training is optimized. The "Adam optimization algorithm" is used to fine-tune the pre-trained multimodal large model until the model converges.

[0065] Step S11, the testing phase, features are extracted from the image and text using a fine-tuned pre-trained multimodal large model, and the cross-modal similarity between text and image is obtained through step S2. Ultimately, based on the similarity between different samples Sort the results from largest to smallest to obtain a sequence of search results.

[0066] The proposed cross-modal person re-identification method based on relation-aware adaptive learning optimization introduces a semantic correspondence correction module and an adaptive negative sample enhancement module. A multimodal pre-trained large model is employed, utilizing its inherent external general knowledge to extract more discriminative image and text features. The semantic correspondence correction module mines inter-sample relationships within a modality and uses these relationships to improve the cross-modal alignment training objective, achieving more accurate and comprehensive cross-modal training objective optimization learning. Furthermore, to improve the coverage of negative samples during model training, while considering the diversity and difficulty of negative samples, an adaptive negative sample enhancement module is designed. This module utilizes inter-sample relationships between modalities to synthesize more difficult negative samples, enhancing the diversity of negative samples. Based on this, dynamic weights are adaptively assigned to each negative sample according to its difficulty, increasing the model's attention to difficult samples during learning, thereby improving the model's learning performance. To address the issues of missing intramodal semantics and insufficient negative sample coverage encountered in existing technologies for cross-modal person re-identification in natural language, the proposed improved method can correct the semantic correspondence during the training process and enhance the coverage of negative samples during training, thereby improving the accuracy of cross-modal person re-identification. Compared with existing methods, the accuracy is significantly improved.

[0067] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0068] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the claims.

[0069] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is defined only by the appended claims.

Claims

1. A cross-modal person re-identification method based on relation-aware adaptive learning optimization, characterized in that, The method includes: The training set images are preprocessed, and the global features of the images are extracted using the visual encoder of a pre-trained vision-language large model. ; The text description is word-vector encoded, and the text encoder of the aforementioned vision-language big data model is used to extract global text features. ; global features of the image and text global features Cosine similarity is calculated by mapping to a unified encoding space to obtain the cross-modal similarity between image and text. Text-to-image cross-modal similarity Align training objectives with ground truth. Construct cross-modal contrastive loss KL divergence loss Simultaneously, pedestrian classification loss is constructed by combining pedestrian identity tags. ; Build a cache to store image features from recent training rounds. and text features Calculate intra-modal similarity of images from batch to cache level. Intramodal similarity of text Image-to-text cross-modal similarity Text-to-image cross-modal similarity ;use , The ground truth alignment training objectives for both image and text directions are improved separately, resulting in semantic correspondence between images and text that corrects the training objectives. Correcting the training objective based on the semantic correspondence between text and image ; based on , , and Construct semantic correspondences to correct the loss. ; according to Calculate the synthesis weight of the negative samples corresponding to the text, and synthesize the hard negative samples of the text modality at the feature level based on the weight. ;according to Calculate the negative sample synthesis weights corresponding to the image, and synthesize hard negative samples of the image modality at the feature level based on these weights. ; Calculate the cross-modal similarity between the original image features and the synthesized text features, and then compare it with... The concatenation yields a text difficulty metric matrix, and the difficulty weight of each text feature is calculated using the standard deviation as the difficulty metric. ; Calculate the cross-modal similarity between the original text features and the synthesized image features, and then compare it with... The concatenation yields an image difficulty metric matrix, and the difficulty weight of each image feature is calculated using the standard deviation as the difficulty metric. ; Will , The weights are applied to the corresponding hardness metric matrices to construct adaptive negative sample augmentation losses for the image-to-text and text-to-image directions, respectively. The average of these values ​​is then used to obtain the adaptive negative sample augmentation loss. ; Joint optimization , , , and Fine-tuning of the pre-trained model; During the testing phase, the fine-tuned model was used to extract image and text features and calculate the text-to-image similarity. The search results are obtained by sorting them according to similarity.

2. The cross-modal person re-identification method based on relation-aware adaptive learning optimization according to claim 1, characterized in that, The preprocessing of the training set images specifically includes: The training set images are normalized to a uniform size, and data augmentation is performed through image flipping and random cropping.

3. The cross-modal person re-identification method based on relation-aware adaptive learning optimization according to claim 1, characterized in that, The construction of cross-modal contrast loss KL divergence loss Specifically: in, This indicates a comparison loss operation. express operate, This indicates that the training objective is aligned with the truth value.

4. The cross-modal person re-identification method based on relation-aware adaptive learning optimization according to claim 1, characterized in that, The method of constructing pedestrian classification loss by combining pedestrian identity tags Specifically: in, and These are pedestrian identification tags corresponding to the images and text, respectively. Indicates the first The pedestrian identity category number to which each image belongs. Indicates the first Each text description corresponds to a pedestrian identity category number. To calculate the total number of all pedestrian identity categories in the training set, Represents cross-entropy loss, A classifier that shares parameters across different modalities. This indicates the size of the training batch.

5. The cross-modal person re-identification method based on relation-aware adaptive learning optimization according to claim 1, characterized in that, The semantic correspondence between the image and the text is used to correct the training objective. Correcting the training objective based on the semantic correspondence between text and image Specifically: in, This indicates that the ground truth of the image is aligned with the training objective. This indicates that the truth values ​​of the text are aligned with the training objective. This is the fusion factor.

6. The cross-modal person re-identification method based on relation-aware adaptive learning optimization according to claim 5, characterized in that, The semantic correspondence correction loss Specifically: 。 7. The cross-modal person re-identification method based on relation-aware adaptive learning optimization according to claim 1, characterized in that, The adaptive negative sample enhancement loss Specifically: in, and These are adaptive negative sample enhancement losses for the image-to-text direction and the text-to-image direction, respectively.

8. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the cross-modal pedestrian re-identification method based on relation-aware adaptive learning optimization as described in any one of claims 1 to 7.

9. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the cross-modal pedestrian re-identification method based on relation-aware adaptive learning optimization as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the cross-modal pedestrian re-identification method based on relation-aware adaptive learning optimization as described in any one of claims 1 to 7 by executing the executable instructions.

Images