Comparison normalization probability representation method for multi-modal image-text retrieval
By introducing Vision Transformer and BERT as encoders, combining probability distribution and comparison normalization modules, the cross-modal search model is optimized, and the problems of image and text modal features alignment and ambiguity matching are solved, achieving efficient and accurate multimodal search.
Patent Information
- Application Number
- CN202510529258.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-24
- Publication Date
- 2025-08-08
AI Technical Summary
In the existing cross-modal graphic and text retrieval technology, it is difficult to align the modal features of images and text, and it is difficult to effectively model the ambiguity matching relationship, resulting in insufficient retrieval accuracy and robustness.
Vision Transformer and BERT are used as visual and text encoders, and the embedding space is optimized through probability distribution modeling and contrast normalization modules, combined with KL divergence or Euclidean distance to calculate similarity, and the model is trained using soft contrast loss function and regularization strategy.
It significantly improves the accuracy and robustness of cross-modal retrieval, can effectively handle multi-object and ambiguity descriptions in complex scenarios, and improves feature alignment effects and model generalization capabilities.
Smart Images
Figure CN120448728A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence and computer vision technology, and in particular to a comparative normalized probability representation method for multimodal image and text retrieval. Background Art
[0002] In recent years, advances in artificial intelligence technology have significantly promoted the development of multimodal data processing. Among them, cross-modal image and text retrieval technology, as a key research direction, has continued to highlight its application value in multiple fields. This technology aims to match related images through text queries, or retrieve semantically consistent text descriptions based on image content. It is widely used in scenarios such as intelligent search engines, e-commerce product intelligent recommendations, and multimedia content management. Existing mainstream methods mostly rely on cross-modal embedding frameworks, which construct a shared vector space for images and texts and map them into vectorized representations whose similarity can be calculated. However, due to the significant differences in modal characteristics between images and texts, how to achieve accurate feature alignment in a unified embedding space remains a core challenge for technical optimization.
[0003] Currently, deterministic embedding models (such as visual-semantic embedding (VSE)) are widely used in cross-modal retrieval. They use neural networks to extract image and text features separately and map them into Euclidean space for similarity matching. These methods assume a strict one-to-one correspondence between images and text. However, in real-world scenarios, complex images may correspond to ambiguous text descriptions (e.g., an image containing multiple objects may correspond to text with different emphases), and vice versa. This many-to-many semantic association makes it difficult for deterministic models to effectively model these relationships, thus limiting retrieval accuracy. To address this ambiguity, researchers have proposed various improvements. For example, the PVSE method, based on a multi-head attention mechanism, generates multiple sets of candidate embedding representations to enhance its ability to capture ambiguity. However, this requires the generation of multiple sets of additional features, significantly increasing computational complexity. Another approach, such as regional feature embedding based on Faster R-CNN, extracts local semantic information from images and establishes fine-grained associations with text words. While this improves performance, its high computational resource requirements limit its practical deployment. The introduction of probabilistic embedding technology offers a new approach to addressing this issue. Methods such as Probabilistic Cross-Modal Embedding (PCME) represent images and text as probability distributions rather than single vectors. By modeling the fuzzy matching relationship between modalities through the uncertainty of the distribution, they naturally support many-to-many semantic associations. However, traditional PCME methods still have limitations: their visual encoders often use convolutional networks such as ResNet, and their text encoders rely on a combination of static word vectors (such as GLOVE) and bidirectional GRU, resulting in insufficient global semantic extraction capabilities. Furthermore, the optimization space for feature normalization and cross-modal alignment strategies is large, which restricts the robustness of the model.
[0004] Therefore, a contrastive normalized probability representation method for multimodal image-text retrieval is expected. By integrating a high-performance visual encoder and a deep context-aware text encoder, and combining it with an innovative contrastive normalization module, the feature expression ability and modality alignment effect can be significantly improved, thereby optimizing the accuracy and efficiency of cross-modal image-text retrieval. Summary of the Invention
[0005] In order to solve the above technical problems, the present application is proposed. The embodiments of the present application provide a comparative normalized probability representation method for multimodal image-text retrieval, which constructs a joint embedding space of images and texts and uses probability distribution to represent the semantic matching relationship between multimodal data, thereby achieving efficient and accurate cross-modal retrieval, which is widely applicable to image search, semantic understanding, human-computer interaction and other fields.
[0006] According to one aspect of the present application, a comparative normalized probability representation method for multimodal image and text retrieval is provided, which includes: S101, extracting initial features of images and texts respectively through a visual encoder and a text encoder; S102, performing probability distribution modeling on the initial features to generate normal distribution representations of images and texts; S103, optimizing the embedding space of the normal distribution through a comparative normalization module to enhance feature alignment between modalities; S104, realizing cross-modal retrieval of images and texts based on similarity calculation based on probability distribution.
[0007] Preferably, in step S102, the probability distribution modeling generates a mean vector μ and a variance vector σ through two independent fully connected layers to construct a normal distribution N(μ,σ 2 )
[0008] Preferably, in step S103, the contrast normalization module optimizes the embedding space by the following operations, including: normalizing the image and text features to a uniform scale; and constraining the consistency of feature distribution through contrast learning to avoid dimensionality collapse.
[0009] Preferably, in step S104, the similarity calculation uses KL divergence or Euclidean distance to measure the matching degree between the probability distributions of the image and the text, and supports modeling of one-to-many or many-to-many matching relationships.
[0010] Preferably, a regularization step is also included: constraining the variance σ by KL divergence to prevent distribution collapse; and a Dropout operation of randomly shutting down some neurons with a probability range of 0.25 to 0.75.
[0011] Preferably, a soft contrast loss function is used in the model training stage, and the loss function is as follows:
[0012]
[0013] Among them, θ represents the parameter set of the model, x a and xβ represent input sample pairs, image and text data, p θ (m|x α ,x β ) represents the matching probability of the sample pair.
[0014] Preferably, adaptive weight allocation is adopted in the training stage, the weights of image and text features are dynamically adjusted through a learnable parameter β, and β is normalized using a sigmoid function.
[0015] Preferably, the data preprocessing of the method includes: uniformly scaling the image to a resolution of 224×224; tokenizing the text using Bert's Tokenizer, and performing an enhancement operation on the text by randomly deleting 10% of the words and randomly replacing 10% of the words.
[0016] According to one aspect of the present application, an electronic device is also provided, characterized in that it includes a memory and a processor, the memory stores an executable program, and when the processor runs the executable program, it implements the above-mentioned comparative normalized probability representation method for multimodal image and text retrieval.
[0017] According to one aspect of the present application, a computer-readable storage medium is also provided, characterized in that the computer-readable storage medium stores computer execution instructions, and when the computer execution instructions are executed by a processor, they are used to implement the above-mentioned comparative normalized probability representation method for multimodal image and text retrieval.
[0018] This paper designs a contrastive normalized probability representation method for multimodal image-text retrieval. By integrating Vision Transformer and BERT as visual and text encoders, high-order semantic features of images and texts are extracted respectively. Subsequently, probability distribution modeling is performed on the features to naturally represent the many-to-many matching relationship between modalities. A contrastive normalization module is introduced to optimize the embedding space through feature normalization and contrastive learning to solve the problem of feature misalignment caused by modal differences. KL divergence or Euclidean distance is used to calculate the similarity of probability distributions, and the model is trained in combination with a soft contrast loss function and a regularization strategy to ultimately achieve efficient and robust cross-modal retrieval, significantly improving retrieval accuracy and generalization capabilities in complex scenarios.
[0019] The beneficial effects of the present invention include:
[0020] By adopting the Vision Transformer (ViT) as the image encoder, the expressive power of visual features is significantly improved. Compared to traditional convolutional neural networks (such as ResNet), ViT can effectively capture the global context of the image through a global self-attention mechanism and model the multi-object and multi-relationship semantics in complex scenes. This improvement is particularly suitable for handling cross-modal matching tasks with high semantic complexity or ambiguous descriptions, thereby improving the semantic accuracy and completeness of image features.
[0021] In terms of text modality, this paper uses BERT (Bidirectional Encoder Representations from Transformers) as a text encoder, which can capture context-dependent dynamic semantic information, surpassing the limitations of static word vectors and simple sequence models (such as GLOVE and bidirectional GRU) in traditional methods. Through its pre-trained model, BERT can handle complex syntactic structures and ambiguous text descriptions, making information extraction in text modality more comprehensive and accurate, significantly improving the performance of text features in cross-modal retrieval tasks.
[0022] By constructing a shared cross-modal embedding space, we represent image and text modalities using a probabilistic distribution. Each modal data is normally distributed using a mean and variance vector. This not only naturally captures one-to-many and many-to-many matching relationships between modalities, but also models the uncertainty of the data. This probabilistic representation effectively addresses the inability of traditional deterministic embedding methods to handle ambiguous matching, making cross-modal retrieval results more robust and interpretable.
[0023] The ContraNorm feature normalization module was introduced to significantly improve modality feature alignment by optimizing the feature distribution in the embedding space. ContraNorm not only improves the representation consistency of image and text modalities in the shared space, but also addresses the uneven distribution of features in the embedding space in existing technologies through normalization and distribution optimization, thereby improving the model's retrieval performance and generalization capabilities.
[0024] The model training employs an optimization strategy combining contrastive learning and regularization, effectively balancing the model's retrieval accuracy and computational efficiency, making it suitable for retrieval of large-scale multimodal data. Overall, the improved model provided by this invention significantly outperforms traditional methods in terms of retrieval accuracy, robustness, and computational efficiency, and has broad practical application value. BRIEF DESCRIPTION OF THE DRAWINGS
[0025] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.
[0026] Figure 1 Flowchart of a comparative normalized probability representation method for multimodal image-text retrieval according to an embodiment of the present application.
[0027] Figure 2 The figure is an overall flow chart of a comparative normalized probability representation method for multimodal image-text retrieval according to an embodiment of the present application.
[0028] Figure 3 Flowchart of the CN-RPM model in the comparative normalized probability representation method for multimodal image-text retrieval according to an embodiment of the present application. DETAILED DESCRIPTION
[0029] Various exemplary embodiments, features, and aspects of the present application will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise indicated.
[0030] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.
[0031] In addition, numerous specific details are provided in the following detailed description to better illustrate the present application. Those skilled in the art will appreciate that the present application can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art are not described in detail in order to highlight the main purpose of the present application.
[0032] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. Throughout the description of this application, "plurality" means two or more, unless otherwise specifically defined.
[0033] Existing multimodal image-text retrieval technologies mainly face the following technical bottlenecks: First, the currently commonly used image encoders (such as ResNet) and text encoders (such as GLOVE and bidirectional GRU) have limitations in feature extraction capabilities, making it difficult to fully capture the global semantic information of complex images and the deep semantic relationships of texts, which in turn affects the accuracy of retrieval; second, due to the natural differences in the way images and texts express information, existing methods find it difficult to effectively bridge the modal differences when aligning their features in the cross-modal embedding space, resulting in insufficient matching performance; third, images and texts are often ambiguous in practical applications (i.e., one-to-many or many-to-many relationships), and traditional deterministic embedding methods cannot fully model the ambiguity and uncertainty between these modalities, which in turn affects the quality and reliability of retrieval results; in addition, existing technologies lack effective mechanisms for feature standardization in the embedding space, resulting in uneven feature distribution, further reducing the robustness and generalization ability of the model. To address the above problems, the present invention proposes an improved cross-modal retrieval method based on probabilistic embedding, which combines advanced visual encoders (such as Vision Transformer), text encoders (such as BERT), and normalization modules that optimize the distribution of feature vectors (such as ContraNorm). It aims to enhance feature extraction capabilities, modality alignment performance, and polysemy modeling capabilities, thereby significantly improving the accuracy and robustness of multimodal image-text retrieval.
[0034] This paper provides an improved model for multimodal image-text retrieval, which shows significant performance improvement in cross-modal retrieval tasks by introducing advanced visual and text encoders (Vision Transformer and BERT) and an enhanced feature normalization mechanism (ContraNorm).
[0035] The present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods:
[0036] Figure 1 FIG is a flow chart of a comparative normalized probability representation method for multimodal image-text retrieval according to an embodiment of the present application. Figure 1 As shown, the comparative normalized probability representation method for multimodal image and text retrieval according to an embodiment of the present application includes the following steps: S101, extracting the initial features of the image and text respectively through a visual encoder and a text encoder; S102, performing probability distribution modeling on the initial features to generate a normal distribution representation of the image and text; S103, optimizing the embedding space of the normal distribution through a comparative normalization module to enhance feature alignment between modalities; S104, realizing cross-modal retrieval of images and texts based on similarity calculation based on probability distribution.
[0037] First, we obtained three large, open-source datasets. After processing, each dataset was divided into training, test, and validation sets. These sets were used to construct a comparative normalized probabilistic representation model (CN-RPM) for multimodal image-text retrieval. The CN-RPM model was trained using the training datasets and then evaluated. Finally, the trained CN-RPM model was used to predict images on the web.
[0038] The specific technical solution of the present invention comprises the following steps:
[0039] The first step is to obtain three large open-source datasets, process and package the data, and obtain the representation information, label information, and indexes of the training set, test set, and validation set in the datasets. The method is: download the coco, flickr30k, and CUB datasets from https: / / cocodataset.org / , https: / / shannon.cs.illinois.edu / DenotationGraph / , and https: / / paperswithcode.com / dataset / cub-200-2011, respectively;
[0040] Clean the dataset and standardize the import formats of different datasets. For the processed dataset, load the file to obtain images and corresponding labels, with one image corresponding to five descriptions. Load the training, validation, and test set indices from the corresponding dataset split file, splitting them into 48% / 32% / 20%. Process the images and resize and crop them to a uniform size of 224*224. Tokenize the descriptions using Bert's Tokenizer, randomly deleting 10% of the words in each sentence and replacing 10% of the words in each sentence.
[0041] The second step is to construct a comparative normalized probability representation model (CN-RPM) for multimodal image and text retrieval. The method is:
[0042] In the embodiment of the present application, the step S101 extracts the initial features of the image and text respectively through the visual encoder and the text encoder. It should be understood that the deep learning environment should be built first. Install anaconda on the server and create a virtual environment. Build Pytorch-GPU in the virtual environment, and then install the pytotch library and the required library packages such as scipy, numpy, munch, etc. Input multimodal data, and for a given multimodal data set (such as image and text pairs), extract image features and text features through a visual encoder (such as ViT) and a text encoder (such as BERT). Image feature F i 0: Input image I to the visual encoder to get the initial embedding representation. Text features : Input text description T to the text encoder to extract context-aware semantic representation.
[0043] In the embodiment of the present application, the step S102 performs probability distribution modeling on the initial features to generate normal distribution representations of images and texts. The probability distribution modeling generates a mean vector μ and a variance vector σ through two independent fully connected layers to construct a normal distribution N(μ,σ 2 ). The probability features of image and text are and This probabilistic representation can effectively capture the uncertainty and ambiguity in multimodal data.
[0044] In an embodiment of the present application, step S103 optimizes the embedding space of the normal distribution through a contrast normalization module to enhance inter-modal feature alignment. The contrast normalization module optimizes the embedding space through the following operations, including: normalizing the image and text features and adjusting them to a uniform scale; and constraining the consistency of feature distribution through contrast learning to avoid dimensionality collapse. It should be understood that through the normalization operation, the embedding features of the image and text are adjusted to a uniform scale to reduce the feature differences between the modalities. Through contrast learning, the distribution consistency of the image and text features is enhanced, while avoiding the dimensionality collapse of the embedding space.
[0045] In an embodiment of the present application, step S104 implements cross-modal retrieval of images and texts based on similarity calculation of probability distribution. The similarity calculation uses KL divergence or Euclidean distance to measure the degree of matching between the probability distributions of images and texts, and supports modeling of one-to-many or many-to-many matching relationships. It should be understood that for each pair of images and texts, the similarity score between the distributions is calculated using KL divergence or Euclidean distance to measure the degree of matching between the images and texts. Through polysemy modeling, one-to-many or many-to-many matching relationships are allowed, thereby improving the robustness of the retrieval.
[0046] Furthermore, in one example of the present application, a regularization step is also included: constraining the variance σ through the KL divergence to prevent distribution collapse; and a Dropout operation that randomly shuts down some neurons with a probability range of 0.25 to 0.75, randomly shutting down some neurons in the embedding space to reduce overfitting and improve generalization ability.
[0047] During the model training phase, a soft contrast loss function is used to optimize the alignment performance of image and text features, while regularization is used to control the stability of the distribution.
[0048] The loss function is as follows:
[0049]
[0050] Among them, θ represents the parameter set of the model, x a and xβ represent input sample pairs, image and text data, p θ (m|x α ,x β ) represents the matching probability of the sample pair.
[0051] In the inference stage: Use the trained CN-RPM model to retrieve the most relevant image for the input text description, or generate the most matching text description for the input image.
[0052] During the training phase, adaptive weight allocation is employed, dynamically adjusting the weights of image and text features through a learnable parameter β, which is then normalized using a sigmoid function. Specifically, the learnable parameter β is introduced to assign a weight of β to image features and a weight of 1-β to text features. Using a sigmoid function for normalization enables adaptability across various multimodal alignment tasks.
[0053] Furthermore, based on the convergence of the loss function curve, the model corresponding to the epoch with the lowest loss on the validation set is selected as the optimal model. Model performance is evaluated using retrieval task metrics (such as Recall@K and R-Precision) to ensure the model's performance in actual retrieval tasks.
[0054] The constructed CN-RPM model is trained and fitted. The specific steps are as follows:
[0055] 1. Initialize the training parameters: epoch = 70, which ranges from 60 to 80 for different datasets; batch size batchsize = 128; learning rate lr = 0.0001, weight decay weight_decay = 0.01.
[0056] 2. Build the CN-RPM model: Use the PyTorch framework to implement the CN-RPM model, set the loss function to the contrastive learning loss function L, the optimizer to the Adamp algorithm, and the learning rate to lr.
[0057] 3. Training and verification: After each epoch, the current model is verified using the verification set. The image and text features of the verification set are calculated by the probability distribution N(μ,σ 2 ) indicates that if the retrieval performance (such as Recall@K) of the validation set is better than that of the training set, and the loss value of the validation set is less than the loss value of the training set, the parameter information of the current model is saved; the training and validation are repeated until the optimal model result is saved.
[0058] Perform multimodal retrieval prediction using the trained CN-RPM model, including: loading the optimal model parameters saved during the training phase, initializing the CN-RPM model, loading the image and text features of the test set, and generating probability distributions using the visual encoder and text encoder, respectively. and Use the test set to evaluate the retrieval performance of the model, count indicators such as Recall@K and R-Precision, and calculate the standard deviation of the model on the test set to measure the stability of the model.
[0059] In summary, the present invention significantly improves the expressiveness of visual features by adopting Vision Transformer (ViT) as an image encoder. Compared with traditional convolutional neural networks (such as ResNet), ViT can effectively capture the global contextual information of images through a global self-attention mechanism, and model multi-object and multi-relationship semantics in complex scenes. This improvement is particularly suitable for processing cross-modal matching tasks with high semantic complexity or ambiguous descriptions, thereby improving the semantic accuracy and completeness of image features. In terms of text modality, the present invention adopts BERT (Bidirectional Encoder Representations from Transformers) as a text encoder, which can capture context-dependent dynamic semantic information, surpassing the limitations of static word vectors and simple sequence models (such as GLOVE and bidirectional GRU) in traditional methods. BERT can cope with complex syntactic structures and ambiguous text descriptions through its pre-training model, making the information extraction of text modality more comprehensive and accurate, and significantly improving the performance of text features in cross-modal retrieval tasks. By constructing a shared cross-modal embedding space, image and text modalities are represented in a probability distribution manner. Each modal data generates a normal distribution through the mean vector and variance vector, which can not only naturally capture the one-to-many and many-to-many matching relationships between modalities, but also model the uncertainty of the data. This probabilistic representation method effectively solves the problem that traditional deterministic embedding methods cannot handle ambiguous matching, making the results of cross-modal retrieval more robust and interpretable. The ContraNorm feature normalization module is introduced, which significantly improves the effect of modal feature alignment by optimizing the feature distribution in the embedding space. ContraNorm not only improves the representation consistency of image and text modalities in the shared space, but also solves the problem of uneven distribution of embedding space features in existing technologies through normalization and distribution optimization, thereby improving the retrieval performance and generalization ability of the model.
[0060] This paper employs an optimization strategy combining contrastive learning and regularization in model training, effectively balancing the model's retrieval accuracy and computational efficiency, making it suitable for retrieval of large-scale multimodal data. Overall, the improved model provided by this paper significantly outperforms traditional methods in terms of retrieval accuracy, robustness, and computational efficiency, and has broad practical application value.
[0061] While various embodiments of the present disclosure have been described above, the above descriptions are illustrative, non-exhaustive, and not intended to be limiting of the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or improvements to existing technologies, or to enable others skilled in the art to understand the embodiments disclosed herein.
Claims
1. A comparative normalized probability representation method for multimodal image-text retrieval, characterized in that: include: S101, extracting initial features of the image and text respectively through the visual encoder and text encoder; S102, performing probability distribution modeling on the initial features to generate normal distribution representations of images and texts; S103, optimizing the embedding space of the normal distribution through a contrast normalization module to enhance inter-modality feature alignment; S104, similarity calculation based on probability distribution, to achieve cross-modal retrieval of images and texts.
2. The comparative normalized probability representation method for multimodal image-text retrieval according to claim 1, characterized in that: In step S102, the probability distribution modeling generates a mean vector μ and a variance vector σ through two independent fully connected layers to construct a normal distribution N(μ,σ 2 ).
3. The comparative normalized probability representation method for multimodal image-text retrieval according to claim 2, characterized in that: In step S103, the contrast normalization module optimizes the embedding space by the following operations, including: Normalize image and text features to a uniform scale; By contrastive learning, the consistency of feature distribution is constrained to avoid dimensionality collapse.
4. The comparative normalized probability representation method for multimodal image-text retrieval according to claim 3, characterized in that: In step S104, the similarity calculation uses KL divergence or Euclidean distance to measure the matching degree between the probability distribution of the image and the text, and supports modeling of one-to-many or many-to-many matching relationships.
5. The comparative normalized probability representation method for multimodal image-text retrieval according to claim 4, characterized in that: It also includes a regularization step: constraining the variance σ by KL divergence to prevent the distribution from collapsing; and a Dropout operation that randomly turns off some neurons with a probability ranging from 0.25 to 0.
75.
6. The comparative normalized probability representation method for multimodal image-text retrieval according to claim 5, characterized in that: It also includes using a soft contrast loss function during the model training phase, and the loss function is as follows: Among them, θ represents the parameter set of the model, x a and x β Represents input sample pairs, image and text data, p θ (m|x α ,x β ) represents the matching probability of the sample pair.
7. The comparative normalized probability representation method for multimodal image-text retrieval according to claim 6, characterized in that: Adaptive weight allocation is used in the training phase to dynamically adjust the weights of image and text features through a learnable parameter β, and β is normalized using the sigmoid function.
8. The comparative normalized probability representation method for multimodal image-text retrieval according to claim 7, characterized in that: The data preprocessing of the method includes: Scale the image to a uniform resolution of 224×224; The text is tokenized using Bert's Tokenizer, and the text is enhanced by randomly deleting 10% of the words and randomly replacing 10% of the words.
9. An electronic device, characterized in that: It includes a memory and a processor, the memory stores an executable program, and when the processor runs the executable program, it implements the comparative normalized probability representation method for multimodal image and text retrieval as described in any one of Claims 1 to 8.
10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the comparative normalized probability representation method for multimodal image-text retrieval according to any one of claims 1 to 8.