Method and apparatus for cross-modal retrieval of generative text and images

Abstract

This invention provides a generative text and image cross-modal retrieval method and apparatus, applied in the field of information retrieval technology. The method includes: inputting image samples from an image candidate set into the visual encoder of a multimodal large language model to obtain a discrete image token sequence; constructing an image token dictionary based on the discrete image token sequence; performing supervised fine-tuning training on the multimodal large language model based on labeled data from text samples and the discrete image token sequence to obtain a supervised fine-tuning model; optimizing the supervised fine-tuning model using reinforcement learning based on a population relative policy optimization algorithm to obtain a generative multimodal retrieval model; and inputting query text into the generative multimodal retrieval model for constrained-bind search to obtain the image retrieval result corresponding to the query text output by the generative multimodal retrieval model. This invention can optimize the accuracy of generative text and image cross-modal retrieval.

Description

Technical Field

[0001] This invention relates to the field of information retrieval technology, and in particular to a method and apparatus for cross-modal retrieval of generative text and images. Background Technology

[0002] Text-image cross-modal retrieval technology can efficiently locate semantically related image content by inputting text query commands, and is a key technology for fully mining the value of multimodal data. Currently, with the development of big data and artificial intelligence technologies, multimodal data is experiencing explosive growth, making text-image cross-modal retrieval technology a rapidly emerging research hotspot.

[0003] Most existing studies have attempted to construct shared semantic spaces to achieve cross-modal representation learning. However, the inherent heterogeneity of text and images leads to a significant semantic gap between the two modalities, making cross-modal representation learning difficult and limiting the improvement of retrieval accuracy.

[0004] It is evident that the cross-modal text image retrieval methods in related technologies suffer from low retrieval accuracy. Summary of the Invention

[0005] This invention provides a method and apparatus for cross-modal retrieval of generative text and images, which addresses the shortcomings of low retrieval accuracy in existing text-image cross-modal retrieval methods and optimizes the accuracy of cross-modal retrieval of generative text and images.

[0006] This invention provides a cross-modal retrieval method for generative text and images, comprising the following steps.

[0007] Image samples from the image candidate set are input into the visual encoder of a multimodal large language model for image transformation to obtain discrete image token sequences. An image token dictionary is constructed based on these discrete image token sequences. Annotated data for text samples and their corresponding discrete image token sequences are obtained. Supervised fine-tuning training is performed on the multimodal large language model based on the annotated data to learn the mapping relationship between the text samples and the discrete image token sequences, resulting in a supervised fine-tuning model. Reinforcement learning optimization is applied to the supervised fine-tuning model using a population relative policy optimization algorithm to obtain a generative multimodal retrieval model. The input query text is obtained and input into the generative multimodal retrieval model for restricted-bind search, resulting in the image retrieval result corresponding to the query text output by the generative multimodal retrieval model. The restricted-bind search is based on the image token dictionary.

[0008] According to the present invention, a generative text and image cross-modal retrieval method includes the following steps: inputting the query text into the generative multimodal retrieval model for restricted-bind search to obtain the image retrieval result corresponding to the query text output by the generative multimodal retrieval model; performing a restricted-bind search based on the query text and the image token trie to obtain multiple sets of discrete image token sequences; sorting the multiple sets of discrete image token sequences in descending order of similarity, and taking the first-ranked discrete image token sequence as the optimal token sequence result; and matching the optimal token sequence result with the image candidate set to obtain the image retrieval result corresponding to the query text.

[0009] According to the present invention, a generative text and image cross-modal retrieval method is provided, wherein supervised fine-tuning training is performed on a multimodal large language model based on the labeled data of the text samples and the discrete image token sequences to learn the mapping relationship between the text samples and the discrete image token sequences, thereby obtaining a supervised fine-tuning model, comprising:

[0010] The multimodal large language model is subjected to supervised fine-tuning training according to the following formula:

[0011]

[0012] in, This indicates the value of the loss that is being monitored and fine-tuned. This represents the total number of discrete image token sequences. Indicates the first corresponding to the text sample A real discrete image token sequence Indicates the first The generated discrete image token sequence precedes the real discrete image token sequence.

[0013] According to the generative text and image cross-modal retrieval method provided by the present invention, the objective function of the population relative policy optimization algorithm is:

[0014]

[0015] in, This represents the policy objective function. This indicates a supervised fine-tuning model. This represents the total number of discrete image token sequences. Indicates the first A discrete image token sequence Indicates the first The first discrete image token sequence A number of tokens to be generated in steps. Represents a text sample. Indicates the first The first discrete image token sequence Tokens generated up to the previous step size. This represents the optimization strategy for the supervised fine-tuning model. This represents a gradient-free strategy for supervised fine-tuning of the model. This represents a reference strategy for supervising fine-tuning the model. Indicates the dominance value. Indicates KL weights, This represents the KL divergence.

[0016] According to the generative text and image cross-modal retrieval method provided by the present invention, the total reward function optimized by reinforcement learning is formulated as follows:

[0017]

[0018] in, Represents the total reward function. Representation of the reward function format, The correctness reward function, This indicates the first reward weight hyperparameter. This represents the hyperparameter for the second reward weight.

[0019] According to the generative text and image cross-modal retrieval method provided by the present invention, the format reward function includes:

[0020]

[0021] in, Representation of the reward function format, Indicates structural division, Indicates the order of division. Indicates length in cents. This represents the minimum value. Indicates the maximum value;

[0022] The correctness reward function includes:

[0023]

[0024] in, The correctness reward function is represented. Indicates the total length of the generated sequence. Indicates the total length of the standard sequence. Represents the first generation of the generated sequence. One token, Represents the first standard sequence One token, This indicates an indicator function, which is 1 when a match is found and 0 otherwise.

[0025] This invention also provides a generative text and image cross-modal retrieval device, comprising the following modules: an image module, used to input image samples from an image candidate set into the visual encoder of a multimodal large language model for image conversion to obtain discrete image token sequences; a construction module, used to construct an image token dictionary based on the discrete image token sequences to obtain an image token dictionary; a text module, used to acquire text samples and the annotation data of the discrete image token sequences corresponding to the text samples; a supervised fine-tuning module, used to perform supervised fine-tuning training on the multimodal large language model based on the annotation data of the text samples and the discrete image token sequences to learn the mapping relationship between the text samples and the discrete image token sequences to obtain a supervised fine-tuning model; a reinforcement learning module, used to perform reinforcement learning optimization on the supervised fine-tuning model based on a population relative policy optimization algorithm to obtain a generative multimodal retrieval model; and a restricted retrieval module, used to acquire input query text, input the query text into the generative multimodal retrieval model for restricted-beam search, and obtain the image retrieval result corresponding to the query text output by the generative multimodal retrieval model, wherein the restricted-beam search is based on the image token dictionary.

[0026] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the cross-modal retrieval method for generative text and images as described above.

[0027] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the cross-modal retrieval method for generative text and images as described above.

[0028] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the cross-modal retrieval method for generative text and images as described above.

[0029] The generative text and image cross-modal retrieval method and apparatus provided by this invention first encodes images into discrete tokens and constructs a trie to establish a structured image representation space. Then, it uses text-image token annotation data to supervise and fine-tune the model, forcing the learning of a precise mapping relationship between text and image tokens. Next, it optimizes the model strategy through reinforcement learning to improve the accuracy and robustness of generated tokens. Finally, in the retrieval stage, it uses the trie to constrain the decoding path and directly performs a restricted-bind search in the structured token space to ensure that the generated image tokens always conform to the predefined semantic space, thereby simultaneously improving retrieval efficiency (avoiding invalid searches) and result accuracy (forcing alignment of image representations). Attached Figure Description

[0030] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced one by one below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0031] Figure 1 This is a flowchart illustrating the cross-modal retrieval method for generative text and images provided by the present invention.

[0032] Figure 2 This is an overall flowchart of the generative text and image cross-modal retrieval method provided by the present invention.

[0033] Figure 3 This is a schematic diagram of the process for calculating the format reward of the candidate set provided by the present invention.

[0034] Figure 4 This is a flowchart illustrating the process of calculating the correctness reward for the candidate set provided by the present invention.

[0035] Figure 5 This is a schematic diagram of the module of the generative text and image cross-modal retrieval device provided by the present invention.

[0036] Figure 6 This is a schematic diagram of the physical structure of the electronic device provided by the present invention. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0038] This invention belongs to the fields of computer vision and pattern recognition, and is applied to the field of information retrieval. Specifically, it is a generative text-image cross-modal retrieval method based on group relative policy reinforcement learning.

[0039] Existing methods heavily rely on global matching operations (such as cosine similarity) of high-dimensional embedding vectors. When faced with massive amounts of multimodal data, their computational overhead spikes non-linearly, making retrieval efficiency a bottleneck in practical applications.

[0040] With the rapid development of multimodal large language model technology, some studies have proposed generative retrieval techniques. These techniques directly generate high-quality identifiers through multimodal large language models, greatly improving the flexibility and efficiency of retrieval. However, this method mainly relies on the existing cross-modal semantic transfer capabilities of multimodal large language models to obtain the final image response results. Due to the significant semantic gap between text and image modalities, the accuracy of generative retrieval techniques based on multimodal large language models in cross-modal retrieval tasks remains insufficient. Therefore, how to further improve the accuracy of cross-modal retrieval remains the core challenge facing generative text-image cross-modal retrieval tasks.

[0041] This invention proposes a generative text-image cross-modal retrieval method based on group relative policy reinforcement learning. It fully leverages the reasoning capabilities of large language models to further improve the accuracy of generative retrieval methods in text-image cross-modal retrieval tasks. This invention enhances the reasoning ability of large language models through group relative policy reinforcement learning, optimizing the response quality to different text input commands. Furthermore, format rewards and correctness rewards are designed to further optimize the accuracy of generated retrieval results.

[0042] Optionally, the generative text and image cross-modal retrieval method of this application embodiment can be executed by a server, by a terminal device, or by both a server and a terminal device. Taking the execution of the generative text and image cross-modal retrieval method of this embodiment by a server as an example.

[0043] Figure 1 This is a flowchart illustrating the generative text and image cross-modal retrieval method provided by the present invention, as shown below. Figure 1 As shown, the method includes the following steps.

[0044] Step 101: Input the image samples in the image candidate set into the visual encoder of the multimodal large language model for image conversion to obtain a discrete image token sequence.

[0045] In this embodiment of the invention, the image candidate set is selected from the publicly available MS COCO training set. This dataset contains diverse real-world images, ensuring the broad applicability and representativeness of the transformation. The multimodal large language model uses the pre-trained model seed-LLaMA-7B, whose visual encoder module is responsible for processing the input image. This encoder is based on the Transformer architecture, first dividing the image into fixed-size blocks (e.g., 16x16 pixel blocks), and then extracting deep features through multi-layer convolution and self-attention mechanisms. These deep features are quantized into discrete token sequences, each token corresponding to a high-dimensional embedding vector, and the final output is a 32-bit fixed-length discrete image token sequence.

[0046] For example, given an image from MS COCO (such as a scene containing objects), the visual encoder converts it into a sequence of tokens like "tok1 tok2 ... tok32", where each token is an integer index representing a discretized representation of the image in the semantic space.

[0047] Step 102: Construct an image token trie based on the discrete image token sequence to obtain the image token trie.

[0048] The image token trie is a hierarchical index structure built on discrete image token sequences. Its core function is to transform the discrete image token sequences (such as tok1 tok2 ... tok32) generated by the visual encoder into a traversable tree-shaped retrieval space.

[0049] Each discrete image token sequence corresponds to an image sample in the image candidate set. Tree nodes are stored in token order, and the path from the root node to the leaf node completely represents a discrete image token sequence. Each node stores a token value and a pointer to its child nodes. The path from the root node to the leaf node uniquely corresponds to an image sample in the image candidate set. The tree depth is fixed at 32 levels (consistent with the length of the token sequence).

[0050] Step 103: Obtain the annotation data of the text sample and the corresponding discrete image token sequence.

[0051] In this embodiment of the invention, the labeled data used for training (i.e., the labeled data of discrete image token sequences) is obtained using discrete image token sequences and corresponding labeled content obtained from the MS COCO dataset.

[0052] Step 104: Based on the labeled data of text samples and discrete image token sequences, supervised fine-tuning training is performed on the multimodal large language model to learn the mapping relationship between text samples and discrete image token sequences, thereby obtaining the supervised fine-tuning model.

[0053] According to the present invention, a generative text and image cross-modal retrieval method is provided, which, based on labeled data of text samples and discrete image token sequences, performs supervised fine-tuning training on a multimodal large language model to learn the mapping relationship between text samples and discrete image token sequences, thereby obtaining a supervised fine-tuning model, including:

[0054] Supervised fine-tuning training of a multimodal large language model is performed according to the following formula:

[0055]

[0056] in, This indicates the value of the loss that is being monitored and fine-tuned. This represents the total number of discrete image token sequences. Indicates the first corresponding to the text sample A real discrete image token sequence Indicates the first The generated discrete image token sequence precedes the real discrete image token sequence.

[0057] In this embodiment of the invention, each training sample includes: a text sample and a corresponding 32-bit discrete image token sequence. After receiving the text sample input, the multimodal large language model generates the image token sequence bit by bit through an autoregressive method.

[0058] Multimodal large language models in generating the first When dealing with a sequence of discrete image tokens, based on the historical generation sequence (i.e., the previously generated discrete image token sequences, the previous...),... A dynamic context is constructed using a sequence of discrete image tokens, enabling the model to gradually establish a link between textual semantics and visual tokens.

[0059] The multimodal large language model is optimized using the negative log-likelihood loss function (refer to the formula above). During training, for each batch... The first sample: Determine the prediction of the multimodal large language model. A real discrete image token sequence conditional probability ; Quantify the prediction bias through negative logarithmic transformation; Obtain the batch loss value by averaging all token positions.

[0060] In some embodiments, the AdamW optimizer is used for parameter updates, and the loss function is minimized through gradient descent. The performance on the validation set is evaluated after each iteration, and the optimal model parameters are saved.

[0061] Through the embodiments of the present invention, a fine-grained mapping relationship between text concepts and visual tokens is learned by conditionally probabilistically modeling a multimodal large language model, which significantly alleviates the heterogeneity barrier of text-image modalities.

[0062] Step 105: The supervised fine-tuning model is optimized by reinforcement learning based on the group relative policy optimization algorithm to obtain a generative multimodal retrieval model.

[0063] According to the generative text and image cross-modal retrieval method provided by the present invention, the objective function of the population relative policy optimization algorithm is:

[0064]

[0065] in, This represents the policy objective function. This indicates a supervised fine-tuning model. This represents the total number of discrete image token sequences. Indicates the first A discrete image token sequence Indicates the first The first discrete image token sequence A number of tokens to be generated in steps. Represents a text sample. Indicates the first The first discrete image token sequence Tokens generated up to the previous step size. This represents the optimization strategy for the supervised fine-tuning model. This represents a gradient-free strategy for supervised fine-tuning of the model. This represents a reference strategy for supervising fine-tuning the model. Indicates the dominance value. Indicates KL weights, This represents the KL divergence.

[0066] In this embodiment of the invention, the generation strategy is dynamically fine-tuned by using the Group Relative Policy Optimization (GRPO) algorithm based on the supervised fine-tuning model.

[0067] For each text sample Parallel generation Group of discrete image token sequences, each group of discrete image token sequences It contains 32 tokens, and retains historical context during generation. (No. The first discrete image token sequence The generated tokens (previously generated tokens) are used as the basis for autoregression.

[0068] Calculate the strategy to be optimized With gradient-free strategies The ratio of is stabilized during training through a gradient cutoff mechanism.

[0069] The advantage function calculates the advantage value based on format reward and correctness reward:

[0070]

[0071] in, This represents the advantage value. Indicates the first The total reward value of a discrete image token sequence (determined based on format reward and correctness reward). This represents the average reward of all discrete image token sequences in the current batch. This represents the standard deviation of the rewards for all discrete image token sequences in the current batch.

[0072] The KL divergence formula is:

[0073]

[0074] in, This represents the Kullback-Leibler divergence (KL divergence), used to quantify two probability distributions. (Strategy to be optimized) and Differences between (reference strategies).

[0075] Through the embodiments of the present invention, a dual regulation mechanism of reward-driven and constraint-balanced is formed, which enables the generative retrieval model to significantly improve response accuracy while maintaining semantic consistency.

[0076] According to the generative text and image cross-modal retrieval method provided by the present invention, the total reward function optimized by reinforcement learning is formulated as follows:

[0077]

[0078] in, Represents the total reward function. Representation of the reward function format, The correctness reward function is represented. This indicates the first reward weight hyperparameter. This represents the hyperparameter for the second reward weight.

[0079] In this embodiment of the invention, the format reward function is used to evaluate the structural compliance of the generated sequence, including label integrity, label order correctness, and token length correctness. The correctness reward function is used to compare the token matching degree between the generated sequence and the standard sequence.

[0080] Through the embodiments of the present invention, format rewards ensure the structural compatibility of the generated sequence with the retrieval system, while correctness rewards promote the accurate mapping between text semantics and visual tokens.

[0081] According to the present invention, a generative text and image cross-modal retrieval method includes a formatted reward function comprising:

[0082]

[0083] in, Representation of the reward function format, Indicates structural division, Indicates the order of division. Indicates length in cents. This represents the minimum value. Indicates the maximum value;

[0084] The correctness reward function includes:

[0085]

[0086] in, The correctness reward function is represented. Indicates the total length of the generated sequence. Indicates the total length of the standard sequence. Represents the first generation of the generated sequence. One token, Represents the first standard sequence One token, This indicates an indicator function, which is 1 if a match is found, and 0 otherwise.

[0087] In an embodiment of the present invention, The score represents the structure, including the thinking label score ( <think>< / think> ) and image tag ( ); The order score refers to whether the order of the labels and image tags is correct. The length segment indicates whether the token length between image tags is 32 bits.

[0088] In some embodiments, the thinking label is detected first. <think>< / think> ) and image tags ( To ensure the integrity of the structure, assign structural elements based on the presence of tags. (0.5 points for a complete label, maximum 1 point); Secondly, verify whether the order of the labels strictly follows the "thinking-output" logical order, that is, the thinking label must appear completely before the image label. If the requirement is met, the order score will be awarded. The value is 1 if the number of tokens is 1, otherwise 0; finally, check if the number of tokens in the image label is a fixed 32-bit length, and assign a length score if it meets the requirements. The value is 1. After summing the three scores, a double-sided limit is applied using the maximum and minimum value functions to ensure that the formatted reward value falls within the [0,1] interval.

[0089] In some embodiments, the correctness reward function A token-level precise matching mechanism is employed to drive cross-modal semantic alignment. Specifically, the token-level precise matching mechanism is used to generate the first... The token and the first standard sequence Each token is compared bit-to-bit similarity: the matching status of each position is quantified by an indicator function (1 for a match, 0 for a non-match), and the calculation is performed before... Number of matching tokens ( Indicates the total length of the generated sequence. (This represents the total length of the standard sequence), and is finally multiplied by a coefficient of 0.0625 (i.e., 1 / 16) to generate the correctness reward.

[0090] The present invention addresses the structural-semantic conflict problem unique to generative retrieval (such as complete but semantically biased sequences); it establishes an interpretable reward feedback pathway, enabling the model to clearly distinguish between format errors and semantic errors.

[0091] Step 106: Obtain the input query text, input the query text into the generative multimodal retrieval model for restricted-bind search, and obtain the image retrieval results corresponding to the query text output by the generative multimodal retrieval model.

[0092] Among them, the restricted bundle search is based on the image token trie.

[0093] According to the generative text and image cross-modal retrieval method provided by the present invention, the query text is input into a generative multimodal retrieval model for restricted-bind search, and the image retrieval result corresponding to the query text output by the generative multimodal retrieval model is obtained, including:

[0094] Based on the query text, a restricted bundle search is performed according to the image token trie to obtain multiple sets of discrete image token sequences;

[0095] Multiple sets of discrete image token sequences are sorted in descending order of similarity, and the discrete image token sequence with the highest similarity is taken as the optimal token sequence result.

[0096] Based on the optimal token sequence results, matching is performed according to the image candidate set to obtain the image retrieval results corresponding to the query text.

[0097] In this embodiment of the invention, "restricted" in restricted bundle search means that the search space is limited to the constructed image token dictionary tree, so as to ensure that the final retrieval result is an image in the image candidate set.

[0098] The restricted bundle search expands only the k child nodes with the highest probabilities in each bundle, ultimately forming K discrete token sequences. The similarity of the token sequence results is the sum of the probability values ​​of each child node in the token sequence.

[0099] In this embodiment of the invention, images are first encoded into discrete tokens and a trie is constructed to establish a structured image representation space. Then, the model is fine-tuned under the supervision of text-image token annotation data to force the learning of the accurate mapping relationship from text to image tokens. Next, the model strategy is optimized through reinforcement learning to improve the accuracy and robustness of generated tokens. Finally, in the retrieval stage, the decoding path is constrained by the trie, and a restricted-bind search is performed directly in the structured token space to ensure that the generated image tokens always conform to the predefined semantic space, thereby simultaneously improving retrieval efficiency (avoiding invalid searches) and result accuracy (forcing alignment of image representations).

[0100] The following describes an example of the cross-modal retrieval method for generative text and images provided by the present invention in practical applications. It specifically includes the following steps.

[0101] In the first step, the image samples in the image candidate set are converted into a discrete image token sequence through the visual encoder of the multimodal large language model and stored through a trie structure.

[0102] Reference Figure 2 , Figure 2 is the overall flowchart of the cross-modal retrieval method for generative text and images provided by the present invention.

[0103] As Figure 2 shown, in step one, supervised fine-tuning training: the base large language model ( M Figure 2 ,

[0106] ) is fine-tuned into the fine-tuned large model ( M SFT ) through supervised learning, and specific text-image mapping examples (such as the association mechanism between English prompts and image outputs) are shown in the process.

[0104] In step two, reinforcement learning training: after receiving the user's prompt words, the fine-tuned large model ( M SFT ) performs population relative policy reinforcement learning (including: policy model, output, reference model, reward model, KL, reward value, advantage value) to obtain the trained large model ( M ), that is, the generative multimodal retrieval model.

[0105] In step three, inference: the trained large model ( M ) receives the query text (such as "Can you give me an image of a plane?"); builds a tree based on the candidate set to obtain a trie; the trained large model ( M ) performs beam search (taking the restricted beam search beam width = 1 as an example) based on the query text according to the trie to obtain the image retrieval result matching the query text.

[0106] In the second step, as Figure 2 shown, based on the MS COCO dataset, the multimodal large language model is trained using the supervised fine-tuning method.

[0107] The training data of the prompt text and the corresponding image tokens are input according to the following format requirements: <'prompt': <USER: content\nASSISTANT>,'completion: <Sure, Here is an image: image token'>, and the supervised fine-tuning method is used to learn the mapping relationship between the two to obtain the supervised fine-tuning model.

[0108] The third step is to calculate the objective function of the policy model in the group relative policy. .

[0109] The fourth part calculates the format reward based on the candidate set in the third step. .

[0110] refer to Figure 3 , Figure 3 This is a schematic diagram of the process for calculating the format reward of the candidate set provided by the present invention. It includes: model-generated content; detection... <think>< / think> (i.e., determine whether a thought label exists); if so, then Otherwise ; detection (i.e., determine if image tags exist); if so, then Otherwise Check if the number of image tokens is 32; if so, then... Otherwise Normalization yields formatted rewards. (Right now ).

[0111] Fifth, calculate the correctness reward for the candidate set based on the results of the third step. .

[0112] refer to Figure 4 , Figure 4 This is a schematic diagram of the process for calculating the correctness reward of the candidate set provided by the present invention. It includes: model-generated content; extraction... Discrete image tokens between tags are obtained Compare the generated token with the target token, and award 0.625 points for tokens that are the same; This yields... Correctness reward is obtained after normalization. .

[0113] The sixth step, based on steps three through five, employs a group relative strategy optimization algorithm to supervise the fine-tuning of the model for reinforcement learning, resulting in the final generative multimodal retrieval model M.

[0114] Step 7, as Figure 2 As shown in step 3, multiple discrete token sequences are generated using a constrained bundle search based on the generative multimodal retrieval model M. The results are sorted in descending order, and the Rank-1 obtained is taken as the optimal token sequence result.

[0115] The eighth step is to match the images in the candidate set obtained from the optimal token sequence in the seventh step, which will be the final retrieval result.

[0116] The effectiveness of this invention is illustrated by experimental data. To evaluate the performance of the proposed method, experiments were conducted using the publicly available dataset MS COCO.

[0117] Training process of generative text-image retrieval model:

[0118] Input: Each training batch contains 64 query commands and the basic model MBASE;

[0119] Output: The optimal model M after training;

[0120] Initialization: Supervised Fine-tuning of the Model ;

[0121] Step 1: Input query and retrieval instructions into the supervised fine-tuning model Generate a sequence of candidate tokens for retrieval. And record the generation probability distribution ;

[0122] Step 2: Calculate the advantage value for relative strategy optimization for the group. Calculate KL divergence (For details on the meaning of specific parameters, please refer to the above text, and they will not be repeated here.)

[0123] Step 3: Calculation (For details on the meaning of specific parameters, please refer to the above text, and they will not be repeated here.)

[0124] Step 4: Obtain the format reward based on the format reward function. The correctness reward is obtained based on the correctness reward function. The sum of the two was calculated. ;

[0125] Step 5: Update the model parameters W through backpropagation and optimization;

[0126] Repeat the above steps, and after three iterations, save the parameters of the model with the best performance as the optimal model. .

[0127] Testing process:

[0128] Step 1: Input the query command into the optimal model M to generate multiple candidate sequences;

[0129] Step 2: Calculate the score of each generated sequence and sort them in descending order, where Rank-1 is the optimal retrieval result.

[0130] The generative text-image cross-modal retrieval method based on group relative policy reinforcement learning designed in this invention can optimize the generative text-image cross-modal retrieval performance by combining a group relative policy with a format reward function and an accuracy reward function through reinforcement learning.

[0131] The following describes the generative text and image cross-modal retrieval device provided by the present invention. The generative text and image cross-modal retrieval device described below and the generative text and image cross-modal retrieval method described above can be referred to in correspondence with each other.

[0132] refer to Figure 5 , Figure 5 This is a schematic diagram of the module of the generative text and image cross-modal retrieval device provided by the present invention.

[0133] Image module 501 is used to input image samples from the image candidate set into the visual encoder of the multimodal large language model for image transformation to obtain a discrete image token sequence;

[0134] Module 502 is used to construct an image token trie based on a discrete image token sequence, thereby obtaining the image token trie.

[0135] Text module 503 is used to obtain the annotation data of text samples and the discrete image token sequences corresponding to the text samples;

[0136] The supervised fine-tuning module 504 is used to perform supervised fine-tuning training on a multimodal large language model based on labeled data of text samples and discrete image token sequences, so as to learn the mapping relationship between text samples and discrete image token sequences and obtain a supervised fine-tuning model.

[0137] The reinforcement learning module 505 is used to perform reinforcement learning optimization on the supervised fine-tuning model based on the group relative policy optimization algorithm to obtain a generative multimodal retrieval model.

[0138] The restricted retrieval module 506 is used to obtain the input query text, input the query text into the generative multimodal retrieval model for restricted bundle search, and obtain the image retrieval results corresponding to the query text output by the generative multimodal retrieval model. The restricted bundle search is based on the image token trie.

[0139] Specifically, the generative text and image cross-modal retrieval device provided by the present invention can realize all the method steps implemented in the above-mentioned generative text and image cross-modal retrieval method embodiments, and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiments and the beneficial effects will not be described in detail.

[0140] Figure 6This is a schematic diagram of the physical structure of the electronic device provided by the present invention, such as... Figure 6 As shown, the electronic device may include: a processor 610, a communications interface 620, a memory 630, and a communications bus 640, wherein the processor 610, the communications interface 620, and the memory 630 communicate with each other through the communications bus 640. The processor 610 can call logic instructions in the memory 630 to execute a generative text and image cross-modal retrieval method. This method includes: inputting image samples from an image candidate set into the visual encoder of a multimodal large language model for image transformation to obtain discrete image token sequences; constructing an image token dictionary based on the discrete image token sequences to obtain an image token dictionary; acquiring annotation data for text samples and the corresponding discrete image token sequences; performing supervised fine-tuning training on the multimodal large language model based on the annotation data of the text samples and discrete image token sequences to learn the mapping relationship between text samples and discrete image token sequences to obtain a supervised fine-tuning model; optimizing the supervised fine-tuning model using a group relative policy optimization algorithm to obtain a generative multimodal retrieval model; acquiring the input query text, inputting the query text into the generative multimodal retrieval model for restricted-beam search, and obtaining the image retrieval result corresponding to the query text output by the generative multimodal retrieval model, wherein the restricted-beam search is based on the image token dictionary.

[0141] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0142] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the generative text and image cross-modal retrieval method provided by the above methods. The method includes: inputting image samples from an image candidate set into the visual encoder of a multimodal large language model for image conversion to obtain discrete image token sequences; constructing an image token dictionary based on the discrete image token sequences to obtain an image token dictionary; obtaining text samples and the annotation data of the discrete image token sequences corresponding to the text samples; performing supervised fine-tuning training on the multimodal large language model based on the annotation data of the text samples and the discrete image token sequences to learn the mapping relationship between the text samples and the discrete image token sequences to obtain a supervised fine-tuning model; performing reinforcement learning optimization on the supervised fine-tuning model based on a population relative policy optimization algorithm to obtain a generative multimodal retrieval model; obtaining the input query text, inputting the query text into the generative multimodal retrieval model for restricted-bind search, and obtaining the image retrieval result corresponding to the query text output by the generative multimodal retrieval model, wherein the restricted-bind search is based on the image token dictionary.

[0143] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the generative text and image cross-modal retrieval method provided by the above methods. The method includes: inputting image samples from an image candidate set into the visual encoder of a multimodal large language model for image transformation to obtain discrete image token sequences; constructing an image token dictionary based on the discrete image token sequences to obtain an image token dictionary; acquiring annotation data of text samples and the discrete image token sequences corresponding to the text samples; performing supervised fine-tuning training on the multimodal large language model based on the annotation data of the text samples and the discrete image token sequences to learn the mapping relationship between the text samples and the discrete image token sequences to obtain a supervised fine-tuning model; performing reinforcement learning optimization on the supervised fine-tuning model based on a population relative policy optimization algorithm to obtain a generative multimodal retrieval model; acquiring the input query text, inputting the query text into the generative multimodal retrieval model for restricted-bind search, and obtaining the image retrieval result corresponding to the query text output by the generative multimodal retrieval model, wherein the restricted-bind search is based on the image token dictionary.

[0144] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0145] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cross-modal retrieval method for generative text and images, characterized in that, include: Image samples from the image candidate set are input into the visual encoder of the multimodal large language model for image transformation, resulting in a discrete image token sequence; An image token dictionary is constructed based on the discrete image token sequence to obtain the image token dictionary. Obtain the annotation data of the text sample and the corresponding discrete image token sequence; Based on the labeled data of the text samples and the discrete image token sequences, the multimodal large language model is subjected to supervised fine-tuning training to learn the mapping relationship between the text samples and the discrete image token sequences, thereby obtaining the supervised fine-tuning model; The supervised fine-tuning model is optimized by reinforcement learning based on the group relative strategy optimization algorithm to obtain a generative multimodal retrieval model; The query text is obtained and input into the generative multimodal retrieval model for restricted-beam search. The image retrieval result corresponding to the query text is output by the generative multimodal retrieval model. The restricted-beam search is based on the image token trie. The multimodal large language model is subjected to supervised fine-tuning training based on the labeled data of the text samples and the discrete image token sequences to learn the mapping relationship between the text samples and the discrete image token sequences, thereby obtaining a supervised fine-tuning model, including: The multimodal large language model is subjected to supervised fine-tuning training according to the following formula: ； in, This indicates the value of the loss that is being monitored and fine-tuned. This represents the total number of discrete image token sequences. Indicates the first corresponding to the text sample A real discrete image token sequence Indicates the first The generated discrete image token sequence preceding the real discrete image token sequence; The objective function of the group relative policy optimization algorithm is: ； in, This represents the policy objective function. This indicates a supervised fine-tuning model. This represents the total number of discrete image token sequences. Indicates the first A discrete image token sequence Indicates the first The first discrete image token sequence A number of tokens to be generated in steps. Represents a text sample. Indicates the first The first discrete image token sequence Tokens generated up to the previous step size. This represents the optimization strategy for the supervised fine-tuning model. This represents a gradient-free strategy for supervised fine-tuning of the model. This represents a reference strategy for supervising fine-tuning the model. Indicates the dominance value. Indicates KL weights, This represents the KL divergence.

2. The cross-modal retrieval method for generative text and images according to claim 1, characterized in that, The step of inputting the query text into the generative multimodal retrieval model for restricted-bind search, and obtaining the image retrieval result corresponding to the query text output by the generative multimodal retrieval model, includes: Based on the query text, a restricted bundle search is performed according to the image token trie to obtain multiple sets of discrete image token sequences; The discrete image token sequences are sorted in descending order of similarity, and the discrete image token sequence with the highest similarity is taken as the optimal token sequence result. Based on the optimal token sequence result, the image retrieval result corresponding to the query text is obtained by matching according to the image candidate set.

3. The cross-modal retrieval method for generative text and images according to claim 1, characterized in that, The formula for the total reward function optimized by reinforcement learning is: ； in, Represents the total reward function, Representation of the reward function format, The correctness reward function is represented. This indicates the first reward weight hyperparameter. This represents the hyperparameter for the second reward weight.

4. The cross-modal retrieval method for generative text and images according to claim 3, characterized in that, The formatted reward function includes: ； in, Representation of the reward function format, Indicates structural division, Indicates the order of division. Indicates length in cents. This represents the minimum value. Indicates the maximum value; The correctness reward function includes: ； in, The correctness reward function is represented. Indicates the total length of the generated sequence. Indicates the total length of the standard sequence. Represents the first generation of the generated sequence. One token, Represents the first standard sequence One token, This indicates an indicator function, which is 1 when a match is found and 0 otherwise.

5. A cross-modal retrieval device for generative text and images, characterized in that, include: The image module is used to input image samples from the image candidate set into the visual encoder of the multimodal large language model for image transformation, and obtain a discrete image token sequence. The construction module is used to construct an image token trie based on the discrete image token sequence to obtain the image token trie. The text module is used to acquire text samples and the annotation data of the discrete image token sequences corresponding to the text samples; The supervised fine-tuning module is used to perform supervised fine-tuning training on the multimodal large language model based on the labeled data of the text samples and the discrete image token sequences, so as to learn the mapping relationship between the text samples and the discrete image token sequences and obtain the supervised fine-tuning model. The reinforcement learning module is used to perform reinforcement learning optimization on the supervised fine-tuning model based on the group relative policy optimization algorithm to obtain a generative multimodal retrieval model. A restricted retrieval module is used to obtain the input query text, input the query text into the generative multimodal retrieval model for restricted bundle search, and obtain the image retrieval result corresponding to the query text output by the generative multimodal retrieval model, wherein the restricted bundle search is based on the image token trie; The multimodal large language model is subjected to supervised fine-tuning training based on the labeled data of the text samples and the discrete image token sequences to learn the mapping relationship between the text samples and the discrete image token sequences, thereby obtaining a supervised fine-tuning model, including: The multimodal large language model is subjected to supervised fine-tuning training according to the following formula: ； in, This indicates the value of the loss that is being monitored and fine-tuned. This represents the total number of discrete image token sequences. Indicates the first corresponding to the text sample A real discrete image token sequence Indicates the first The generated discrete image token sequence preceding the real discrete image token sequence; The objective function of the group relative policy optimization algorithm is: ； in, This represents the policy objective function. This indicates a supervised fine-tuning model. This represents the total number of discrete image token sequences. Indicates the first A discrete image token sequence Indicates the first The first discrete image token sequence A number of tokens to be generated in steps. Represents a text sample. Indicates the first The first discrete image token sequence Tokens generated up to the previous step size. This represents the optimization strategy for the supervised fine-tuning model. This represents a gradient-free strategy for supervised fine-tuning of the model. This represents a reference strategy for supervising fine-tuning the model. Indicates the dominance value. Indicates KL weights, This represents the KL divergence.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the cross-modal retrieval method for generative text and images as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the generative text and image cross-modal retrieval method as described in any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the generative text and image cross-modal retrieval method as described in any one of claims 1 to 4.

Images