Cross-modal retrieval method based on relation enhancement strategy gradient and parallel weight control
By adopting the method of relationship-enhanced policy gradient and parallel weight control in cross-modal retrieval, the problem of insufficient control of image and text features is solved, the efficiency and effect of cross-modal retrieval is improved, and the implementation of multiple databases is realized. performance of competitiveness.
Patent Information
- Application Number
- CN202410969012.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-19
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2044-07-19
AI Technical Summary
Existing cross-modal retrieval methods have insufficient participation in controlling image and text features, resulting in limited improvements in attention mechanisms and low training and reasoning efficiency.
Using a method of enhanced policy gradient and parallel weight control based on relationships, model the relationship between image and text local features and all features through self-attention mechanism, and use policy gradient and parallel weight control to guide the weight size of image and text global features. , and use hinge losses with difficult-to-negative sample mining for supervision.
The training and inference efficiency of cross-modal retrieval models have been significantly improved, competitive search results have been achieved, and high training and inference efficiency have been shown on multiple databases.
Smart Images

Figure CN118939860A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a cross-modal retrieval method, and in particular to a cross-modal retrieval method based on relationship enhancement strategy gradient and parallel weight control. Background Art
[0002] With the rapid development of multimedia, there is a large amount of information on the Internet, such as images, text, video, audio, etc. It is becoming increasingly difficult to manually obtain useful information between different modalities in massive data. Naturally, we need a powerful method to help us get the text, image or video we need. Cross-modal retrieval uses one modality of data as a query to retrieve relevant data in another modality. For example, we can use text to retrieve images of interest (as we do on Google Image Search), or use images to retrieve corresponding text. Of course, modalities are not limited to images and text, and other modalities such as speech, physical signals, and videos can also be a component of cross-modal retrieval. Many recent cross-modal retrieval methods use pre-trained object detectors to process image features. Images usually contain a lot of visual information, but not all target regions are mentioned in the image description, so it is necessary to control the degree of participation of each target region feature in the final image global feature vector. Previous work used a direct averaging method for each region feature, that is, all target regions were treated equally, but the effect was not good. Some work proposed to use the attention mechanism to determine the importance of each region feature to the final image global feature, and made some progress. However, these methods generally lack effective supervision of the attention mechanism, which makes the improvement of the attention mechanism of cross-modal retrieval tasks still limited. Similar to the image branch, text sentences usually contain nouns, verbs, adjectives, adverbs, prepositions, etc., and not all words contribute to the global feature vector of the final text. Therefore, it is necessary to control the degree of participation of each word feature in the final text global feature vector. Summary of the invention
[0003] In order to better perform cross-modal retrieval, the present invention provides a cross-modal retrieval method based on relationship enhancement policy gradient and parallel weight control.
[0004] The objective of the present invention is achieved through the following technical solutions:
[0005] A cross-modal retrieval method based on relation-enhanced policy gradient and parallel weight control comprises the following steps:
[0006] Step 1: For the image part in the image-text pair, use BUA (Bottom Up Attention) as the basic image network of the image branch, select the image in the image-text pair as the input of the image branch, and output a set of salient area features V = {v1,…,vi ,…,v n} to represent the image, where: v i The feature vector representing the i-th salient region of the encoding, i = 1, ..., n, n represents the number of salient regions in the image, and d represents the dimension of the feature vector;
[0007] Step 2: For the text part of the image-text pair, use word encoding to encode each token into a word vector, and then use the bidirectional GRU as the basic text network of the text branch to convert the word sequence into a text representation U = {u1,…,u i ,…,u m},in: u i The feature vector representing the i-th text word encoded, i = 1, ..., m, where m represents the number of words in the sentence;
[0008] Step 3: Use the self-attention mechanism to model the relationship between any image region feature and all image region features, and use policy gradient and parallel weight control to guide the weights of all image region features in the final image global feature to obtain the image global feature v gl ;
[0009] Step 4: Use the self-attention mechanism to model the relationship between any text word feature and all text word features, and use policy gradient and parallel weight control to guide the weight of all text word features in the final text global feature to obtain the text global feature u gl ;
[0010] Step 5: Use the global features of the image and the global features of the text to perform similarity matching, and use the hinge loss with hard negative sample mining for supervision. Use the optimization objective of the discrete continuous policy gradient to supervise the weights of the image region features and the text word features.
[0011] Compared with the prior art, the present invention has the following advantages:
[0012] 1. The present invention uses the self-attention mechanism to model the relationship between any local feature and all local features within the modality, so as to more accurately use discrete and continuous policy gradients to estimate the weight of the local feature in the final global feature.
[0013] 2. The parallel weight control proposed in this invention significantly improves the training and reasoning efficiency of the cross-modal retrieval model.
[0014] 3. The method of the present invention can perform cross-modal retrieval well, achieves competitive results on multiple databases, and has high training and reasoning efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 The cross-modal retrieval flow chart based on relation enhancement strategy gradient and parallel weight control of the present invention;
[0016] Figure 2 The training process of the present invention on Flickr30k for 30 epochs, DCPG represents the baseline method of the present invention, PGRR represents the method proposed by the present invention, the x-axis and y-axis of (a) represent Step and Loss respectively, indicating that the method of the present invention converges to a lower Loss under the same epoch condition, the x-axis and y-axis of (b) represent Time and Loss respectively, indicating that the method of the present invention takes less time than the baseline method for the same epoch training;
[0017] Figure 3 Reward curves of the method of the present invention for 30 epochs on Flickr30k and MS-COCO, (a) Flickr30k, (b) MS-COCO;
[0018] Figure 4 Visualize the attention weights of the present invention for a given image-text pair input;
[0019] Figure 5 The visual results of the text retrieval of a given image query on the Flickr30K dataset by the method of the present invention are as follows;
[0020] Figure 6 The figure shows the visual results of image retrieval given a text query on the Flickr30K dataset using the method of the present invention. DETAILED DESCRIPTION
[0021] The technical solution of the present invention is further described below in conjunction with the accompanying drawings, but is not limited thereto. Any modification or equivalent replacement of the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention should be included in the protection scope of the present invention.
[0022] The present invention provides a cross-modal retrieval method based on relation enhancement strategy gradient and parallel weight control. Figure 1 The method comprises the following steps:
[0023] Step 1: For the image part in the image-text pair, follow the common spatial learning method to extract the image features. Use BUA (Bottom Up Attention) as the basic image network of the image branch, select the image in the image-text pair as the input of the image branch, and output a set of salient area features V = {v1,…,v i ,…,v n} to represent the image, where: v i The feature vector representing the i-th salient region of the encoding, i = 1, ..., n, n represents the number of salient regions in the image, and d represents the dimension of the feature vector. The specific steps are as follows:
[0024] Step 1: Use BUA to obtain the features of n salient regions of the input image. For any salient region i, extract its average pooled features to obtain the average pooled feature vector
[0025] Step 1 and 2: Apply a fully connected layer to Converted to a d-dimensional vector, as shown in the following formula:
[0026]
[0027] Among them, W v and b v represents the learnable parameters of the fully connected layer.
[0028] Step 2: For the text part of the image-text pair, follow the common spatial learning method to extract the features of the text. Use word encoding to encode each token into a word vector, and then use the bidirectional GRU as the basic text network of the text branch to convert the word sequence into a text representation U = {u1,…,u i ,…,u m},in: u i Represents the feature vector of the i-th text word encoded, i = 1, ..., m, m represents the number of words in the sentence, the specific steps are as follows:
[0029] Step 21: For the text part, a sentence containing m words is converted from a fixed vocabulary into a one-hot encoding W = {w1,…,w i ,…,w m}, where w i Represents the one-hot encoding of the i-th text word;
[0030] Step 22: One-hot encode w of the i-th text word through the word embedding layer i Convert to Vector Where: W e is the learnable parameter matrix of the word embedding layer, represents the dense encoding of the i-th text word;
[0031] Step 23: Enter Bi-GRU, as shown in the following formula:
[0032]
[0033] in and Represent the hidden states of forward and reverse directions respectively;
[0034] Step 24: Word embedding u i is the mean of the two-way hidden state, indicating u i Contains the word w i The context is centered, as shown in the following formula:
[0035]
[0036] Step 3: Use the self-attention mechanism to model the relationship between any image region feature and all image region features, and use policy gradient and parallel weight control to guide the weights of all image region features in the final image global feature to obtain the image global feature v gl , the specific steps are as follows:
[0037] Step 31: Define each image region Where n represents the number of salient regions in the image. The relationship between any image region and all image regions is learned based on the self-attention mechanism as follows:
[0038]
[0039] Where: W Q 、b Q , W K 、b K , W V and b V They represent the learnable parameters of the fully connected layer, f Q 、f K and f V Respectively represent the corresponding activation functions; Q i , K i and V i denote the query vector, key vector and value vector of the i-th image feature in the self-attention mechanism; h i represents the relationship between the i-th image feature and all image features modeled by the self-attention mechanism.
[0040] Step 3.2: Use policy gradient and parallel weight control to obtain global image features
[0041] Step 321: Define a discrete action space containing N candidate actions A = {a1, a2…a N}, define the state s at time t' t' ={h1,h2,…,h n}, the process of generating discrete actions in parallel by the agent parameterized by θ is as follows:
[0042]
[0043] Where: W μ and b μ They represent the learnable parameters of the fully connected layer, f μ represents the corresponding activation function;
[0044] Step 322: Introduce Gumbel-softmax to adjust μ according to probability and randomness t' Sampling is done to select discrete actions:
[0045]
[0046] in: represents the discrete action selected at time t', Denotes a random strategy π θ Choose an action at time t' The probability of , G represents the noise that follows the Gumbel distribution;
[0047] Step 3: Obtaining discrete actions at time t' Then, construct a mean The standard deviation is σ t' and sample continuous actions from a normal distribution The process of the agent generating continuous actions is as follows:
[0048]
[0049] Where: W σ and b σ They represent the learnable parameters of the fully connected layer, f σ represents the corresponding activation function, represents the continuous action selected at time t', Denotes a random strategy π θ Choose an action at time t' The probability of
[0050] Step 324: The weights of all image regions are defined as continuous actions Image global features v gl Defined as:
[0051]
[0052] Where T' represents the number of states in the reinforcement learning process;
[0053] Step 325, the loss functions of discrete and continuous policy gradients are as follows:
[0054]
[0055] Where: R(τ) represents the cumulative reward of trajectory τ; B represents the number of samples in the mini-batch.
[0056] Step 4: Use the self-attention mechanism to model the relationship between any text word feature and all text word features, and use policy gradient and parallel weight control to guide the weight of all text word features in the final text global feature to obtain the text global feature u gl , the specific steps are as follows:
[0057] Step 4.1. Define each text word m represents the number of words in the sentence. The relationship between any text word and all text words is learned based on the self-attention mechanism as follows:
[0058]
[0059] Where: W Q 、b Q , W K 、b K , W V and b V They represent the learnable parameters of the fully connected layer, f Q 、f K and f V Respectively represent the corresponding activation functions;
[0060] Step 4.2: Use policy gradient and parallel weight control to obtain global text features
[0061] Step 421: Define a discrete action space containing N candidate actions A = {a1, a2…a N}, define the state s at time t' t' ={h1,h2,…,h m}, the process of generating discrete actions in parallel by the agent parameterized by θ is as follows:
[0062]
[0063] Where: W μ and b μ They represent the learnable parameters of the fully connected layer, f μ represents the corresponding activation function;
[0064] Step 422: Introduce Gumbel-softmax to adjust μ according to probability and randomness t'Sampling is done to select discrete actions:
[0065]
[0066] in: represents the discrete action selected at time t', Denotes a random strategy π θ Choose an action at time t' The probability of , G represents the noise that follows the Gumbel distribution;
[0067] Step 423: Obtaining discrete actions at time t' Then, construct a mean The standard deviation is σ t' and sample continuous actions from a normal distribution The process of the agent generating continuous actions is as follows:
[0068]
[0069] Where: W σ and b σ They represent the learnable parameters of the fully connected layer, f σ represents the corresponding activation function, represents the continuous action selected at time t', Denotes a random strategy π θ Choose an action at time t' The probability of
[0070] Step 424: The weight of each text word is defined as a continuous action Global feature of text gl Defined as:
[0071]
[0072] Where T' represents the number of states in the reinforcement learning process.
[0073] Step 425, the loss functions of discrete and continuous policy gradients are as follows:
[0074]
[0075] Where: R(τ) represents the cumulative reward of trajectory τ; B represents the number of samples in the mini-batch.
[0076] Step 5: Use the global features of the image and the global features of the text to perform similarity matching, and use the hinge loss with hard negative sample mining for supervision. Use the optimization target of the discrete continuous policy gradient to supervise the weights of the image region features and the text word features. The specific steps are as follows:
[0077] Step 5.1: Use hinge loss with hard negative sample mining as the objective function to optimize image-text matching. When text is used as the query, images that match the query and images that do not match the query are sampled separately in each mini-batch to form positive pairs and negative pairs. The similarity of the positive pair should be at least γ higher than the similarity of the negative pair. Similarly, when images are used as queries, the positive pair represents the text that matches the given query image in each mini-batch, and the negative pair represents the text that does not match the given query image in each mini-batch. The similarity between the negative pair and the positive pair also satisfies the above constraints. Therefore, the ranking loss The definition is as follows:
[0078]
[0079] Where: γ represents the residual parameter; <·,·> represents the similarity function of two vectors, usually cosine similarity; represents the jth global feature of text in the mini-batch, Indicates the number of mini-batch The global features of the matched images, Indicates that the mini-batch is not The global features of the matched images, Indicates that the mini-batch is not Global features of the matched text.
[0080] Step 52: The cumulative reward R(τ) is of great guiding significance to the policy gradient optimization process. This invention defines it as the R@K and average precision (AP) scores in the mini-batch. Specifically, each sample is regarded as a query to calculate the R@1 and AP of all samples in the mini-batch. The cumulative reward R(τ) is defined as follows:
[0081] R(τ)=R@1+AP (33)
[0082] Step 5.3: The policy gradient loss functions of the image and text branches are defined as follows:
[0083]
[0084] Step 54: Optimize the method of the present invention by minimizing the following objective function:
[0085]
[0086] Experimental results:
[0087] For image features, the present invention sets the image feature extraction network with reference to BUA and SCAN. Each image is represented by 36 salient regions, and a 2048-dimensional feature is extracted from each salient region. For text features, the present invention sets the dimension of word embedding to 300, and sets the dimension d of both the image latent space and the text latent space to 1024. In all experiments, the present invention sets the hyperparameter γ of formula (30) to 0.2, and the number of candidate actions in the discrete action space N=500. The present invention uses the Adam optimizer to train PGRR for 30 epochs on the training sets of the Flickr30K dataset and the MS-COCO dataset, respectively, and each epoch verifies PGRR on the corresponding validation set. The mini-batch size of the present invention is 128, the initial learning rate is set to 0.0002, and the learning rate decays by 10% every 15 epochs. For the evaluation of the test set, the present invention uses the recall rate at r (R@r, r=1,5,10) and rSum to measure the performance of PGRR. R@r indicates whether a query image or text contains at least one correct text or image in the first r samples retrieved. rSum indicates the sum of all R@r on image retrieval text tasks and text retrieval image tasks:
[0088]
[0089] Figure 2 The training process of the present invention for 30 epochs on Flickr30k is shown. DCPG represents the baseline method of the present invention (i.e., policy gradient control), and PGRR represents the method proposed by the present invention (i.e., policy gradient and parallel weight control). The x-axis and y-axis of Figure (a) represent Step and Loss, respectively, indicating that the method of the present invention converges to a lower Loss under the same epoch conditions. The x-axis and y-axis of Figure (b) represent Time and Loss, respectively, indicating that for the same epoch training, the method of the present invention takes less time than the baseline method.
[0090] Figure 3 The reward curves of the present invention for 30 epochs on Flickr30k (Figure (a)) and MS-COCO (Figure (b)) are shown. In different cases, the reward of the present invention can gradually increase and eventually converge with the training process, which shows its stability in the reward curve.
[0091] Figure 4The present invention shows the visualization result of the attention weights for a given image-text pair input. The present invention controls the weights of all significant area features to generate the final image global vector, and controls the weights of all word features to generate the final text global vector. From left to right, the first column represents the input image. The second column represents the attention map of the image. The third column represents the weights of the word features in the final text global feature vector. It can be seen that by assigning higher weights to important areas in the image and assigning higher weights to important words in the text, the present invention can generate more robust global features containing accurate semantic information for cross-modal retrieval.
[0092] Figure 5 The visual results of the text retrieval of a given image query on the Flickr30K dataset are shown. Figure 6 The visualization result of image retrieval given a text query on the Flickr30K dataset of the present invention is shown. Here, the samples that are truly matched are marked in green.
[0093] Table 1 shows the quantitative evaluation results of the present invention and mainstream cross-modal retrieval methods in the MS-COCO dataset using the evaluation criteria R@1, R@5, and R@10. For a more comprehensive comparison, the present invention uses two settings (1K and 5K) of the test set on MS-COCO. It can be seen that the present invention achieves the best performance in most indicators in both image retrieval and sentence retrieval tasks. In particular, under the 5K setting, the present invention has achieved significant improvements over previous methods.
[0094] Table 1 Comparison of experimental results with the current mainstream cross-modal retrieval results on the MS-COCO database
[0095]
[0096] Table 2 shows the comparison results of the present invention on the Flickr30K dataset. It can be found that under most evaluation indicators, the present invention is better than other methods, which shows the superiority of the present invention.
[0097] Table 2 Comparison of experimental results with the current best cross-modal retrieval results on the Flickr30K database
[0098]
Claims
1. A cross-modal retrieval method based on relation-enhanced policy gradient and parallel weight control, characterized in that The method comprises the following steps: Step 1: For the image part in the image-text pair, use BUA as the basic image network of the image branch, select the image in the image-text pair as the input of the image branch, and output a set of salient region features V = {v1, ..., v i , …, v n } to represent the image, where: v i The feature vector representing the i-th salient region of the encoding, i = 1, ..., n, n represents the number of salient regions in the image, and d represents the dimension of the feature vector; Step 2: For the text part of the image-text pair, use word encoding to encode each token into a word vector, and then use a bidirectional GRU as the basic text network of the text branch to convert the word sequence into a text representation U = {u1, ..., u i ,…,u m },in: u i The feature vector representing the i-th text word encoded, i = 1, ..., m, where m represents the number of words in the sentence; Step 3: Use the self-attention mechanism to model the relationship between any image region feature and all image region features, and use policy gradient and parallel weight control to guide the weights of all image region features in the final image global feature to obtain the image global feature v gl ; Step 4: Use the self-attention mechanism to model the relationship between any text word feature and all text word features, and use policy gradient and parallel weight control to guide the weight of all text word features in the final text global feature to obtain the text global feature u gl ; Step 5: Use the global features of the image and the global features of the text to perform similarity matching, and use the hinge loss with hard negative sample mining for supervision. Use the optimization objective of the discrete continuous policy gradient to supervise the weights of the image region features and the text word features.
2. The cross-modal retrieval method based on relation enhancement policy gradient and parallel weight control according to claim 1 is characterized in that The specific steps of step one are as follows: Step 1: Use BUA to obtain the features of n salient regions of the input image. For any salient region i, extract its average pooled features to obtain the average pooled feature vector Step 1 and 2: Apply a fully connected layer to Converted to a d-dimensional vector, as shown in the following formula: Among them, W v and b v represents the learnable parameters of the fully connected layer.
3. The cross-modal retrieval method based on relation enhancement policy gradient and parallel weight control according to claim 1 is characterized in that The specific steps of step 2 are as follows: Step 21: For the text part, a sentence containing m words is converted from a fixed vocabulary into a one-hot encoding W = {w1, ..., w i ,…,w m }, where w i Represents the one-hot encoding of the i-th text word; Step 22: One-hot encode w of the i-th text word through the word embedding layer i Convert to Vector Where: W e is the learnable parameter matrix of the word embedding layer, represents the dense encoding of the i-th text word; Step 23: Enter Bi-GRU, as shown in the following formula: in and Represent the hidden states of the forward and reverse directions respectively; Step 24: Word embedding u i is the mean of the two-way hidden state, indicating u i Contains the word w i The context is centered, as shown in the following formula:
4. The cross-modal retrieval method based on relation enhancement policy gradient and parallel weight control according to claim 1 is characterized in that The specific steps of step three are as follows: Step 31: Define each image region i = 1, ..., n, where n represents the number of salient regions in the image. The relationship between any image region and all image regions is learned based on the self-attention mechanism as follows: Where: W Q , b Q , W K , b K , W V and b V They represent the learnable parameters of the fully connected layer, f Q 、f K and f V Respectively represent the corresponding activation functions; Q i , K i and V i denote the query vector, key vector and value vector of the i-th image feature in the self-attention mechanism; h i Represents the relationship between the i-th image feature and all image features modeled by the self-attention mechanism; Step 3.2: Use policy gradient and parallel weight control to obtain global image features Step 321: Define a discrete action space containing N candidate actions A = {a1, a2…a N }, define the state s at time t′ t′ ={h1, h2, ..., h n }, the process of generating discrete actions in parallel by the agent parameterized by θ is as follows: Where: W μ and b μ They represent the learnable parameters of the fully connected layer, f μ represents the corresponding activation function; Step 322: Introduce Gumbel-softmax to adjust μ according to probability and randomness t′ Sampling is done to select discrete actions: in: represents the discrete action selected at time t′, Denotes a random strategy π θ Choose an action at time t′ The probability of , G represents the noise that follows the Gumbel distribution; Step 3: Obtaining discrete actions at time t' Then, construct a mean The standard deviation is σ t′ and sample continuous actions from a normal distribution The process of the agent generating continuous actions is as follows: Where: W σ and b σ They represent the learnable parameters of the fully connected layer, f σ represents the corresponding activation function, represents the continuous action selected at time t′, Denotes a random strategy π θ Choose an action at time t′ probability; Step 324: The weights of all image regions are defined as continuous actions Image global features v gl Defined as: Where T′ represents the number of states in the reinforcement learning process; Step 325, the loss functions of discrete and continuous policy gradients are as follows: Where: R(τ) represents the cumulative reward of trajectory τ; B represents the number of samples in the mini-batch.
5. The cross-modal retrieval method based on relation enhancement policy gradient and parallel weight control according to claim 1 is characterized in that The specific steps of step 4 are as follows: Step 4.
1. Define each text word i = 1, ..., m, where m represents the number of words in a sentence. The relationship between any text word and all text words is learned based on the self-attention mechanism as follows: Where: W Q , b Q , W K , b K , W V and b V They represent the learnable parameters of the fully connected layer, f Q 、f K and f V Respectively represent the corresponding activation functions; Step 4.2: Use policy gradient and parallel weight control to obtain global text features Step 421: Define a discrete action space containing N candidate actions A = {a1, a2…a N }, define the state s at time t′ t′ ={h1, h2, ..., h m }, the process of generating discrete actions in parallel by the agent parameterized by θ is as follows: Where: W μ and b μ They represent the learnable parameters of the fully connected layer, f μ represents the corresponding activation function; Step 422: Introduce Gumbel-softmax to adjust μ according to probability and randomness t′ Sampling is done to select discrete actions: in: represents the discrete action selected at time t′, Denotes a random strategy π θ Choose an action at time t′ The probability of , G represents the noise that follows the Gumbel distribution; Step 423: Obtaining discrete actions at time t′ Then, construct a mean The standard deviation is σ t′ and sample continuous actions from a normal distribution The process of the agent generating continuous actions is as follows: Where: Wσ and bσ represent the learnable parameters of the fully connected layer, fσ represents the corresponding activation function, represents the continuous action selected at time t′, Denotes a random strategy π θ Choose an action at time t′ probability; Step 424: The weight of each text word is defined as a continuous action Global feature of text gl Defined as: Where T′ represents the number of states in the reinforcement learning process. Step 425, the loss functions of discrete and continuous policy gradients are as follows: Where: R(τ) represents the cumulative reward of trajectory τ; B represents the number of samples in the mini-batch.
6. The cross-modal retrieval method based on relation enhancement policy gradient and parallel weight control according to claim 4 is characterized in that The specific steps of step five are as follows: Step 51: Use hinge loss with hard negative sample mining as the objective function to optimize image-text matching. When using text as the query, samples are taken from images matching the query and images not matching the query in each mini-batch to form positive pairs and negative pairs. When using images as the query, positive pairs represent texts matching the given query image in each mini-batch, and negative pairs represent texts not matching the given query image in each mini-batch. Ranking loss The definition is as follows: Where: γ represents the residual parameter; <·, ·> represents the similarity function of two vectors; represents the jth global feature of text in the mini-batch, Indicates the number of mini-batch The global features of the matched images, Indicates that the mini-batch is not The global features of the matched images, Indicates that the mini-batch is not Global features of the matched text; Step 52: Treat each sample as a query to calculate the R@1 and AP of all samples in the mini-batch. The cumulative reward R(τ) is defined as follows: R(τ)=R@1+AP; Step 5.3: The policy gradient loss functions of the image and text branches are defined as follows: Step 54: Optimize the method of the present invention by minimizing the following objective function:
Citation Information
Patent Citations
Acceleration method for reinforcement learning control of process industry based on local data enhancement
CN113836788A
Single stream multi-level alignment for vision-language pretraining
US20230281963A1
