A visual reasoning method based on retrieval-enhanced generation and thought chaining technology

Abstract

The present application relates to a visual reasoning method based on retrieval enhancement generation and thought chain technology, comprising: preprocessing the input original image and dividing the preprocessed original image into multiple regions of interest; performing word segmentation and word embedding on the input question to obtain a feature representation of the question; using the thought chain technology to perform step-by-step reasoning on each region of interest, and combining the obtained reasoning texts in sequence to generate a multi-step reasoning text; using the retrieval enhancement generation technology to search in an external knowledge base based on the question and the multi-step reasoning text to obtain a knowledge fragment; inputting the multi-step reasoning text and the knowledge fragment into the optimized generation model together to obtain a preliminary reasoning result; using the BERT pre-training model to test the logical consistency of the preliminary reasoning result, and after the test is reasonable, streamlining the preliminary reasoning result through the BERT pre-training model to obtain the final reasoning result. This method can effectively improve the accuracy and stability of the visual reasoning model.

Description

Technical Field

[0001] The present application relates to the field of visual reasoning technology, and in particular to a visual reasoning method based on retrieval enhancement generation and thought chain technology. Background Art

[0002] Although many recent studies have attempted to incorporate external knowledge bases (such as encyclopedias) into visual reasoning to enhance the model's reasoning capabilities, existing knowledge incorporation methods still have limitations. Current methods often incorporate external knowledge as independent modules, rather than effectively integrating it with the image and text reasoning process. This approach not only increases model complexity but can also lead to semantic inconsistencies between external knowledge and image content or inferred text, thus affecting the quality of reasoning results. Furthermore, many models rely solely on static, predefined knowledge bases, which poses significant limitations for handling unknown domains or tasks requiring real-time reasoning.

[0003] In addition to the challenge of efficiently incorporating external knowledge, existing models are often treated as a "black box" during inference, lacking transparency and interpretability. This black box nature makes it difficult for users to understand the model's reasoning logic. In applications requiring high interpretability, this black box nature can become a significant obstacle. The lack of visualization and explanation of the model's reasoning process limits its trust and acceptance in real-world applications. Summary of the Invention

[0004] Based on this, it is necessary to provide a visual reasoning method based on retrieval enhancement generation and thought chain technology, which includes:

[0005] S1: preprocess the input original image and divide the preprocessed original image into multiple regions of interest;

[0006] Perform word segmentation and word embedding on the input question to obtain the question feature representation;

[0007] S2: using the thought chain technology to perform step-by-step reasoning on each region of interest, and combining the obtained reasoning texts in sequence to generate a multi-step reasoning text;

[0008] S3: performing feature extraction on the preprocessed original image and the multi-step inference text respectively to obtain an image feature vector and a text feature vector; constructing a loss function based on the similarity between the image feature vector and the text feature vector, and optimizing the generation model based on the loss function;

[0009] S4: Using retrieval-enhanced generation technology to search an external knowledge base based on the question and the multi-step reasoning text to obtain knowledge fragments; inputting the multi-step reasoning text and the knowledge fragments into the optimized generation model to obtain preliminary reasoning results;

[0010] S5: Use the BERT pre-training model to verify the logical consistency of the preliminary reasoning result. If the verification is reasonable, simplify the preliminary reasoning result through the BERT pre-training model to obtain the final reasoning result.

[0011] Beneficial effects: This method enriches the semantic information of the image through thought chain technology and retrieval-enhanced generation technology, and ensures the semantic consistency between the image and the generated reasoning text, thereby effectively improving the accuracy and stability of the visual reasoning model. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0013] Figure 1 This is a flowchart of the visual reasoning method based on retrieval enhancement generation and thought chain technology in an embodiment of the present application. DETAILED DESCRIPTION

[0014] To make the above-mentioned objects, features, and advantages of the present application more clearly understood, the specific embodiments of the present application are described in detail below with reference to the accompanying drawings. The following description sets forth many specific details to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar improvements without violating the scope of the present application. Therefore, the present application is not limited to the specific embodiments disclosed below.

[0015] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of such features. Throughout the description of this application, "plurality" means at least two, for example, two, three, etc., unless otherwise specifically defined.

[0016] like Figure 1 As shown, this embodiment provides a visual reasoning method based on retrieval enhancement generation and thought chain technology, the method comprising:

[0017] S1: Preprocess the input original image and divide the preprocessed original image into multiple regions of interest.

[0018] Specifically, in order to improve the stability and convergence speed of model training, the input original image is preprocessed, and the preprocessing includes normalization processing; the formula for normalization processing is:

[0019] ;

[0020] ;

[0021] ;

[0022] in, represents the normalized image; represents the original image; Represents the mean pixel value of the original image; Represents the standard deviation of the pixel values ​​of the original image; M represents the total number of pixels in the original image; Indicates the original image i The value of pixels;

[0023] Mask-R-CNN is used to divide the normalized image into multiple regions of interest, each of which contains a semantic part of the image, helping the model to better focus on important information in the image.

[0024] Perform word segmentation and word embedding on the input question to obtain the question feature representation.

[0025] Obtaining the problem feature representation includes:

[0026] Segment the question into word sequences and convert the word sequences into embedding vectors using the BERT pre-trained model;

[0027] Add position encoding to the embedding vector to get the initial representation of the problem, calculated as:

[0028] ;

[0029] in, represents the initial representation of question Q; Embedding vector representing question Q; Represents the positional encoding of question Q;

[0030] The initial representation is input into the BERT pre-trained model to obtain the problem feature representation.

[0031] Example:

[0032] Suppose the input image shows a man holding a ring, kneeling on one knee in a heart-shaped wreath, facing a smiling woman. After image segmentation, the following regions of interest are obtained: R1: man; R2: woman; R3: ring; R4: heart-shaped wreath; R5: man's kneeling posture; R6: woman's expression.

[0033] Suppose the input question is: "What is this man doing?", and its word sequence is ["this", "man", "is", "doing", "what", "behavior"].

[0034] S2: Using the thought chain technology to perform step-by-step reasoning on each of the regions of interest, and combining the obtained reasoning texts in sequence to generate a multi-step reasoning text.

[0035] Specifically, generating a multi-step reasoning text includes:

[0036] A convolutional neural network is used to extract features from each region of interest to obtain the feature representation corresponding to each region of interest. The calculation formula is:

[0037] ;

[0038] in, Indicates the area of ​​interest The corresponding feature representation is, , d Represents feature dimension; represents a convolutional neural network;

[0039] The feature representation corresponding to each region of interest is input into the generative model to generate the inference text corresponding to each region of interest. The calculation formula is:

[0040] ;

[0041] in, Indicates the area of ​​interest corresponding reasoning text; represents a generative model;

[0042] The reasoning texts corresponding to all regions of interest are combined in a logical order to generate the multi-step reasoning text. The calculation formula is:

[0043] ;

[0044] in, Represents a multi-step reasoning text, which contains a detailed description of each important area of ​​the image and the reasoning process, enriching the semantic information of the image; Indicates a text link operation; Indicates the reasoning text corresponding to the nth region of interest.

[0045] In this embodiment, the generation model includes LSTM or Transformer.

[0046] Continuing with the above example, feature representations are extracted for each region of interest and inference text is generated: s1: A man kneels on one knee; s2: A woman stands in front of the man; s3: The man holds a ring; s4: They are surrounded by a heart-shaped wreath, which often symbolizes romantic love; s5: The woman smiles, possibly expressing joy or acceptance. By combining this enriched semantic information, a multi-step inference text is generated: A man kneels on one knee, holding a ring; a woman stands in front of the man, smiling, which often indicates acceptance or joy; they are surrounded by a heart-shaped wreath, which often symbolizes romantic love.

[0047] S3: Feature extraction is performed on the preprocessed original image and the multi-step inference text respectively to obtain an image feature vector and a text feature vector; a loss function is constructed based on the similarity between the image feature vector and the text feature vector, and a generation model is optimized based on the loss function.

[0048] Specifically, obtaining the image feature vector and the text feature vector includes:

[0049] The preprocessed original image is divided into m fixed-size image blocks, each image block is expanded into a vector, and linear embedding is performed to obtain the feature representation of each image block. The calculation formula is:

[0050] ;

[0051] in, represents the feature representation of the i-th image block; represents a linear transformation; Represents a flattening operation; represents the i-th image block;

[0052] The second position code is added to the feature representation of each image block to obtain the initial features of each image block. The calculation formula is:

[0053] ;

[0054] in, Indicates the i The initial features of the image blocks; Indicates the i a second position code of an image block;

[0055] The initial features of all image blocks are input into the Transformer to obtain the image feature representation, which is calculated as follows:

[0056] ;

[0057] in, represents image feature representation, , m represents the number of image blocks; Represents the Transformer model;

[0058] The multi-step reasoning text is segmented to obtain words , convert each word into a word vector , and add the third position code to each word vector to obtain the initial text features of each word vector. The calculation formula is:

[0059] ;

[0060] in, Indicates the j Initial text features of word vectors; Indicates the j word vectors; Indicates the j The third position encoding of the word vector;

[0061] The initial text features are input into the Transformer to obtain the text feature representation, which is calculated as follows:

[0062] ;

[0063] in, represents text feature representation, , Indicates the number of words.

[0064] In order to deeply fuse the image feature representation with the text feature representation, this embodiment introduces a cross-attention mechanism. First, based on the image feature representation and the text feature representation, the first attention weight matrix from image to text is calculated. The calculation formula is:

[0065] ;

[0066] in, Represents the first attention weight matrix from image to text; represents the softmax activation function; Represents the query matrix, which is used to convert the input feature representation F (text feature representation or image feature representation) into a query vector, calculate the similarity with the key, and determine the feature parts that need attention; The linear transformation matrix representing the key converts the input feature representation F into a key vector, which is then dot-producted with the query vector to measure the correlation between different modalities. Represents the scaling factor to prevent the inner product value from being too large; T represents transpose;

[0067] The text feature representation is weighted based on the first attention weight matrix to obtain an image feature vector, which is calculated as follows:

[0068] ;

[0069] in, represents the image feature vector; The linear transformation matrix representing the value converts the input feature representation F into a value vector for the final weighted summation to generate a new feature representation;

[0070] Based on the image feature representation and the text feature representation, the second attention weight matrix from text to image is calculated as follows:

[0071] ;

[0072] in, Represents the second attention weight matrix from text to image; represents the softmax activation function; Represents the query matrix, which is used to convert the input feature representation F (text feature representation or image feature representation) into a query vector, calculate the similarity with the key, and determine the feature parts that need attention; The linear transformation matrix representing the key converts the input feature representation F into a key vector, which is then dot-producted with the query vector to measure the correlation between different modalities. Represents the scaling factor to prevent the inner product value from being too large; T represents transpose;

[0073] The image feature representation is weighted based on the second attention weight matrix to obtain a text feature vector, which is calculated as follows:

[0074] ;

[0075] in, Represents text feature vector; The linear transformation matrix representing the value converts the input feature representation F into a value vector for the final weighted summation to generate a new feature representation.

[0076] The text feature representation generated by thought chain reasoning represents the intermediate reasoning steps generated by the model during the reasoning process. These features contain rich semantic information and logical relationships. However, relying solely on text features for reasoning may not fully utilize the visual information in the image. By introducing the cross-attention mechanism, the text feature representation can be effectively combined with the image feature representation, enhancing the semantic expression and contextual association of the text feature representation.

[0077] The cross-attention mechanism integrates the fusion of text and image feature representations, dynamically focusing on image regions relevant to the current text reasoning step. This attention mechanism ensures that each step in text reasoning leverages visual cues in the image, thereby improving the accuracy of the reasoning task. For example, when identifying a "marriage proposal scene," the cross-attention mechanism can associate the "ring" and "kneeling on one knee" features in the text representation with specific visual regions in the image (such as the ring's position and the kneeling posture), making the reasoning more consistent with the actual visual content.

[0078] Furthermore, constructing a loss function based on the similarity between the image feature vector and the text feature vector, and optimizing the generation model based on the loss function includes:

[0079] Calculate the first cosine similarity between the image feature vector and the text feature vector;

[0080] Calculating the second cosine similarity between the image feature vector and the text feature vector of the negative sample;

[0081] Constructing a contrast loss function based on the first cosine similarity and the second cosine similarity;

[0082] Adding a regularization term to the contrast loss function to obtain the loss function;

[0083] Minimize the loss function, update the model parameters of the generative model, and obtain an optimized generative model.

[0084] In this embodiment, the loss function is expressed as:

[0085] ;

[0086] ;

[0087] ;

[0088] ;

[0089] in, represents the loss function; represents the contrast loss function; represents the regularization term; represents cosine similarity; represents the image feature vector; Represents text feature vector; represents the temperature parameter; N represents the number of negative samples; Indicates the i The text feature vector of negative samples; represents the regularization coefficient;

[0090] The model parameter update formula is:

[0091] ;

[0092] in, represents the updated model parameters; Represents the model parameters before updating; represents the learning rate; Represents the loss function Gradients with respect to the model parameters.

[0093] S4: Using retrieval-enhanced generation technology to search in an external knowledge base based on the question and the multi-step reasoning text to obtain knowledge fragments; inputting the multi-step reasoning text and the knowledge fragments into the optimized generation model together to obtain preliminary reasoning results.

[0094] Specifically, the question and the multi-step reasoning text are input into the retrieval model, and the text related to the question is retrieved from the external knowledge base based on the retrieval enhancement generation technology to obtain the knowledge fragment, which is expressed as:

[0095] ;

[0096] in, Represents a piece of knowledge; represents the retrieval model; Q represents the question; represents the multi-step reasoning text; D represents the external knowledge base (such as Wikipedia, Baidu Encyclopedia, etc.);

[0097] The retrieval enhancement generation technology includes a retrieval method based on the FAISS or BM25 model.

[0098] The multi-step reasoning text and the knowledge fragment are input into the optimized generation model to obtain the preliminary reasoning result, which is expressed as:

[0099] ;

[0100] in, Indicates preliminary reasoning results; represents the optimized generative model.

[0101] Continuing with the above example, based on the terms "man," "kneeling on one knee," and "holding a ring" in the multi-step reasoning text, and the question "What is this man doing?", we can retrieve snippets from the external knowledge base: "Holding a ring is usually a sign of proposal," "Kneeling on one knee is a common gesture for proposals," and "Proposal scenes usually feature heart-shaped wreaths." Combining these retrieved knowledge snippets with the multi-step reasoning text generates a more detailed and accurate preliminary reasoning result: a man kneeling on one knee, holding a ring, and a woman standing in front of him with a smile, which typically indicates acceptance or happiness. They are situated within a heart-shaped wreath, which often symbolizes romantic love. This gesture and context indicate that the man is proposing to the woman.

[0102] S5: Use the BERT pre-training model to verify the logical consistency of the preliminary reasoning result. If the verification is reasonable, simplify the preliminary reasoning result through the BERT pre-training model to obtain the final reasoning result.

[0103] Specifically, this step includes:

[0104] Input any sentence pair in the preliminary inference result into the BERT pre-trained model and output the causal relationship judgment result of the two sentences in the sentence pair, which is expressed as:

[0105] ;

[0106] in, Expressing sentences With sentence The result of the causal relationship determination between the two sentences is a probability value ranging from 0 to 1; when the probability value is higher than the preset probability, it is determined that there is a reasonable causal relationship between the two sentences; when the probability value is greater than 0 and less than or equal to the preset probability, it is determined that there is an unreasonable causal relationship between the two sentences; when the probability value is equal to 0, it is determined that there is no causal relationship between the two sentences; represents the sigmoid activation function; W represents the weight of the BERT pre-training model; T represents transposition; Represents the sentence vector after the transformation of the i-th sentence in the preliminary reasoning result; Represents the sentence vector after the j-th sentence is transformed in the preliminary reasoning result; Represents the concatenation of sentence vectors; b represents the bias term;

[0107] If it is determined that there is a reasonable causal relationship between the two sentences, the two sentences are marked as reasonable causal relationship, and the causal relationship determination results of the two sentences in the remaining sentence pairs are continued to be determined until all the sentence pairs in the preliminary reasoning results are determined;

[0108] If it is determined that there is an unreasonable causal relationship between the two sentences, optimizing the generation model;

[0109] If it is determined that there is no causal relationship between the two sentences, the two sentences are marked as having no causal relationship, and the causal relationship determination results of the two sentences in the remaining sentence pairs are determined until all the sentence pairs in the preliminary reasoning results are determined;

[0110] After the preliminary reasoning results are verified, the preliminary reasoning results are simplified through the BERT pre-training model to obtain the final reasoning results.

[0111] Continuing with the above example, the BERT pre-trained model simplifies the preliminary inference result to: the man is proposing to the woman.

[0112] The visual reasoning method based on retrieval enhancement generation and thought chain technology provided in this embodiment has the following beneficial effects:

[0113] 1. Enhanced multi-step reasoning capabilities: By introducing thought chaining technology, the visual reasoning model can reason step by step, avoiding the errors or omissions that can occur in the traditional method of rapid reasoning. Each step of reasoning has clear logic and intermediate steps, improving the accuracy of reasoning and the interpretability of the reasoning process. This can effectively reduce the occurrence of incorrect reasoning when handling complex, multi-step reasoning tasks.

[0114] 2. Enhance the model's commonsense reasoning capabilities: The introduction of retrieval-augmented generation technology enables the model to obtain relevant commonsense or background information from external knowledge bases, making up for the lack of contextual support in images and text, thereby generating more accurate reasoning results. This improves the model's performance in complex scenarios containing symbolic information.

[0115] 3. Improved reasoning performance in complex scenarios: By combining multi-step reasoning with commonsense reasoning, this method can handle more complex reasoning tasks. By combining thought chaining technology with retrieval-enhanced generation technology, this method not only enables multi-level, multi-step reasoning, but also retrieves rich background information from external knowledge bases, greatly improving the diversity and robustness of reasoning tasks.

[0116] 4. Enhanced cross-modal understanding and reasoning: By combining contrastive learning with a cross-attention mechanism, this method further enhances cross-modal understanding and reasoning between images and text. The deep fusion of image and text feature representations enables the inference model to more accurately capture the relationship between images and text, improving the model's reasoning capabilities in complex scenarios.

[0117] 5. Improve the practicality of the model in practical applications. The model can adapt to a variety of visual reasoning tasks, such as visual question answering, image description generation, and visual common sense reasoning, and has broad application prospects.

[0118] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0119] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A visual reasoning method based on retrieval-enhanced generation and thought chain technology, characterized in that: include: S1: preprocess the input original image and divide the preprocessed original image into multiple regions of interest; Perform word segmentation and word embedding on the input question to obtain the question feature representation; S2: using the thought chain technology to perform step-by-step reasoning on each region of interest, and combining the obtained reasoning texts in sequence to generate a multi-step reasoning text; S3: performing feature extraction on the preprocessed original image and the multi-step inference text respectively to obtain an image feature vector and a text feature vector; constructing a loss function based on the similarity between the image feature vector and the text feature vector, and optimizing the generation model based on the loss function; Obtaining the image feature vector and the text feature vector includes: The preprocessed original image is divided into m fixed-size image blocks, each image block is expanded into a vector, and linearly embedded to obtain the feature representation of each image block; a second position code is added to the feature representation of each image block to obtain the initial feature of each image block; Input the initial features of all image blocks into the Transformer to obtain the image feature representation; Segmenting the multi-step reasoning text to obtain multiple words, converting each word into a word vector, and adding a third position code to each word vector to obtain initial text features of each word vector; Inputting the initial text features into the Transformer to obtain text feature representation; Calculating a first image-to-text attention weight matrix based on the image feature representation and the text feature representation; Weighting the text feature representation based on the first attention weight matrix to obtain an image feature vector; Calculating a second attention weight matrix from text to image based on the image feature representation and the text feature representation; Weighting the image feature representation based on the second attention weight matrix to obtain a text feature vector; S4: Using retrieval-enhanced generation technology to search an external knowledge base based on the question and the multi-step reasoning text to obtain knowledge fragments; inputting the multi-step reasoning text and the knowledge fragments into the optimized generation model to obtain preliminary reasoning results; S5: Use the BERT pre-training model to verify the logical consistency of the preliminary reasoning result. If the verification is reasonable, simplify the preliminary reasoning result through the BERT pre-training model to obtain the final reasoning result.

2. The visual reasoning method based on retrieval enhancement generation and thought chain technology according to claim 1 is characterized in that: The preprocessing includes normalization processing; Mask-R-CNN is used to divide the normalized image into multiple regions of interest, each of which contains a semantically significant part of the image.

3. The visual reasoning method based on retrieval enhancement generation and thought chain technology according to claim 1 is characterized in that: Obtaining the problem feature representation includes: Segment the question into word sequences and convert the word sequences into embedding vectors using the BERT pre-trained model; Add positional encoding to the embedding vector to obtain an initial representation of the problem; The initial representation is input into the BERT pre-trained model to obtain the problem feature representation.

4. The visual reasoning method based on retrieval enhancement generation and thought chain technology according to claim 1 is characterized in that: Generating multi-step reasoning text includes: Using a convolutional neural network to extract features from each region of interest, and obtaining feature representations corresponding to each region of interest; Input the feature representation corresponding to each region of interest into the generative model to generate the inference text corresponding to each region of interest; The reasoning texts corresponding to all the regions of interest are combined in a logical order to generate the multi-step reasoning text.

5. The visual reasoning method based on retrieval enhancement generation and thought chain technology according to claim 1 is characterized in that: The constructing of a loss function based on the similarity between the image feature vector and the text feature vector, and optimizing the generation model based on the loss function include: Calculate the first cosine similarity between the image feature vector and the text feature vector; Calculating the second cosine similarity between the image feature vector and the text feature vector of the negative sample; Constructing a contrast loss function based on the first cosine similarity and the second cosine similarity; Adding a regularization term to the contrast loss function to obtain the loss function; Minimize the loss function, update the model parameters of the generative model, and obtain an optimized generative model.

6. The visual reasoning method based on retrieval enhancement generation and thought chain technology according to claim 5 is characterized in that: The expression of the loss function is: ； ； ； ； in, represents the loss function; represents the contrast loss function; represents the regularization term; represents cosine similarity; represents the image feature vector; Represents text feature vector; represents the temperature parameter; N represents the number of negative samples; Indicates the i The text feature vector of negative samples; represents the regularization coefficient.

7. The visual reasoning method based on retrieval enhancement generation and thought chain technology according to claim 1 is characterized in that: In S4, Inputting the question and the multi-step reasoning text into a retrieval model, retrieving text related to the question in an external knowledge base based on retrieval enhancement generation technology to obtain knowledge fragments; The retrieval enhancement generation technology includes a retrieval method based on the FAISS or BM25 model.

8. The visual reasoning method based on retrieval enhancement generation and thought chain technology according to claim 1 is characterized in that: In S5, Input any sentence pair in the preliminary inference result into the BERT pre-trained model, and output the causal relationship determination result between the two sentences in the sentence pair; If it is determined that there is a reasonable causal relationship between the two sentences, the two sentences are marked as reasonable causal relationship, and the causal relationship determination results of the two sentences in the remaining sentence pairs are continued to be determined until all the sentence pairs in the preliminary reasoning results are determined; If it is determined that there is an unreasonable causal relationship between the two sentences, optimizing the generation model; If it is determined that there is no causal relationship between the two sentences, the two sentences are marked as having no causal relationship, and the causal relationship determination results of the two sentences in the remaining sentence pairs are determined until all the sentence pairs in the preliminary reasoning results are determined; After the preliminary reasoning results are verified, the preliminary reasoning results are simplified through the BERT pre-training model to obtain the final reasoning results.

9. The visual reasoning method based on retrieval enhancement generation and thought chain technology according to claim 1 is characterized in that: The generation model includes LSTM or Transformer.

Images