A method for overcoming prior influence in visual question answering by cumulative learning strategy

Abstract

The application discloses a method for overcoming the influence of priori in visual question answering through a cumulative learning strategy, aiming at overcoming the influence of priori bias on performance in the existing visual question answering field. The method comprises the following steps: inputting multimodal data of vision and language, and constructing an efficient learning framework for main and secondary categories by using a double representation learning branch; performing statistics on candidate answers of the language modal data to obtain priori representation of training data; realizing robust representation of the main category and step-by-step attention to the secondary category through a joint learning module, and effectively fusing multimodal features; under the guidance of the cumulative learning strategy, firstly focusing on the main category in the data set, and then gradually shifting the focus to the secondary category, so that more balanced learning is realized. The method can be widely applied to the fields of natural language processing and computer vision, and has better effect and stronger generalization ability.

Description

Technical Field

[0001] This invention relates to the field of computer vision question answering technology, and more specifically to a method for overcoming prior influences in visual question answering through a cumulative learning strategy. Background Technology

[0002] In recent years, Visual Question Answering (VQA) has made significant progress. VQA tasks require machines to automatically answer questions about a given image, which involves a sophisticated semantic understanding of both the image and the question, as well as the ability to predict accurate answers through visual reasoning. However, many current VQA models tend to cheat by exploiting superficial semantic relevance between questions and answers. This often results in the model failing to adequately learn fused multimodal knowledge from both visual and linguistic sources, leading to significant performance losses.

[0003] To address this issue, Agrawal et al. proposed the VQA-CP v2.0 dataset, which significantly suppresses the gains achieved through surface semantic relevance by reshaping the distribution of the training and test sets of VQA v2.0. Nevertheless, achieving high performance on real-world datasets still requires effectively learning a small number of classes. Similar to most real-world datasets, VQA-CP v2.0 exhibits a typical long-tailed distribution, where a small number of dominant classes constitute the majority of samples, while most other classes have very few samples.

[0004] In dealing with long-tailed datasets, some rebalancing methods focus on addressing the problem of undertraining on a small number of classes by resampling training data or reweighting training loss.

[0005] However, none of the above methods take into account the data distribution in the dataset, which hinders the model's in-depth understanding of the image. In the process of researching and proposing the invention technical solution, it was found that the above influence can be overcome by considering the data distribution in the dataset. Summary of the Invention

[0006] The purpose of this invention is to address the aforementioned deficiencies in the prior art and provide a method for overcoming prior influences in visual question answering through a cumulative learning strategy.

[0007] The objective of this invention can be achieved by adopting the following technical solutions:

[0008] A method for overcoming prior influences in visual question answering through a cumulative learning strategy, the method comprising the following steps:

[0009] S1. Dataset Construction: This step aims to collect real-world images along with their associated questions and answers, forming question-image-answer triples. These triples are aggregated into a dataset specifically designed for visual question answering, and the dataset is sampled using a uniform sampling strategy.

[0010] S2. Data Processing: Perform statistical analysis on the question-answer pairs within the dataset and calculate the answer distribution bias;

[0011] S3. Feature Learning: Input the questions and images in the dataset into two independent first and second feature learning branches. Each branch performs word embedding and feature extraction on the questions and extracts features from the images, obtaining question features Q and image features I respectively. Then, input the question features Q and image features I into the dot product attention module to generate secondary features I2 of the image. The secondary features I2 of the image and the question features Q are multiplied element-wise to obtain secondary fusion features. The secondary fusion features in the first feature learning branch are denoted as f1, and the secondary fusion features in the second feature learning branch are denoted as f2. f1 and f2 are fed into two classifiers respectively. The two classifiers map f1 and f2 to a one-dimensional probability array, where each value in the array represents the probability that a candidate answer is true. The output probabilities of the two classifiers are denoted as p1 and p2 respectively.

[0012] S4. Loss Calculation and Fusion: The predicted probability p is obtained by weighted summation of the classifier output probabilities p1 and p2. The predicted probability p is compared with the correct answer corresponding to the image to calculate the binary cross-entropy (BCE) loss and LMH loss respectively. The BCE loss and LMH loss are weighted summation to obtain the final loss used to optimize the visual question answering model.

[0013] S5. Training parameter adjustment: During the training process, the weighting parameter α is adjusted to control the proportion of BCE loss and LMH loss in the final loss, and to control the proportion of p1 and p2 in the prediction probability p.

[0014] S6. Parameter optimization: Calculate the gradient of the parameters of the visual question answering model using the final loss and optimize the parameters using the Adam optimizer.

[0015] S7. Model Training: Repeat steps S2 to S6 for the specified number of training epochs to complete the training of the visual question answering model.

[0016] S8. Model Application: Input the actual captured images and their corresponding questions into the trained visual question answering model, and the model will output the corresponding answers.

[0017] Furthermore, in step S1, the data in the dataset is uniformly sampled to maintain the distribution characteristics of the original data. Specifically, the probability P of the i-th question category being sampled is... i Represented as:

[0018]

[0019] Where N i Let represent the number of samples of the i-th question category in the training set, and n represent the total number of question categories.

[0020] Furthermore, step S2 calculates the answer distribution bias so that its impact can be considered when calculating the LMH loss. To this end, for each question category, the frequency of occurrence of all candidate answers within that category is first calculated to obtain the frequency of occurrence of each answer, thus forming the answer distribution bias array B. i,type , is represented as:

[0021]

[0022] B i,type The array represents the bias in the answer distribution for the i-th question category, a. j Let C be the number of the j-th candidate answer, C represent the total number of candidate answers, and A be the sum of the occurrences of all candidate answers.

[0023] Furthermore, the dot product attention module in step S3 is implemented through the following attention mechanism:

[0024] I2=softmax(linear(Q)·linear(I) T )

[0025]

[0026] Where linear represents a fully connected linear layer applied to the problem feature Q and the image feature I. T This represents the transpose operation of a matrix. The function softmax() is an activation function widely used in deep learning, x u Let represent the value of the u-th element of x. The function softmax() can convert the vector x into an array of probability distributions that sum to 1. The dot product attention module makes the visual question answering model pay more attention to objects in the image mentioned in the question.

[0027] Furthermore, in step S4, the BCE loss and LMH loss are used in combination. The BCE loss is calculated as follows:

[0028]

[0029] Where d is the score corresponding to the correct answer, d′ is the score predicted by the model, and C is the number of candidate answers. k This represents the value of the k-th element of d. k ' represents the value of the k-th element of d'.

[0030] The LMH loss is calculated as follows:

[0031] LMH(s,s′)=BCE(softmax(log(s)+g(f2)log(B) i,type ),s'))+R

[0032] R = ωH(softmax(g(f2))log(B) i,type ))

[0033] H(z)=∑ r z r log(z r )

[0034] Where s is the score corresponding to the correct answer, s′ is the score predicted by the model, g(·) is a learnable neural network layer, R is the penalty term, ω is a hyperparameter that adjusts the influence of the penalty term, and z r It is the value of the r-th element of z.

[0035] Furthermore, in steps S4 and S5, by adjusting the parameter α, the proportions of BCE loss and LMH loss in the final loss can be changed:

[0036] loss=αBCE(p,y)+(1-α)LMH(p,y)

[0037] Where y is the corresponding correct answer.

[0038] α also changes the proportions of p1 and p2 in the predicted probability p:

[0039] p = αp1 + (1-α)p2.

[0040] Furthermore, the weight parameter α gradually decreases as the visual question answering model iterates, and the decrease in α is as follows:

[0041]

[0042] Where τ is a fixed hyperparameter controlling the rate of change of the weights; the larger τ is, the faster α decreases. T is the current iteration number. max It represents the maximum number of iterations.

[0043] Furthermore, the image and text feature extraction in step S3 is achieved through a neural network structure. Specifically, the input image is processed by a convolutional neural network (CNN) to extract image features, while text features are first embedded using word embedding techniques such as Word2Vec or GloVe, and then extracted using a recurrent neural network (RNN) or Transformer structure.

[0044] The present invention has the following advantages and effects compared with the prior art:

[0045] 1. By simultaneously employing BCE loss and LMH loss, this invention significantly improves performance on classes with scarce samples, thereby enhancing overall performance. LMH loss excels in classifying classes with scarce samples, while BCE loss performs better in terms of feature learning quality. This strategy effectively combines the advantages of both, optimizing both feature learning quality and classification accuracy for scarce classes.

[0046] 2. This method employs an adaptive weight parameter α, enabling a smooth transition from feature learning to classification learning during training. In the early stages of training, since α is close to 1, the BCE loss dominates the final loss, prompting the model to prioritize learning high-quality feature representations. As training progresses, the number of training iterations T gradually approaches the maximum number of training iterations T. max As α approaches zero, the impact of the LMH loss gradually increases, making the model more focused on the scarce data categories. This adaptive adjustment not only solves the generalization problem of traditional methods when dealing with scarce data categories, but also significantly enhances the robustness of the model.

[0047] 3. In this method, the weight-sharing strategy between the two feature learning branches allows the high-quality representation capabilities acquired early in training to be shared between the two branches. This design significantly reduces the number of training parameters required while maintaining model performance, achieving a balance between parameter efficiency and performance. Attached Figure Description

[0048] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0049] Figure 1 This is a reference diagram of the model disclosed in this invention that overcomes prior influences in visual question answering through a cumulative learning strategy;

[0050] Figure 2 This is a schematic diagram illustrating the predicted performance of some test cases of the present invention on the VQA-CP v2.0 test set;

[0051] Figure 3This is a flowchart of a method for overcoming prior influences in visual question answering through a cumulative learning strategy, as described in an embodiment of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] Example 1

[0054] This embodiment provides a method for overcoming prior influences in visual question answering through a cumulative learning strategy. The implementation process is as follows: Figure 3 As shown, the specific process is as follows:

[0055] S1. Constructing the dataset. Collect real-world images and their associated questions and answers, forming question-image-answer triples. These triples are aggregated into a dataset specifically designed for visual question answering. The dataset uses a uniform sampling strategy for sample selection. Specifically, the probability P of the i-th question category being sampled... i Represented as:

[0056] Where N i Let represent the number of samples of the i-th question category in the training set, and n represent the total number of question categories.

[0057] S2. Perform statistical analysis on the question-answer pairs within the dataset to calculate the answer distribution bias, so that the impact of the answer distribution bias can be considered when calculating the LMH loss. In existing technologies, to capture language modality bias, mainstream ensemble-based methods typically rely on an additional neural network branch to independently handle the bias, thereby balancing the impact of the answer distribution bias in subsequent processing. However, this invention adopts an innovative strategy, directly using the answer distribution bias to replace the output of this independent neural network branch. This improvement not only simplifies the processing flow but also reduces the demand for computational resources while maintaining performance. Specifically, in this invention, for each question type, the occurrence frequency of all possible answers is first counted, and the frequency of each answer is calculated based on this, forming an array B. i,type , representing the deviation in the distribution of answers, can be expressed as:

[0058] Among them B i,type The array represents the bias in the answer distribution for the i-th question category, a.j Let C be the number of the j-th candidate answer, C represent the total number of candidate answers, and A be the sum of the occurrences of all candidate answers.

[0059] S3. Input the question and image from the dataset in step S1 into two independent feature learning branches, namely the first and second feature learning branches. Within each branch, word embedding and feature extraction operations are performed on the question, and feature extraction is performed on the image to obtain question features Q and image features I, respectively. Then, through the dot product attention mechanism, a secondary feature I2 of the image is generated based on the question features Q and image features I, expressed as:

[0060] I2=softmax(linear(Q)·linear(I) T )

[0061]

[0062] Where `linear()` represents a fully connected linear layer applied to the problem features Q and image features I. `T` represents the transpose of the matrix. The function `softmax` is a widely used activation function in deep learning. u Let represent the value of the u-th element of x, which can transform the vector x into a probability distribution array that sums to 1. The secondary image feature I2 and the question feature Q are multiplied element-wise to obtain a secondary fusion feature. The secondary fusion feature in the first feature learning branch is denoted as f1, and the secondary fusion feature in the second feature learning branch is denoted as f2. f1 and f2 are fed into two classifiers, which map f1 and f2 to a one-dimensional probability array, where each element represents the probability that a candidate answer is true. p1 and p2 are summed using a weighted summation method to obtain the predicted probability p, expressed as:

[0063] p = αp1 + (1-α)p2

[0064] Where α is a weight parameter, it gradually decreases as the visual question answering model iterates. The decrease in α is as follows:

[0065] Where τ is a fixed hyperparameter controlling the rate of weight change; the larger τ is, the faster α decreases. In this embodiment, it is 1.5, used to control the transformation rate. T is the current iteration number. max It represents the maximum number of iterations.

[0066] S4. Compare the predicted probability p with the correct answer corresponding to the image to calculate the loss. Two methods are used to calculate the loss: binary cross-entropy (BCE) loss and LMH loss. The BCE loss is calculated as follows:

[0067]

[0068] Where d is the score corresponding to the correct answer, d′ is the score predicted by the model, and C is the number of candidate answers. k This represents the value of the k-th element of d. k ' represents the value of the k-th element of d';

[0069] The LMH loss is calculated as follows:

[0070] LMH(s,s′)=BCE(softmax(log(s)+g(f2)log(B) i,type ),s'))+R

[0071] R = ωH(softmax(g(f2))log(B) i,type ))

[0072] H(z)=∑ r z r log(z r )

[0073] Where s is the score corresponding to the correct answer, s′ is the score predicted by the model, g(·) is a learnable neural network layer, R is the penalty term, ω is a hyperparameter that adjusts the influence of the penalty term, and z r It is the value of the r-th element of z.

[0074] S5. The BCE loss and LMH loss are weighted and summed to obtain the final loss used to optimize the visual question answering model. The final loss is expressed as:

[0075] loss=αBCE(p,y)+(1-α)LMH(p,y)

[0076] Where y is the corresponding correct answer.

[0077] At the operational level, the BCE loss is responsible for maintaining the feature extractor's ability to represent mainstream categories, while the LMH loss is specifically optimized for categories with scarce data. The LMH loss modifies the BCE calculation formula by introducing a regularization term R. This ensures that while incorporating answer distribution bias, the g(f2) term in the LMH loss is not ignored. If the g(f2) term is reduced to zero, the LMH loss will be downgraded to the standard BCE loss. Furthermore, the hyperparameter α is designed to automatically decrease as training progresses, gradually transitioning the focus from feature learning to classification learning. Specifically, as α decreases, the model gradually shifts its focus from feature representation to classification accuracy. This strategy significantly improves model performance on the VQA-CPv2.0 dataset.

[0078] The combined BCE and LMH loss strategy employed in this invention not only overcomes the shortcomings of traditional rebalancing techniques in feature learning but also addresses the weak generalization ability of previous models when dealing with rare categories. This method enhances the model's ability to handle long-tailed distributed data and demonstrates more robust and effective performance in handling various types of problems, especially rare or atypical problem categories.

[0079] S6. Parameter Optimization. The gradients of the visual question-answering model's parameters are calculated using the final loss function. The parameters are then optimized using the Adam optimizer. In this embodiment, the automatic gradient calculation and the Adam optimizer provided by the deep learning framework PyTorch are used.

[0080] S7. Model Training. Repeat steps S2 to S6 for the specified number of training epochs to complete the training of the visual question answering model.

[0081] S8. Model Application. Input the actual captured image and its corresponding question into the trained visual question answering model, and the model will output the corresponding answer.

[0082] Table 1 shows the performance of different visual question answering methods on the VQA v2.0 dataset. Tables 2 and 3 show the hardware and software operating environment for this method.

[0083] Table 1. Comparison of the performance of various methods of the present invention on VQA-CP v2.0

[0084] method Basic Model all whether number VGQE UpDn 48.75 none none DLR UpDn 48.87 70.99 18.72 HINT UpDn 46.73 67.27 10.61 SCR UpDn 49.45 72.36 10.93 AttAlign UpDn 39.37 43.02 11.89 RUBi UpDn 44.23 67.05 17.48 AdvReg. UpDn 41.17 65.49 15.48 LM UpDn 48.78 72.78 14.61 LM-H UpDn 52.01 72.58 31.12 CF-VQA(SUM) UpDn 53.55 91.15 13.03 This method UpDn 55.60 82.45 39.76

[0085] On the VQA-CP v2 test set, our method demonstrates a comprehensive performance improvement over existing techniques, particularly in the "Yes / No" and "Digit" categories. This result validates our method's significant ability to effectively reduce language bias and more efficiently utilize image features. Specifically, in the "Digit" category, we achieved an accuracy of 39.76%, ranking first among all compared methods. Simultaneously, in the "Yes / No" category, we achieved an excellent second-place result with an accuracy of 82.45%, second only to the CF-VQA(SUM) model. This comparison further demonstrates the outstanding performance of our method.

[0086] Table 2. Hardware Environment Table

[0087]

[0088] Table 3. Software Environment Table

[0089] software Version operating system Ubuntu 18.04 LTS Python 3.7.13 PyTorch 1.8.0 Graphics card driver NVIDIA graphics card driver 510.108.03 CUDA 11.1.105 CUDNN 8.0.4

[0090] Example 2

[0091] This embodiment provides a method for overcoming prior influences in visual question answering through a cumulative learning strategy. The implementation process is as follows: Figure 3 As shown, the operation steps are largely the same as in Example 1. The difference lies in the setting of parameter τ in this example:

[0092] S1. Refer to step S1 in Example 1;

[0093] S2. Refer to step S2 in Example 1;

[0094] S3. Input the question and image from the dataset in step S1 into two independent feature learning branches, namely the first and second feature learning branches. Within each branch, word embedding and feature extraction operations are performed on the question, and feature extraction is performed on the image to obtain question features Q and image features I, respectively. Then, through the dot product attention mechanism, a secondary feature I2 of the image is generated based on the question features Q and image features I, expressed as:

[0095] I2=softmax(linear(Q)·linear(I) T )

[0096]

[0097] Where `linear` represents a fully connected linear layer applied to the problem features Q and image features I. `T` represents the transpose of the matrix. The function `softmax` is a widely used activation function in deep learning. u Let represent the value of the u-th element of x, which can transform the vector x into a probability distribution array that sums to 1. The secondary image feature I2 and the question feature Q are multiplied element-wise to obtain a secondary fusion feature. The secondary fusion feature in the first feature learning branch is denoted as f1, and the secondary fusion feature in the second feature learning branch is denoted as f2. f1 and f2 are fed into two classifiers, which map f1 and f2 to a one-dimensional probability array, where each element represents the probability that a candidate answer is true. p1 and p2 are summed using a weighted summation method to obtain the predicted probability p, expressed as:

[0098] p = αp1 + (1-α)p2

[0099] α is a weight parameter that gradually decreases as the visual question answering model iterates. The decrease in α is as follows:

[0100]

[0101] Where T is the current iteration number, T maxτ is the maximum number of iterations, and it is a fixed hyperparameter that controls the rate of change of weights. The larger the value of τ, the faster α decreases, which directly affects the smooth transition from feature learning to classification learning during training. To explore the impact of τ on model performance in depth, this embodiment selects different values ​​of τ for experimental comparison, specifically including 0.5, 0.8, 1, 1.4, 1.5, 1.8, and 2.

[0102] S4. Refer to step S4 in Example 1;

[0103] S5. Refer to step S5 in Example 1;

[0104] S6. Refer to step S6 in Example 1;

[0105] S7. Refer to step S7 in Example 1;

[0106] S8. Refer to step S8 in Example 1;

[0107] Table 4 shows the performance of this method on the VQA-CP v2.0 dataset with different values ​​of τ.

[0108] Table 4. The impact of the value of τ on the performance of this method

[0109] τ all whether number other 0.5 53.40 81.11 41.30 42.20 0.8 55.01 79.20 45.03 45.07 1 53.83 76.86 45.51 44.05 1.4 54.53 78.00 44.32 45.03 1.5 57.45 82.45 39.76 49.21 1.8 54.37 77.31 45.62 44.74 2 54.22 78.71 42.60 44.57

[0110] In this method, the model performance reaches its optimum when the hyperparameter τ is set to 1.5. Experimental results show that this method can maintain a high performance level consistently within the τ value range of 0.5 to 2. This finding confirms that the method has a high tolerance for the choice of τ value, that is, it can achieve excellent performance without excessive fine-tuning within a wide range of τ values. This characteristic highlights the flexibility and ease of use of this method in practical applications, making it easier to adjust and optimize for different visual question answering tasks, thereby effectively reducing the prior influence in visual question answering and utilizing image features to provide accurate question answers.

[0111] The above embodiments are preferred implementations of this method, but the implementation of this invention is not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of this invention should be considered equivalent substitutions and are included within the protection scope of this invention.

Claims

1. A method for overcoming prior influences in visual question answering through a cumulative learning strategy, characterized in that, The method includes the following steps: S1. Constructing the dataset: Collect images taken in the real world along with their related questions and answers to form question-image-answer triples. Aggregate the triples into a dataset specifically designed for visual question answering. The dataset uses a uniform sampling strategy to select samples. S2. Data Processing: Perform statistical analysis on the question-answer pairs within the dataset to calculate the answer distribution deviation. In step S2, for each question category, first calculate the occurrence frequency of all candidate answers within that category, then determine the frequency of each answer, and use this to form the answer distribution deviation array B. i,type , represented as: B i,type The array represents the bias in the answer distribution for the i-th question category, a. j Let C be the number of the j-th candidate answer, C represent the total number of candidate answers, and A be the sum of the occurrences of all candidate answers. S3. Feature Learning: Input the questions and images in the dataset into two independent first and second feature learning branches. Each branch performs word embedding and feature extraction on the questions and extracts features from the images, obtaining question features Q and image features I respectively. Then, input the question features Q and image features I into the dot product attention module to generate secondary features I2 of the image. The secondary features I2 of the image and the question features Q are multiplied element-wise to obtain secondary fusion features. The secondary fusion features in the first feature learning branch are denoted as f1, and the secondary fusion features in the second feature learning branch are denoted as f2. f1 and f2 are fed into two classifiers respectively. The two classifiers map f1 and f2 to a one-dimensional probability array, where each value in the array represents the probability that a candidate answer is true. The output probabilities of the two classifiers are denoted as p1 and p2 respectively. S4. Loss Calculation and Fusion: The predicted probability p is obtained by weighted summation of the classifier output probabilities p1 and p2. The predicted probability p is compared with the correct answer corresponding to the image to calculate the binary cross-entropy (BCE) loss and LMH loss respectively. The BCE loss and LMH loss are weighted summation to obtain the final loss used to optimize the visual question answering model. S5. Training parameter adjustment: During the training process, the proportion of BCE loss and LMH loss in the final loss is controlled by adjusting the weight parameter α, and the proportion of p1 and p2 in the prediction probability p is also controlled. S6. Parameter optimization: Calculate the gradient of the parameters of the visual question answering model using the final loss and optimize the parameters using the Adam optimizer. S7. Model Training: Repeat steps S2 to S6 for the specified number of training epochs to complete the training of the visual question answering model. S8. Model Application: Input the actual captured images and their corresponding questions into the trained visual question answering model, and the model will output the corresponding answers.

2. The method for overcoming prior influences in visual question answering through a cumulative learning strategy according to claim 1, characterized in that, In step S1, the data in the dataset is uniformly sampled, and the probability P of the i-th question category being sampled is... i Represented as: Where N i Let represent the number of samples of the i-th question category in the training set, and n represent the total number of question categories.

3. The method for overcoming prior influences in visual question answering through a cumulative learning strategy according to claim 1, characterized in that, The dot product attention module in step S3 is implemented through the following attention mechanism: I2=softmax(linear(Q)·linear(I) T ) Where linear represents a fully connected linear layer applied to problem features Q and image features I, () T The function `softmax()` represents the transpose of a matrix. It is an activation function used in deep learning. u Let x represent the value of the u-th element. The function softmax() converts the vector x into a probability distribution array that sums to 1.

4. The method for overcoming prior influences in visual question answering through a cumulative learning strategy according to claim 1, characterized in that, The formula for calculating the final loss is as follows: loss=αBCE(p,y)+(1-α)LMH(p,y) Where y is the corresponding correct answer.

5. The method for overcoming prior influences in visual question answering through a cumulative learning strategy according to claim 1, characterized in that, The formula for calculating the prediction probability p is as follows: p = αp1 + (1-α)p2.

6. The method for overcoming prior influences in visual question answering through a cumulative learning strategy according to claim 1, characterized in that, The update and iteration formula for the weight parameter α is as follows: Where τ is a fixed hyperparameter controlling the rate of change of the weights; the larger τ is, the faster α decreases. T is the current iteration number. max It represents the maximum number of iterations.

7. The method for overcoming prior influences in visual question answering through a cumulative learning strategy according to claim 1, characterized in that, The image feature extraction process in step S3 is as follows: the input image is extracted using a convolutional neural network (CNN), and the text feature extraction process is as follows: first, word embedding is performed using Word2Vec or GloVe, and then recurrent neural networks (RNN) or Transformer are used for extraction.

Images