Multi-modal pre-training data filtering method based on multi-modal large model
By classifying and calculating multimodal pretrained data, the noise data is filtered out, and the model pays attention to key information is improved, and the problem of noise and hallucination phenomena in the training process of multimodal pretrained models is solved. Improved model performance.
Patent Information
- Application Number
- CN202411437441.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-15
- Publication Date
- 2025-02-11
AI Technical Summary
The existing multimodal pre-trained models have a lot of noise during the training process, the model pays little attention to key multimodal information, and there is a hallucination of training models.
Using multimodal pretrained data filtering method based on multimodal large model, the reference model is adjusted using standard visual instructions, the reference loss of text markers is calculated, and the data is calculated according to the loss value. Sorting and filtering to obtain high-quality multimodal pre-training data.
Significantly reduce noise during training, improve the model's attention to key multimodal information, reduce the illusion of training models, and improve the overall performance of multimodal pre-trained models.
Smart Images
Figure CN119415827A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a data filtering method, and more particularly to a multimodal pre-training data filtering method based on a multimodal large model. Background Art
[0002] Multimodal pre-training models have been a hot research topic in the field of artificial intelligence in recent years. By combining data from different modalities (such as text, images, videos, audio, etc.), the model can learn cross-modal semantic associations and common feature representations. This model has demonstrated excellent performance in multiple downstream tasks, such as joint understanding of images and text, visual question answering, image description generation, etc. However, during model pre-training, the received data of different modalities is directly processed. The direct processing method will lead to more noise in training, the model will not pay much attention to key multimodal information, and the training model will have more hallucinations. Summary of the invention
[0003] In view of the deficiencies in the prior art, an object of the present invention is to provide a multimodal pre-training data filtering method based on a multimodal large model that can avoid one or more of the above-mentioned problems.
[0004] To achieve the above object, the present invention provides the following technical solution: a multimodal pre-training data filtering method based on a multimodal large model, characterized in that it comprises the following steps:
[0005] Step 1: obtain multimodal pre-training data for training models and classify the multimodal pre-training data; Step 2: use standard visual instructions to adjust the reference model trained on external high-quality multimodal data;
[0006] Step 3: Calculate the reference loss of all text tags based on the log probability obtained by the reference model RM;
[0007] Step 4: sort and filter the multimodal pre-training data obtained in step 1 by using the reference loss calculated in step 3 to obtain the final high-quality multimodal pre-training data.
[0008] As a further improvement of the present invention, the method of classifying the multimodal pre-training data in step one is to divide the multimodal pre-training data into three types, namely, text that is highly relevant to the image, text that is less relevant to the image, and text that conflicts with the image content.
[0009] As a further improvement of the present invention, the text highly related to the image includes entities, quantities, colors and texts, and these tags directly correspond to the image information and are used for multimodal alignment.
[0010] As a further improvement of the present invention, the text with low correlation with the image includes transition words or content that can be inferred from the previous text, and these tags are used to train the pure text function of the MLLM.
[0011] As a further improvement of the present invention, the text conflicting with the image content includes tags inconsistent with the image information, and this tag provides misleading information and has a negative impact on the multi-modal alignment process.
[0012] As a further improvement of the present invention, the reference loss in step three is calculated by the following formula: L_{RM}(x_i) = -\logP(x_i|x_{<i},v_{<i})
[0013] Where x_i is the i-th token in the sequence, and x_{<i} and v_{<i} are the text and visual tokens before i.
[0014] As a further improvement of the present invention, the specific method of sorting and filtering data in step four is as follows: It is expected that the text highly relevant to the image has a high L_{\theta} and a low L_{\text{RM}}, so L_{\Delta} is also high. It is expected that the text with low correlation with the image has low L_{\theta} and L_{\text{RM}}. It is expected that the text conflicting with the image content has a high L_{\text{RM}}. Then, by distinguishing the token types through L_{\Delta}, and then selectively retaining the tokens with high L_{\Delta}, the focus of training can be effectively shifted to the data of the text highly relevant to the image, while filtering out the data of the text conflicting with the image content.
[0015] As a further improvement of the present invention, in step four, the tokens are prioritized according to the L_{\Delta} value in the batch, and only the tokens in the top k percentile are focused on for loss calculation.
[0016] The beneficial effects of the present invention are as follows: By using the Rho-1 technology for high-quality data selection to improve the performance of the multi-modal pre-training model, the method of the present invention will carefully examine the data set to find data samples with the highest representativeness and training value. Compared with the multi-modal pre-training in the prior art without data screening, the method of the present invention can significantly reduce the noise in training, improve the model's attention to key multi-modal information, and reduce the hallucination phenomenon of the training model. Brief Description of the Drawings
[0017] Figure 1 It is a flowchart of the multi-modal pre-training data filtering method based on the multi-modal large model of the present invention. Detailed Embodiments
[0018] The present invention will be further described in detail below with reference to the embodiments given in the accompanying drawings.
[0019] Referring to Figure 1 As shown, a multi-modal pre-training data filtering method based on a multi-modal large model in this embodiment includes the following steps:
[0020] Step 1: Obtain the multi-modal pre-training data of the training model and classify the multi-modal pre-training data. In this embodiment, the starting point of classification is that all tokens in the corpus are not equally important for multi-modal training, and the tokens in the multi-modal data can be divided into three different types:
[0021] Type 1: Texts highly related to images: entities (e.g., people, animals, objects), quantities, colors, texts, etc. These tokens directly correspond to image information and are crucial for multi-modal alignment.
[0022] Type 2: Texts less related to images: transition words or content that can be inferred from the previous text. These tokens are mainly used to train the pure text functions of the MLLM.
[0023] Type 3: Texts conflicting with image content: These tokens are inconsistent with the image information, may provide misleading information, and have a negative impact on the multi-modal alignment process.
[0024] Step 2: Use standard visual instructions to adjust and train the reference model on external high-quality multi-modal data;
[0025] Step 3: Calculate the reference loss of all text tokens according to the log probabilities obtained by the reference model RM, specifically as follows:
[0026] L_{RM}(x_i) = -\logP(x_i|x_{<i},v_{<i})
[0027] where x_i is the i-th token in the sequence, x_{<i} and v_{<i} are the text and visual tokens before i, and then calculate the difference between the loss functions of the current model and the reference model.
[0028] L_{\Delta}(x_i) = L_{\theta}(x_i) - L_{\text{RM}}(x_i)
[0029] where L_{\theta}(x_i) = -\logP(x_i|x_{<i},v_{<i};\theta), which is the standard causal language modeling loss of the model being trained.
[0030] Step 4: Sort and filter the multimodal pre-training data obtained in step 1 by the reference loss calculated in step 3 to obtain the final high-quality multimodal pre-training data. Specifically, in Llava-like fine-tuning, the basic language model has shown strong language capabilities. We expect type 1 tags to have higher L_{\theta} and lower L_{\text{RM}}, so L_{\Delta} is also higher. In contrast, we expect type 2 tags to have lower L_{\theta} and L_{\text{RM}} because no visual input is required to predict these tags. Type 3 tags are expected to have higher L_{\text{RM}} due to the contradiction with the visual environment. Therefore, L_{\Delta} is a reliable indicator for distinguishing tag types. By selectively retaining tags with high L_{\Delta}, the focus of training can be effectively shifted to type 1 data while filtering out type 3 data.
[0031] Finally, we prioritize the tokens according to their L_{\Delta} values in the batch and focus only on the top k percentile tokens for loss calculation.
[0032] The following examples are provided in this embodiment:
[0033] First, L_{\text{RM}} of the multimodal corpus is calculated offline and merged into the batch data so that L_{\Delta} can be calculated in real time during training. It is worth noting that loading L_{\text{RM}} and ranking L_{\Delta} have little impact on the overall computation time, ensuring minimal impact on the training process. We set k to 60%, that is, we only train the 60% of the data with the highest quality, and the remaining 40% of low-quality data has no effect on the model. Finally, the high-quality multimodal pre-training data is used to train the training model.
[0034] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.
Claims
1. A multimodal pre-training data filtering method based on a multimodal large model, characterized in that: It includes the following steps: Step 1, obtain multi-modal pre-training data for the training model and classify the multi-modal pre-training data; Step 2, use standard visual instructions to adjust and train the reference model on external high-quality multi-modal data; Step 3, calculate the reference loss of all text tokens according to the log probabilities obtained by the reference model RM; Step 4, sort and filter the multi-modal pre-training data obtained in Step 1 through the reference loss calculated in Step 3 to obtain the final high-quality multi-modal pre-training data.
2. The multimodal pre-training data filtering method based on a multimodal large model according to claim 1, characterized in that: The method of classifying the multi-modal pre-training data in Step 1 is to divide the multi-modal pre-training data into three types, namely text highly relevant to the image, text with low relevance to the image, and text conflicting with the image content.
3. The multimodal pre-training data filtering method based on a multimodal large model according to claim 2 is characterized in that: The text highly relevant to the image includes entities, quantities, colors, and text, and these tokens directly correspond to the image information and are used for multi-modal alignment.
4. The multimodal pre-training data filtering method based on a multimodal large model according to claim 3 is characterized in that: The text with low relevance to the image includes transition words or content that can be inferred from the previous text, and these tokens are used to train the pure text function of the MLLM.
5. The multimodal pre-training data filtering method based on a multimodal large model according to claim 4 is characterized in that: The text conflicting with the image content includes tokens inconsistent with the image information, and this token provides misleading information and has a negative impact on the multi-modal alignment process.
6. The multimodal pre-training data filtering method based on a multimodal large model according to any one of claims 2 to 5, characterized in that: The reference loss in Step 3 is calculated by the following formula: L_{RM}(x_i)=-\log P(x_i|x_{<i},v_{<i}) where x_i is the i-th token in the sequence, x_{<i} and v_{<i} are the text and visual tokens before i, and then calculate the difference between the loss functions of the current model and the reference model: L_{\Delta}(x_i)=L_{\theta}(x_i)-L_{\text{RM}}(x_i) where L_{\theta}(x_i)=-\log P(x_i|x_{<i},v_{<i};\theta), which is the standard causal language modeling loss of the model being trained.
7. The multimodal pre-training data filtering method based on a multimodal large model according to claim 6, characterized in that: The specific method of sorting and filtering data in Step 4 is as follows: It is expected that the text highly relevant to the image has a higher L_{\theta} and a lower L_{\text{RM}}, so L_{\Delta} is also higher. It is expected that the text with low relevance to the image has lower L_{\theta} and L_{\text{RM}}. It is expected that the text conflicting with the image content has a higher L_{\text{RM}}. Then, the token types are distinguished by L_{\Delta}, and by selectively retaining the tokens with high L_{\Delta}, the focus of training can be effectively shifted to the data of the text highly relevant to the image, while filtering out the data of the text conflicting with the image content.
8. The multimodal pre-training data filtering method based on a multimodal large model according to claim 7, characterized in that: In Step 4, the tokens are prioritized according to the L_{\Delta} values in the batch, and only the tokens in the top k percentiles are focused on for loss calculation.