Method, apparatus, storage medium, system, and computer program product for enhancing the inference ability of a multimodal inference model

Abstract

Disclosed are a method, an apparatus, a storage medium, a system, and a computer program product for enhancing the inference ability of a multimodal inference model. The method includes: constructing a multi-source data set; determining a decision set for solving problems; for each question-answer pair, performing the following processing: inputting the decision set and the current question in the current question-answer pair into the multimodal inference model to obtain first state transition data of the current question, where the first state transition data is logical data for inferring a predicted answer to the current question based on at least one of multiple decisions; determining a first data state of the first state transition data based on the true answer in the current question-answer pair, where the first data state indicates that the first state transition data is correct data or incorrect data; in response to all question-answer pairs having completed the above processing, training the multimodal inference model using all correct data and the question-answer pairs corresponding to the correct data to obtain a first multimodal inference model with enhanced inference ability.

Description

Technical Field

[0001] The present disclosure generally relates to the field of artificial intelligence technology, and more specifically, to methods, devices, storage media, systems, and computer program products for enhancing the reasoning ability of multimodal reasoning models. Background Art

[0002] With the rapid development of artificial intelligence technology, multimodal large models have been widely used in visual question answering (VQA) tasks. The VQA task requires the large model to comprehensively process image and text information to generate accurate answers. However, the existing technologies still face many challenges and limitations in dealing with complex reasoning tasks. For example, when facing complex questions, existing multimodal large models often show the defect of insufficient reasoning ability. Because traditional large models usually can only handle simple visual queries, it is difficult to generate a coherent and logically consistent reasoning chain for complex questions that require deep reasoning, resulting in the accuracy and reliability of the answers being affected. In this case, the reasoning process of the large model lacks transparency and interpretability, making it difficult to meet the needs of practical applications.

[0003] Currently, the Chain-of-Thought (CoT) reasoning method has made some progress in enhancing the reasoning ability of large models, but there are still many challenges in its application in a multimodal environment. Especially in the zero-shot CoT method, the large model needs to reason through prompts without explicit examples, which poses higher requirements for the logical consistency and reasoning quality of the large model. However, existing CoT methods often have difficulty maintaining the logical consistency and high quality of the reasoning process when dealing with diverse and complex tasks, resulting in unreliable reasoning results. In addition, existing multimodal large models usually lack an adaptive reasoning strategy when dealing with visual question answering tasks. For example, when facing different types of questions, the large model needs to be able to dynamically adjust the reasoning path to adapt to the needs of different tasks. However, the large models in the existing technologies often adopt a fixed reasoning path, lacking flexibility and adaptability, and it is difficult to handle diverse task scenarios. This fixed-path reasoning method limits the performance of the model in complex tasks and cannot fully exert the potential of the multimodal model. Summary of the Invention

[0004] Embodiments of the present disclosure provide a method, device, storage medium, system, and computer program product for enhancing the reasoning ability of a multimodal reasoning model, which can effectively solve the problems that the existing technology model has insufficient reasoning ability and is difficult to handle diverse task scenarios.

[0005] In one general aspect, a method for enhancing the inference ability of a multimodal inference model is provided, including: constructing a multi-source data set, where the multi-source data set contains question-and-answer pairs of multiple modalities, and each question-and-answer pair contains a question and the true answer to the question; determining a decision set for solving the question, where the decision set contains multiple decisions, and the estimated answer to any question can be determined based on at least one of the multiple decisions; for each question-and-answer pair of the multiple modalities, perform the following processing: input the decision set and the current question in the current question-and-answer pair into the multimodal inference model to obtain first state transition data of the current question, where the first state transition data is logical data for inferring the estimated answer to the current question based on at least one of the multiple decisions; based on the true answer in the current question-and-answer pair, determine the first data state of the first state transition data, where the first data state indicates whether the first state transition data is correct data or incorrect data; in response to all question-and-answer pairs having completed the above processing, use all the correct data and the question-and-answer pairs corresponding to the correct data to train the multimodal inference model to obtain a first multimodal inference model with enhanced inference ability.

[0006] Optionally, the processing performed for each question-and-answer pair of the multiple modalities further includes: in response to the first data state indicating that the first state transition data is correct data, input the first state transition data into the multimodal evaluation model to obtain the state score and scoring reason of each state data in the first state transition data; in response to the average state score of all state data in the first state transition data being greater than or equal to a preset threshold, determine that the first state transition data is high-quality data; where, after obtaining the first multimodal inference model with enhanced inference ability, it further includes: using all the high-quality data and the question-and-answer pairs corresponding to the high-quality data to train the first multimodal inference model to obtain a second multimodal inference model with enhanced inference ability.

[0007] Optionally, the processing performed for each question-and-answer pair of the multiple modalities further includes: in response to the average state score of all state data in the first state transition data being less than the preset threshold, input the first state transition data, the state score, and the scoring reason into the multimodal correction model to obtain second state transition data; in response to the second state transition data being high-quality data, determine the first state transition data and the second state transition data as a first training sample pair; where, after obtaining the second multimodal inference model with enhanced inference ability, it further includes: using all the first training sample pairs to train the second multimodal inference model to obtain a third multimodal inference model with enhanced inference ability.

[0008] Optionally, the processing performed for each question-answer pair among multiple-modal question-answer pairs further includes: in response to the first data status indicating that the first state transition data is error data, inputting the first state transition data into a multi-modal correction model to obtain third state transition data; in response to the third state transition data being high-quality data, determining the first state transition data and the third state transition data as a second training sample pair; wherein, after obtaining the second multi-modal inference model with enhanced inference ability, it further includes: using all the second training sample pairs to train the second multi-modal inference model to obtain a fourth multi-modal inference model with enhanced inference ability.

[0009] Optionally, the processing performed for each question-answer pair among multiple-modal question-answer pairs further includes: in response to the average status score of all status data in the first state transition data being greater than or equal to a preset threshold, inputting the current question into the second multi-modal inference model to obtain fourth state transition data; in response to the fourth state transition data being error data, determining the fourth state transition data and the first state transition data as a third training sample pair; wherein, after obtaining the second multi-modal inference model with enhanced inference ability, it further includes: using all the third training sample pairs to train the second multi-modal inference model to obtain a fifth multi-modal inference model with enhanced inference ability.

[0010] Optionally, determining the first data status of the first state transition data based on the true answer in the current question-answer pair includes: inputting the first state transition data and the true answer in the current question-answer pair into a multi-modal evaluation model to obtain the first data status of the first state transition data.

[0011] In another general aspect, there is provided an apparatus for enhancing the inference ability of a multi-modal inference model, including: a construction unit configured to construct a multi-source data set, where the multi-source data set includes multiple-modal question-answer pairs, and each question-answer pair includes a question and the true answer to the question; a determination unit configured to determine a decision set for solving the question, where the decision set includes multiple decisions, and the estimated answer to any question can be determined based on at least one of the multiple decisions; an execution unit configured to perform the following processing for each question-answer pair among multiple-modal question-answer pairs: inputting the decision set and the current question in the current question-answer pair into the multi-modal inference model to obtain first state transition data of the current question, where the first state transition data is logical data for inferring the estimated answer to the current question based on at least one of the multiple decisions; determining the first data status of the first state transition data based on the true answer in the current question-answer pair, where the first data status indicates that the first state transition data is correct data or error data; a training unit configured to, in response to all the question-answer pairs having completed the above processing, use all the correct data and the question-answer pairs corresponding to the correct data to train the multi-modal inference model to obtain a first multi-modal inference model with enhanced inference ability.

[0012] Optionally, the execution unit is further configured to, in response to the first data status indicating that the first state transition data is correct data, input the first state transition data into the multi-modal evaluation model to obtain the status scores and scoring reasons of each status data in the first state transition data; in response to the average status score of all status data in the first state transition data being greater than or equal to a preset threshold, determine that the first state transition data is high-quality data; wherein, the training unit is further configured to, after obtaining the first multi-modal inference model with enhanced inference ability, use all high-quality data and the question-and-answer pairs corresponding to the high-quality data to train the first multi-modal inference model to obtain the second multi-modal inference model with enhanced inference ability.

[0013] Optionally, the execution unit is further configured to, in response to the average status score of all status data in the first state transition data being less than the preset threshold, input the first state transition data, the status scores and the scoring reasons into the multi-modal correction model to obtain the second state transition data; in response to the second state transition data being high-quality data, determine the first state transition data and the second state transition data as the first training sample pair; wherein, the training unit is further configured to, after obtaining the second multi-modal inference model with enhanced inference ability, use all the first training sample pairs to train the second multi-modal inference model to obtain the third multi-modal inference model with enhanced inference ability.

[0014] Optionally, the execution unit is further configured to, for each question-and-answer pair among the multi-modal question-and-answer pairs, the processing further includes: in response to the first data status indicating that the first state transition data is incorrect data, input the first state transition data into the multi-modal correction model to obtain the third state transition data; in response to the third state transition data being high-quality data, determine the first state transition data and the third state transition data as the second training sample pair; wherein, the training unit is further configured to, after obtaining the second multi-modal inference model with enhanced inference ability, use all the second training sample pairs to train the second multi-modal inference model to obtain the fourth multi-modal inference model with enhanced inference ability.

[0015] Optionally, the execution unit is further configured to, for each question-and-answer pair among the multi-modal question-and-answer pairs, the processing further includes: in response to the average status score of all status data in the first state transition data being greater than or equal to the preset threshold, input the current question into the second multi-modal inference model to obtain the fourth state transition data; in response to the fourth state transition data being incorrect data, determine the fourth state transition data and the first state transition data as the third training sample pair; wherein, the training unit is further configured to, after obtaining the second multi-modal inference model with enhanced inference ability, use all the third training sample pairs to train the second multi-modal inference model to obtain the fifth multi-modal inference model with enhanced inference ability.

[0016] Optionally, an execution unit is configured to input the first state transition data and the true answer in the current question-and-answer pair into a multi-modal evaluation model to obtain a first data state of the first state transition data.

[0017] In another general aspect, there is provided a computer-readable storage medium storing instructions, which, when executed by at least one computing device, cause the at least one computing device to perform the method for enhancing the inference ability of a model as described above in any one of the preceding paragraphs.

[0018] In another general aspect, there is provided a system including at least one computing device and at least one storage device storing instructions, which, when executed by the at least one computing device, cause the at least one computing device to perform the method for enhancing the inference ability of a model as described above in any one of the preceding paragraphs.

[0019] In another general aspect, there is provided a computer program product including computer instructions that implement the method for enhancing the inference ability of a model as described above when executed by a processor.

[0020] According to the method, apparatus, storage medium, system, and computer program product for enhancing the inference ability of a multi-modal inference model according to an embodiment of the present disclosure, a decision set including multiple decisions is introduced, enabling the model to adaptively adjust the inference path, thereby flexibly meeting different types of task requirements, overcoming the limitations of traditional models that adopt a fixed inference path, enabling the model to effectively generate a coherent and logically consistent inference chain, and ensuring the accuracy and reliability of the answers when the model processes complex problems. Furthermore, by using the correct data output by the multi-modal inference model and its corresponding question-and-answer pairs to train the multi-modal model, the inference ability of the multi-modal large model in visual question-and-answer tasks can be significantly improved. The present disclosure enhances the adaptability and expressiveness of the model in diverse task scenarios, providing more solid technical support for the practical application of the multi-modal large model. Therefore, through the present disclosure, the problem that the existing technology model has insufficient inference ability and is difficult to handle diverse task scenarios can be effectively solved.

[0021] Additional aspects and / or advantages of the general concept of the present disclosure will be set forth in part in the description that follows, and in part will be obvious from the description, or may be learned by practice of the general concept of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Through the following description with reference to the drawings showing embodiments, the above and other objects and features of the embodiments of the present disclosure will become more apparent, wherein:

[0023] Figure 1 is a flowchart showing the method for enhancing the inference ability of a multi-modal inference model according to an embodiment of the present disclosure;

[0024] Figure 2 is a schematic diagram of the overall process of a method for enhancing the reasoning ability of an enhanced multimodal reasoning model according to an embodiment of the present disclosure;

[0025] Figure 3 is a schematic diagram of the overall structure of a method for enhancing the reasoning ability of an enhanced multimodal reasoning model according to an embodiment of the present disclosure;

[0026] Figure 4 is a comparison diagram of the reasoning effects of the Llama-CoMMoD and Llama-CoMMoD models according to an embodiment of the present disclosure;

[0027] Figure 5 is a comparison diagram of the reasoning effects of two models that do not apply the method of the present disclosure and a model that applies the method of the present disclosure according to an embodiment of the present disclosure;

[0028] Figure 6 is a block diagram of an apparatus for enhancing the reasoning ability of an enhanced multimodal reasoning model according to an embodiment of the present disclosure. Detailed implementation manners

[0029] The following detailed implementation manners are provided to assist the reader in obtaining a comprehensive understanding of the methods, devices, and / or systems described herein. However, various changes, modifications, and equivalents of the methods, devices, and / or systems described herein will be apparent after understanding the disclosure of the present application. For example, the order of operations described herein is merely illustrative and is not limited to those set forth herein, but may be changed as will be apparent after understanding the disclosure of the present application, except for operations that must occur in a specific order. In addition, descriptions of features known in the art may be omitted for greater clarity and conciseness.

[0030] The features described herein may be implemented in different forms and should not be construed as limited to the examples described herein. On the contrary, the examples described herein are provided only to illustrate some of the many feasible ways of implementing the methods, devices, and / or systems described herein, which will be apparent after understanding the disclosure of the present application.

[0031] As used herein, the term "and / or" includes any one of the associated listed items and any combination of any two or more of them.

[0032] Although terms such as "first", "second", and "third" may be used herein to describe various components, elements, regions, layers, or sections, these components, elements, regions, layers, or sections should not be limited by these terms. Instead, these terms are only used to distinguish one component, element, region, layer, or section from another. Thus, a first component, first element, first region, first layer, or first section as referred to in the examples described herein may also be referred to as a second component, second element, second region, second layer, or second section without departing from the teachings of the examples.

[0033] In the specification, when an element (such as a layer, region, or substrate) is described as being "on", "connected to", or "coupled to" another element, the element may be directly "on", directly "connected to", or "coupled to" the other element, or there may be one or more other elements therebetween. In contrast, when an element is described as being "directly on", "directly connected to", or "directly coupled to" another element, there may be no other elements therebetween.

[0034] The terms used herein are only for describing various examples and are not intended to limit the disclosure. Unless the context clearly indicates otherwise, the singular forms are also intended to include the plural forms. The terms "comprising", "including", and "having" specify the presence of the stated features, quantities, operations, components, elements, and / or combinations thereof, but do not preclude the presence or addition of one or more other features, quantities, operations, components, elements, and / or combinations thereof.

[0035] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains after understanding this disclosure. Unless explicitly defined herein, terms (such as those defined in a general dictionary) should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and should not be interpreted in an idealized or overly formal manner.

[0036] Furthermore, in the description of the examples, when a detailed description of related structures or functions that are considered to be well-known would cause an ambiguous interpretation of the present disclosure, such detailed descriptions will be omitted.

[0037] The method and apparatus for enhancing the reasoning ability of the enhanced multimodal reasoning model of the present disclosure will be described in detail below with reference to the accompanying drawings.

[0038] The present disclosure proposes a method for enhancing the reasoning ability of an enhanced multimodal reasoning model, Figure 1 is a flowchart showing the method for enhancing the reasoning ability of the enhanced multimodal reasoning model according to an embodiment of the present disclosure. Refer to Figure 1, the method for enhancing the reasoning ability of the enhanced multi-modal reasoning model includes the following steps:

[0039] In step S101, a multi-source data set is constructed, where the multi-source data set contains question-and-answer pairs of multiple modalities, and each question-and-answer pair contains a question and the true answer to the question.

[0040] As an example, the question-and-answer pairs in the above multi-source data set can be sourced from a variety of open-source data sets, covering multi-modal question-and-answer pairs for various types of multi-modal question-and-answer tasks. Among them, the various types can include, but are not limited to: data questions, general knowledge, table questions, and comprehensive questions, etc. The above multi-modal question-and-answer pairs generally contain images and text, which are not limited in this disclosure. The form of the above question-and-answer pairs can include multiple-choice questions, fill-in-the-blank questions, and subjective short-answer questions, which are also not limited in this disclosure. It should be noted that the question options of the above multiple-choice questions can be randomly arranged to create new question-and-answer pairs, enabling the model to reason about the same question from different perspectives to obtain the correct answer.

[0041] As an example, the above open-source data sets can include, but are not limited to: ChartQA, DocVQA, TextVQA, InfoVQA, ScienceQA, Geometry3K, GeoQA, Super-CLEVR, and MathV360K. Among them, ChartQA, DocVQA, TextVQA, InfoVQA, and ScienceQA focus on simple visual tasks, with a total of approximately 18,200 question-and-answer pairs; Geometry3K, GeoQA, and Super-CLEVR are for complex reasoning tasks; MathV360K is a multi-source data set containing simple visual tasks and complex reasoning tasks. By constructing a multi-source data set through the above diverse data sets, it is ensured that this disclosure is robust and capable of handling a wide range of multi-modal question-and-answer tasks.

[0042] In step S102, a decision set for solving the problem is determined, where the decision set contains multiple decisions, and the estimated answer to any question can be determined based on at least one of the multiple decisions.

[0043] As an example, the above decision set can make multiple decisions, and the estimated answer to any question can be determined based on at least one of the multiple decisions. For example, the above decision set can include twelve different decisions, and these twelve decisions can be systematically divided into four components: problem understanding, problem solving, verification, and answering. Each component reflects a key stage of human cognitive processing, thereby enhancing the model's ability to process and respond to complex queries. This structured decision set ensures that each stage of the problem reasoning logic is thoroughly addressed, promoting a more detailed and accurate response to complex problems. However, the present disclosure does not limit this.

[0044] As an example, the above problem understanding stage can include decisions such as identification, analysis, research, and formalization. These decisions focus on the initial understanding and identification of text and images in the question-and-answer pair, laying a foundation for more complex reasoning tasks. The above problem solving stage can include decisions such as decomposition, planning, solving sub-problems, and solving parent problems. These decisions help the model gradually break down complex problems and find solutions. The above verification stage includes decisions such as re-examination, verification, and review. These decisions allow the model to self-check, verify, and correct its reasoning process. The above answering stage includes one decision, which is to give the final answer to the question. The model synthesizes the entire reasoning process in this answering stage and outputs the answer according to the given paradigm. It should be noted that the decisions included in the above stages are not limited to this, and the required decisions can be set according to needs.

[0045] In step S103, for each question-and-answer pair in the multiple-modal question-and-answer pairs, the following processing is performed: input the decision set and the current question in the current question-and-answer pair into the multi-modal reasoning model to obtain the first state transition data of the current question, where the first state transition data is logical data for reasoning the estimated answer to the current question based on at least one of the multiple decisions; based on the true answer in the current question-and-answer pair, determine the first data state of the first state transition data, where the first data state indicates that the first state transition data is correct data or incorrect data.

[0046] As an example, the above decision set can be integrated into the prompt of the multi-modal reasoning model to guide the multi-modal reasoning model to autonomously select appropriate decisions from the decision set at each state, so that the multi-modal reasoning model generates state transition data that is both coherent and contextually relevant.

[0047] Specifically, it can be expressed in the following way: "Describe each decision function and inform the model that any one of the above can be adopted at each state of solving the problem" to integrate into the prompt , enabling it to utilize the above multi-source datasets; for the images and text in each question-answer pair, and can be used to adaptively generate state transition data , and this process can be expressed as:

[0048]

[0049] wherein, represents the number of question-answer pairs, represents the current question-answer pair.

[0050] It should be noted that the above multi-modal inference model is a large model, and its basic structure adopts GPT-4o, which is not limited in this disclosure.

[0051] As an example, after obtaining the state transition data of the current question, it can be compared with the true answer in the current question-answer pair to determine whether the first state transition data is correct or incorrect. The comparison between the state transition data and the true answer can be performed manually or by using a model, which is not limited in this disclosure.

[0052] According to the embodiments of the present disclosure, the first state transition data and the true answer in the current question-answer pair can be input into the multi-modal evaluation model to obtain the first data state of the first state transition data. Through this embodiment, by introducing the multi-modal evaluation model, the correctness of the first state data can be accurately evaluated, thereby obtaining an accurate first data state.

[0053] As an example, the above multi-modal evaluation model can also be a large model, such as Qwen2-VL-7B, and the evaluation model can be pre-trained, which is not limited in this disclosure.

[0054] Specifically, the first state transition data and the true answer in the current question-answer pair can be input into the multi-modal evaluation model, and the multi-modal evaluation model will compare with the true answer in the previous question-answer pair and output the state data and , wherein, indicates that the first state transition data is correct data, indicates that the first state transition data is incorrect data, and it can be expressed as follows:

[0055]

[0056] wherein, represents the prompt of the multi-modal evaluation model, represents the number of question-answer pairs, Indicates the current Q&A pair.

[0057] In step S104, in response to all Q&A pairs having completed the above processing, the multi-modal inference model is trained using all correct data and the Q&A pairs corresponding to the correct data to obtain a first multi-modal inference model with enhanced inference ability.

[0058] According to an embodiment of the present disclosure, the processing performed for each Q&A pair among multiple-modal Q&A pairs may further include: in response to the first data status indicating that the first state transition data is correct data, inputting the first state transition data into the multi-modal evaluation model to obtain the state scores and scoring reasons for each state data in the first state transition data; in response to the average state score of all state data in the first state transition data being greater than or equal to a preset threshold, determining that the first state transition data is high-quality data; wherein, after obtaining the first multi-modal inference model with enhanced inference ability, the first multi-modal inference model may further be trained using all high-quality data and the Q&A pairs corresponding to the high-quality data to obtain a second multi-modal inference model with enhanced inference ability.

[0059] Through this embodiment, a state evaluation mechanism is introduced to determine high-quality data in the correct answers, and the multi-modal model is continuously trained using this high-quality data, enabling the trained model to maintain high-quality information transmission during the learning process and avoiding problems such as information loss and logical inconsistency.

[0060] As an example, the above multi-modal evaluation model may also be a large model, such as GPT-4o can be adopted, and the evaluation model is pre-trained in advance, which is not limited in this disclosure. The above preset threshold can be set as needed, such as it can be set to 0.8 or 0.7, which is also not limited in this disclosure.

[0061] Specifically, the multi-modal evaluation model can be used to evaluate each state in the correct data and score it from -1 to 1 based on its positive or negative impact on problem-solving, and provide the scoring reason; then, manual verification can be performed, such as calculating the average score of all states in if the average score does not exceed 0.8, mark the correct data as low-quality data if the average score is equal to or exceeds 0.8, mark the correct data as high-quality data

[0062] ) + Manual verification

[0063] wherein, represents the prompt of the multi-modal evaluation model.

[0064] Then, using this high-quality data, high-quality SFT data is obtained. , which can be expressed as follows:

[0065]

[0066] It should be noted that the full English name of SFT is Supervised Fine-Tuning, which means supervised fine-tuning.

[0067] According to an embodiment of the present disclosure, the processing performed on each question-and-answer pair in multiple-modal question-and-answer pairs may further include: in response to the average state score of all state data in the first state transition data being less than a preset threshold, inputting the first state transition data, the state score, and the scoring reason into a multi-modal correction model to obtain second state transition data; in response to the second state transition data being high-quality data, determining the first state transition data and the second state transition data as the first training sample pair; wherein, after obtaining the second multi-modal inference model with enhanced inference ability, all the first training sample pairs may further be used to train the second multi-modal inference model to obtain a third multi-modal inference model with enhanced inference ability.

[0068] Through this embodiment, the low-quality data generated in the previous training process is passed through the multi-modal correction model to obtain the corresponding second state transition data. The high-quality second state transition data is selected and combined with the low-quality first state transition data to continue training the second multi-modal inference model, thereby obtaining a model with better inference ability, which can avoid generating the above-mentioned low-quality data.

[0069] As an example, the above multi-modal correction model can also be a large model, such as GPT-4o, and this evaluation model can be pre-trained. The present disclosure does not limit this.

[0070] As an example, when it is determined that the first state data is low-quality data , this low-quality data and the state evaluation result (the score and scoring reason for each state) can be added to the prompt , and this prompt is input into the multi-modal correction model to generate second state transition data. If this second state transition data is high-quality data, this second state transition data is retained. If this second state transition data is low-quality data, this second state transition data and its corresponding question-and-answer pair are deleted. Specifically, it can be expressed as:

[0071]

[0072] Wherein, A prompt representing a multimodal correction model.

[0073] After obtaining high-quality second state transition data, the second state transition data and the low-quality first state transition data can be used as a training pair to continue training the above-mentioned second multimodal inference model to obtain a third multimodal inference model with better inference ability.

[0074] It should be noted that the method for judging the quality of the second state transition data can adopt the method mentioned in the above embodiments, and the present disclosure does not limit this.

[0075] According to an embodiment of the present disclosure, the processing performed on each question-and-answer pair in a plurality of modalities of question-and-answer pairs may further include: in response to the first data state indicating that the first state transition data is incorrect data, inputting the first state transition data into the multimodal correction model to obtain third state transition data; in response to the third state transition data being high-quality data, determining the first state transition data and the third state transition data as a second training sample pair; wherein, after obtaining the second multimodal inference model with enhanced inference ability, all the second training sample pairs can be used to train the second multimodal inference model to obtain a fourth multimodal inference model with enhanced inference ability.

[0076] Through this embodiment, the incorrect data generated in the previous training process is used to obtain the corresponding third state transition data through the multimodal correction model, and the high-quality third state transition data and the incorrect data are selected to continue training the second multimodal inference model, so that a model with better inference ability can be obtained, and this model can avoid generating the above-mentioned incorrect data.

[0077] As an example, when the first state transition data is incorrect data in this case, the incorrect data can be input into the multimodal correction model. The multimodal correction model identifies the incorrect steps and continues the previous correct steps to generate the third state transition data. If the third state transition data is the correct answer then the third state transition data is retained. If the third state transition data is the incorrect answer, the third state transition data and its corresponding question-and-answer pair are deleted. Specifically, it can be expressed as:

[0078]

[0079] Among them, A prompt representing a multimodal correction model.

[0080] After obtaining the correct third state transition data, the correct third state transition data and the incorrect first state transition data can be used as a training pair to continue training the second multi-modal inference model described above, thereby obtaining a fourth multi-modal inference model with better inference ability.

[0081] It should be noted that the method for determining whether the third state transition data is correct can be the method mentioned in the above embodiments, and the present disclosure does not limit this.

[0082] According to an embodiment of the present disclosure, the processing performed on each question-answer pair in a plurality of modal question-answer pairs may further include: in response to the average state score of all state data in the first state transition data being greater than or equal to a preset threshold, inputting the current question into the second multi-modal inference model to obtain fourth state transition data; in response to the fourth state transition data being incorrect data, determining the fourth state transition data and the first state transition data as a third training sample pair; wherein, after obtaining the second multi-modal inference model with enhanced inference ability, the second multi-modal inference model can also be trained using all third training sample pairs to obtain a fifth multi-modal inference model with enhanced inference ability.

[0083] Through this embodiment, the high-quality data generated during the previous training process is used as test data to be input into the first multi-modal inference model to obtain corresponding fourth state transition data. The incorrect fourth state transition data is selected and combined with the high-quality first state transition data to continue training the second multi-modal inference model. The trained model can avoid generating the above incorrect fourth state transition data, further optimizing the model and making the inference ability of the model more excellent.

[0084] As an example, when it is determined that the first state data is high-quality data the question included in the question-answer pair corresponding to this high-quality data is input into the second multi-modal inference model to obtain fourth state transition data. If this fourth state transition data is incorrect data then this fourth state transition data is retained. If this fourth state transition data is correct data, then this fourth state transition data is deleted. The above processing is performed on all high-quality data and can be specifically expressed as:

[0085]

[0086] wherein, represents the prompt of the multi-modal correction model.

[0087] After obtaining the incorrect fourth state transition data, the fourth state transition data and the high-quality first state transition data are combined into a training pair, and the second multi-modal inference model is further trained to obtain a fifth multi-modal inference model with better inference ability.

[0088] It should be noted that the above first training pair, second training pair, and third training pair can be combined to obtain high-quality DPO data. . This process is formalized in the following equation:

[0089]

[0090] The full English spelling of the above DPO is Direct Preference Optimization, that is, direct preference optimization. Then, the second multi-modal model can be trained through three groups of training (that is, ) at the same time, and then a multi-modal inference model with further enhanced inference ability can be obtained.

[0091] To better understand the present disclosure, the following will be combined with Figure 2 and Figure 3 for a systematic description.

[0092] Figure 2 shows the overall process of the present disclosure. As shown in Figure 2 , the overall process includes the following steps:

[0093] S201: Integrate multiple open-source data sets to ensure data diversity, and rearrange the options of multiple-choice questions to create new data, enabling the model to reason out the correct answer from different perspectives.

[0094] S202: Define the decision set for the integration result of the previous step as four major stages: question understanding, question solving, verification, and answer. Among them, the question understanding stage focuses on the preliminary understanding and recognition of graphics and text; the question solving stage realizes step-by-step reasoning through decomposition and planning; the verification stage allows the model to self-check, verify, and correct its reasoning process; the answer stage integrates the reasoning process and outputs the final answer.

[0095] S203: Embed the decision set of the previous step into the prompt template, and drive the multi-modal inference model to dynamically select decisions from the decision set according to the images and texts in the input question and generate state transition data, that is, to achieve a context-related reasoning path by adaptively generating a coherent thinking chain, ensuring that the generated state transition data is deeply adapted to the multi-modal task.

[0096] S204: Construct high-quality SFT data: Compare the state transition data obtained in the previous step of introducing the multi-modal evaluation model with the true answers in the question-and-answer pairs to determine whether the state transition data is correct or incorrect. Then, use the state evaluation mechanism to quantitatively score each state of the correct data (-1 to 1), and obtain the average of the states calculated manually. Mark the state transition data with an average score ≥ 0.8 as high-quality SFT data, and form a high-quality supervised fine-tuning data set based on the SFT data.

[0097] S205: Construct high-quality DPO data through three parts: 1) For the incorrect data in step S204, correct the incorrect steps and continue the incorrect steps to generate corresponding correct answers. The specific steps have been described in detail above and will not be elaborated here; then, regard the incorrect data and the corresponding correct data as a set of training pairs. 2) For the low-quality data that did not reach 0.8 in step S204, regenerate an optimized answer in combination with the evaluation results, that is, generate corresponding high-quality data. The specific steps have been described in detail above and will not be elaborated here; then, form a set of training pairs with the low-quality data and the corresponding high-quality data. 3) For the SFT data, use the SFT model to re-infer the questions corresponding to the SFT data to obtain new state transition data, and extract the incorrect state transition data and the corresponding SFT data to form a set of training pairs. Among them, the SFT model is the model trained using the SFT data, that is, the second multi-modal inference model mentioned above. 。

[0098] S206, Training of the model: After obtaining the correct data in S204, the multi-modal inference model can be trained using the correct data to obtain the first multi-modal inference model with enhanced inference ability; then, continue to train the first multi-modal inference model using the high-quality data to obtain the second multi-modal inference model with further enhanced inference ability; then, the second multi-modal inference model can also be trained using the three sets of training pairs respectively to obtain the third multi-modal inference model, the fourth multi-modal inference model, and the fifth multi-modal inference model with further enhanced inference ability; finally, the second multi-modal inference model can be trained using the three sets of training pairs simultaneously to obtain a multi-modal inference model with further enhanced inference ability.

[0099] Figure 3 shows the overall structure of the present disclosure, such as Figure 3As shown in the figure, the overall structure of the present disclosure may include the following parts: a state transition data generation part, a high-quality SFT data construction part, a high-quality DPO data construction part, and a model training part. Among them, the state transition data generation part includes constructing a multi-task dataset, determining a decision set, and generating and evaluating state transition data. The high-quality SFT data construction part includes evaluating the state transition data to obtain corresponding scores and scoring reasons. The high-quality DPO construction part includes correcting error data and low-quality data. It should be noted that this part also includes re-inferring the SFT data, which is not shown. The model training part includes training a multi-modal inference model using the SFT data and re-training the SFT model using the DPO data. It should be noted that this part also includes training with other data, which is not shown.

[0100] In summary, the present disclosure proposes a new method to enhance the reasoning ability of multi-modal large models, which can be called CoMMoD (the full English name is Controlled Multi-Modal Decision-Making for Chain-of-Thought Reasoning in Multi-Modal Language Models). By introducing a decision set for adaptive problem-solving and a state evaluation mechanism, the reasoning ability of multi-modal large models in visual question answering tasks is significantly improved. Compared with the prior art, the present disclosure can effectively generate coherent and logically consistent reasoning chains, ensuring the answer accuracy and reliability of the trained model when dealing with complex problems. Moreover, by adaptively adjusting the reasoning path and ensuring the integrity of the intermediate reasoning steps, the trained model can flexibly respond to different types of task requirements, overcoming the limitations of the fixed reasoning path of traditional models. In addition, the introduction of the state evaluation mechanism enables the trained model to maintain high-quality information transmission during the learning process, avoiding problems such as information loss and logical inconsistency. Therefore, the present disclosure enhances the adaptability and expressiveness of the model in diverse task scenarios, providing more solid technical support for the practical application of multi-modal large models.

[0101] Figure 4 Shows the state transition data output by the Llama-CoMMoD and Llama-CoMMoD models for the same question. It can be seen that after applying the method of the present disclosure, coherent and logically consistent state transition data can be generated, and accurate and reliable answers are given.

[0102] Figure 5 Shows a comparison chart of the reasoning effects of two models that do not apply the method of the present disclosure and the model that applies the method of the present disclosure, as Figure 5As shown, the model to which the method of the present disclosure is applied gives coherent and logically consistent state transition data and gives accurate and reliable answers.

[0103] The experimental results show that, compared with the prior art, the present disclosure significantly improves the reasoning ability and answer accuracy of the model in complex scenarios, verifying the effectiveness and superiority of the method of the present invention.

[0104] Figure 6 is a block diagram of a device for enhancing the reasoning ability of the enhanced multi-modal reasoning model according to an embodiment of the present disclosure, as Figure 6 shown, the device includes a construction unit 60, a determination unit 62, an execution unit 64 and a training unit 66.

[0105] The construction unit 60 is configured to construct a multi-source data set, where the multi-source data set includes question-and-answer pairs of multiple modalities, and each question-and-answer pair includes a question and the true answer to the question; the determination unit 62 is configured to determine a decision set for solving the question, where the decision set includes multiple decisions, and the estimated answer to any question can be determined based on at least one of the multiple decisions; the execution unit 64 is configured to perform the following processing for each question-and-answer pair in the question-and-answer pairs of multiple modalities: input the decision set and the current question in the current question-and-answer pair into the multi-modal reasoning model to obtain first state transition data of the current question, where the first state transition data is logical data for reasoning the estimated answer to the current question based on at least one of the multiple decisions; based on the true answer in the current question-and-answer pair, determine the first data state of the first state transition data, where the first data state indicates that the first state transition data is correct data or incorrect data; the training unit 66 is configured to, in response to all question-and-answer pairs having completed the above processing, use all the correct data and the question-and-answer pairs corresponding to the correct data to train the multi-modal reasoning model to obtain a first multi-modal reasoning model with enhanced reasoning ability.

[0106] According to an embodiment of the present disclosure, the execution unit 64 is further configured to, in response to the first data state indicating that the first state transition data is correct data, input the first state transition data into the multi-modal evaluation model to obtain the state score and the scoring reason of each state data in the first state transition data; in response to the average state score of all state data in the first state transition data being greater than or equal to a preset threshold, determine that the first state transition data is high-quality data; where the training unit 66 is further configured to, after obtaining the first multi-modal reasoning model with enhanced reasoning ability, use all the high-quality data and the question-and-answer pairs corresponding to the high-quality data to train the first multi-modal reasoning model to obtain a second multi-modal reasoning model with enhanced reasoning ability.

[0107] According to an embodiment of the present disclosure, the execution unit 64 is further configured to, in response to the average state score of all state data in the first state transition data being less than a preset threshold, input the first state transition data, the state score, and the scoring reason into a multimodal correction model to obtain second state transition data; and in response to the second state transition data being high-quality data, determine the first state transition data and the second state transition data as a first training sample pair; wherein, the training unit 66 is further configured to, after obtaining the second multimodal inference model with enhanced inference ability, use all the first training sample pairs to train the second multimodal inference model to obtain a third multimodal inference model with enhanced inference ability.

[0108] According to an embodiment of the present disclosure, the execution unit 64 is further configured to, for each question-answer pair among multiple-modal question-answer pairs, the processing further includes: in response to the first data state indicating that the first state transition data is incorrect data, input the first state transition data into a multimodal correction model to obtain third state transition data; and in response to the third state transition data being high-quality data, determine the first state transition data and the third state transition data as a second training sample pair; wherein, the training unit 66 is further configured to, after obtaining the second multimodal inference model with enhanced inference ability, use all the second training sample pairs to train the second multimodal inference model to obtain a fourth multimodal inference model with enhanced inference ability.

[0109] According to an embodiment of the present disclosure, the execution unit 64 is further configured to, for each question-answer pair among multiple-modal question-answer pairs, the processing further includes: in response to the average state score of all state data in the first state transition data being greater than or equal to a preset threshold, input the current question into the second multimodal inference model to obtain fourth state transition data; and in response to the fourth state transition data being incorrect data, determine the fourth state transition data and the first state transition data as a third training sample pair; wherein, the training unit 66 is further configured to, after obtaining the second multimodal inference model with enhanced inference ability, use all the third training sample pairs to train the second multimodal inference model to obtain a fifth multimodal inference model with enhanced inference ability.

[0110] According to an embodiment of the present disclosure, the execution unit 64 is configured to input the first state transition data and the true answer in the current question-answer pair into a multimodal evaluation model to obtain the first data state of the first state transition data.

[0111] According to an embodiment of the present disclosure, a computer-readable storage medium storing instructions is provided, wherein when the instructions are run by at least one computing device, at least one computing device is caused to execute the method for enhancing the inference ability of the multimodal inference model as described in any one of the above embodiments.

[0112] According to an embodiment of the present disclosure, a system is provided that includes at least one computing device and at least one storage device storing instructions, wherein when the instructions are run by the at least one computing device, the at least one computing device is caused to execute a method for enhancing the inference ability of the enhanced multimodal inference model according to any of the above embodiments.

[0113] According to an embodiment of the present disclosure, a computer program product is provided that includes computer instructions which, when executed by a processor, implement a method for enhancing the inference ability of any of the above enhanced multimodal inference models.

[0114] Although some embodiments of the present disclosure have been shown and described, those skilled in the art should understand that these embodiments can be modified without departing from the principles and spirit of the present disclosure as defined by the claims and their equivalents.

Claims

1. A method for enhancing the reasoning ability of a multi-modal reasoning model, characterized in that, Including: Constructing a multi-source dataset, where the multi-source dataset contains question-and-answer pairs of multiple modalities, and each question-and-answer pair contains a question and the true answer to the question; Determining a decision set for solving questions, where the decision set contains multiple decisions, and the predicted answer to any question can be determined based on at least one of the multiple decisions; For each question-and-answer pair in the question-and-answer pairs of multiple modalities, perform the following processing: Input the decision set and the current question in the current question-and-answer pair into a multi-modal inference model to obtain first state transition data of the current question, where the first state transition data is logical data for inferring the predicted answer to the current question based on at least one of the multiple decisions; Based on the true answer in the current question-and-answer pair, determine the first data state of the first state transition data, where the first data state indicates whether the first state transition data is correct data or incorrect data; In response to all question-and-answer pairs having completed the above processing, use all correct data and the question-and-answer pairs corresponding to the correct data to train the multi-modal inference model to obtain a first multi-modal inference model with enhanced inference ability.

2. The method according to claim 1, characterized in that, The processing performed for each question-and-answer pair in the question-and-answer pairs of multiple modalities further includes: In response to the first data state indicating that the first state transition data is correct data, input the first state transition data into a multi-modal evaluation model to obtain the state score and scoring reason for each state data in the first state transition data; In response to the average state score of all state data in the first state transition data being greater than or equal to a preset threshold, determine that the first state transition data is high-quality data; Wherein, after obtaining the first multi-modal inference model with enhanced inference ability, it further includes: Using all high-quality data and the question-and-answer pairs corresponding to the high-quality data to train the first multi-modal inference model to obtain a second multi-modal inference model with enhanced inference ability.

3. The method according to claim 2, wherein The processing performed for each question-and-answer pair in the question-and-answer pairs of multiple modalities further includes: In response to the average state score of all state data in the first state transition data being less than the preset threshold, input the first state transition data, the state score, and the scoring reason into a multi-modal correction model to obtain second state transition data; In response to the second state transition data being high-quality data, determine the first state transition data and the second state transition data as a first training sample pair; Wherein, after obtaining the second multi-modal inference model with enhanced inference ability, it further includes: Using all first training sample pairs to train the second multi-modal inference model to obtain a third multi-modal inference model with enhanced inference ability.

4. The method according to claim 2, wherein The processing performed for each question-and-answer pair in the question-and-answer pairs of multiple modalities further includes: In response to the first data state indicating that the first state transition data is incorrect data, input the first state transition data into a multi-modal correction model to obtain third state transition data; In response to the third state transition data being high-quality data, determine the first state transition data and the third state transition data as a second training sample pair; Among them, after obtaining the second multi-modal inference model with enhanced inference ability, it further includes: Use all the second training sample pairs to train the second multi-modal inference model to obtain a fourth multi-modal inference model with enhanced inference ability.

5. The method according to claim 2, wherein For the processing performed on each question-and-answer pair among the multiple-modal question-and-answer pairs, it further includes: In response to the average state score of all state data in the first state transition data being greater than or equal to the preset threshold, input the current question into the second multi-modal inference model to obtain fourth state transition data; In response to the fourth state transition data being incorrect data, determine the fourth state transition data and the first state transition data as a third training sample pair; Among them, after obtaining the second multi-modal inference model with enhanced inference ability, it further includes: Use all the third training sample pairs to train the second multi-modal inference model to obtain a fifth multi-modal inference model with enhanced inference ability.

6. The method according to claim 1, wherein The determining the first data state of the first state transition data based on the true answer in the current question-and-answer pair includes: Input the first state transition data and the true answer in the current question-and-answer pair into a multi-modal evaluation model to obtain the first data state of the first state transition data.

7. An apparatus for enhancing the reasoning ability of a multimodal reasoning model, characterized in that, It includes: A construction unit configured to construct a multi-source data set, where the multi-source data set contains multiple-modal question-and-answer pairs, and each question-and-answer pair contains a question and the true answer to the question; A determination unit configured to determine a decision set for solving the problem, where the decision set contains multiple decisions, and the estimated answer to any question can be determined based on at least one of the multiple decisions; An execution unit configured to perform the following processing on each question-and-answer pair among the multiple-modal question-and-answer pairs: Input the decision set and the current question in the current question-and-answer pair into a multi-modal inference model to obtain the first state transition data of the current question, where the first state transition data is logical data for inferring the estimated answer to the current question based on at least one of the multiple decisions; Determine the first data state of the first state transition data based on the true answer in the current question-and-answer pair, where the first data state indicates that the first state transition data is correct data or incorrect data; A training unit configured to, in response to all question-and-answer pairs having completed the above processing, use all the correct data and the question-and-answer pairs corresponding to the correct data to train the multi-modal inference model to obtain a first multi-modal inference model with enhanced inference ability.

8. A computer-readable storage medium for storing instructions, characterized in that, When the instruction is run by at least one computing device, cause the at least one computing device to execute the method for enhancing the inference ability of the multi-modal inference model according to any one of claims 1 to 6.

9. A system comprising at least one computing device and at least one storage device storing instructions, characterized in that, When the instructions are run by the at least one computing device, the at least one computing device is caused to perform the method of enhancing the reasoning ability of the enhanced multimodal reasoning model as described in any one of claims 1 to 6.

10. A computer program product comprising computer instructions, characterized in that, When the computer instructions are executed by a processor, the method of enhancing the reasoning ability of the enhanced multimodal reasoning model as described in any one of claims 1 to 6 is implemented.

Images