A model reasoning method and device based on visual language model

Abstract

The present invention discloses a model reasoning method and device based on a visual language model, and relates to the field of artificial intelligence technology. The method comprises: encoding a text to be reasoned and an image to be reasoned, and generating an initial reasoning tree; based on the feasibility parameters of the reasoning action in the image to be reasoned and the state parameters of the current node, the initial reasoning tree is extended and updated to generate a target reasoning tree; based on the rationality parameters of all end nodes in the target reasoning tree, a candidate reasoning path is determined; according to the consistency parameters of the intermediate nodes in the candidate reasoning path and the rationality parameters of the end nodes, the target reasoning path is determined, and finally the target response data is output through the target reasoning path. The present invention can improve the reasoning efficiency and accuracy of the visual language model in complex reasoning tasks.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a model reasoning method and device based on a visual language model. Background Art

[0002] Visual Language Model (VLM) can understand visual images and text content, integrate visual information and language information, and complete image understanding tasks such as image description generation, visual question answering, image caption generation, etc. However, VLM has difficulty performing systematic and structured reasoning in complex reasoning tasks, especially in tasks that require multi-step reasoning, and often makes mistakes.

[0003] At present, the thinking chain or Monte Carlo Tree Search (MCTS) can be combined with the reasoning of VLM to enhance the multi-step reasoning ability of VLM, reduce model hallucinations, and improve the accuracy of model reasoning. However, in the Monte Carlo tree search algorithm, the search tree will expand rapidly, and the amount of data processing is huge, which leads to low reasoning efficiency. In addition, there is no scientific path selection strategy for subsequent path selection, and it is easy to select unreliable reasoning paths for reasoning, resulting in low reasoning efficiency and accuracy of visual language models in complex reasoning tasks. Summary of the invention

[0004] In view of the above problems, the purpose of the present invention is to provide a model reasoning method and device based on a visual language model, which can improve the reasoning efficiency and accuracy of the visual language model in complex reasoning tasks.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] In one aspect, the present invention provides a model reasoning method based on a visual language model, the method comprising:

[0007] Acquire data to be inferred, wherein the data to be inferred includes text to be inferred and images to be inferred;

[0008] Encoding the text to be inferred and the image to be inferred through a visual language model to generate state parameters of an initial node in an initial inference tree;

[0009] Based on the feasibility parameters of the reasoning action in the image to be reasoned and the state parameters of the current node, the nodes of the initial reasoning tree are extended and updated to obtain a target reasoning tree, wherein the current node takes the initial node as the starting point of change, and each node in the target reasoning tree has corresponding state parameters and rationality parameters;

[0010] Based on the rationality parameters of all the end nodes in the target inference tree, determine a candidate inference path from the inference path of the target inference tree, the state parameter of the end node includes the response data corresponding to the data to be inferred, and the inference path includes the initial node, the intermediate node and the end node;

[0011] Determining a target reasoning path from the candidate reasoning paths according to consistency parameters of the intermediate nodes in the candidate reasoning paths and rationality parameters of the end nodes;

[0012] The visual language model is controlled to output target response data corresponding to the data to be inferred according to the target reasoning path.

[0013] On the other hand, the present invention also provides a model reasoning device based on a visual language model, the device comprising:

[0014] An acquisition module, used to acquire data to be inferred, wherein the data to be inferred includes text to be inferred and images to be inferred;

[0015] An initial module, used to encode the text to be inferred and the image to be inferred through a visual language model to generate state parameters of an initial node in an initial inference tree;

[0016] An extended update module, used to perform extended update processing on the nodes of the initial reasoning tree based on the feasibility parameters of the reasoning action in the image to be reasoned and the state parameters of the current node, to obtain a target reasoning tree, wherein the current node takes the initial node as the starting point of change, and each node in the target reasoning tree has corresponding state parameters and rationality parameters;

[0017] A candidate determination module, configured to determine a candidate reasoning path from the reasoning path of the target reasoning tree based on the rationality parameters of all the end nodes in the target reasoning tree, wherein the state parameters of the end nodes include the response data corresponding to the data to be inferred, and the reasoning path includes the initial node, the intermediate node and the end node;

[0018] A target determination module, used to determine a target reasoning path from the candidate reasoning paths according to consistency parameters of the intermediate nodes in the candidate reasoning paths and rationality parameters of the end nodes;

[0019] The inference module is used to control the visual language model to output target response data corresponding to the data to be inferred according to the target inference path.

[0020] On the other hand, the present invention also provides an electronic device, including a processor and a memory, wherein the memory stores multiple instructions; the processor loads instructions from the memory to execute the steps in any one of the model reasoning methods based on the visual language model provided by the present invention.

[0021] On the other hand, the present invention also provides a computer-readable storage medium, which stores multiple instructions, and the instructions are suitable for a processor to load to execute the steps in any model reasoning method based on a visual language model provided by the present invention.

[0022] On the other hand, the present invention also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps in any one of the model reasoning methods based on the visual language model provided by the present invention.

[0023] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0024] In an embodiment of the present invention, the nodes in the initial reasoning tree are expanded and updated by utilizing the feasibility parameters of the reasoning action in the image to be reasoned and the state of the current node, which can avoid introducing unreasonable nodes in the expansion, reduce the amount of data processing, and achieve rapid generation of a target reasoning tree. In the target reasoning tree, candidate reasoning paths with reliable results are screened out based on the rationality parameters of the end node, and the quality of the candidate reasoning paths is comprehensively evaluated by utilizing the consistency parameters of the intermediate nodes in the candidate reasoning paths and the rationality parameters of the end nodes, ensuring that a target reasoning path with a reliable reasoning process can be screened out for reasoning, thereby ensuring that the visual language model can perform reasoning efficiently and accurately in complex reasoning scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 Schematic diagram of an application scenario of a model reasoning method based on a visual language model provided by an embodiment of the present invention;

[0027] Figure 2 is a flowchart of a model reasoning method based on a visual language model provided by an embodiment of the present invention;

[0028] Figure 3 It is a schematic diagram of a process of generating a target inference tree provided by an embodiment of the present invention;

[0029] Figure 4 is a schematic diagram of a target inference tree provided by an embodiment of the present invention;

[0030] Figure 5 is a schematic diagram of the structure of a model reasoning device based on a visual language model provided by an embodiment of the present invention;

[0031] Figure 6 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0033] The present invention proposes a model reasoning method and device based on a visual language model to improve the reasoning efficiency and accuracy of the visual language model in complex reasoning tasks.

[0034] It is understandable that in the specific implementation of the present invention, data related to user information, etc., needs to obtain user permission or consent, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0035] See also Figure 1 , shows a schematic diagram of an application scenario of a model reasoning method based on a visual language model. The application scenario may include a terminal 101 and a server 102, and data can be exchanged between the terminal 101 and the server 102 through a network, and an application program related to question and answer can be installed on the terminal 101. The terminal 101 may be a mobile phone, a tablet computer, a smart Bluetooth device, a computer, a large screen, a robot, and other devices; the server 102 may be a single server or a server cluster consisting of multiple servers.

[0036] The user can send the data to be inferred to the server 102 through the terminal 101, so that the server 102 can obtain the data to be inferred; then the server 102 can use the visual language model to encode the text to be inferred and the image to be inferred, and generate the state parameters of the initial node in the initial reasoning tree; based on the feasibility parameters of the reasoning action in the image to be inferred and the state parameters of the current node, the nodes of the initial reasoning tree are expanded and updated to obtain the target reasoning tree; based on the rationality parameters of all end nodes in the target reasoning tree, the candidate reasoning path is determined from the reasoning path of the target reasoning tree; according to the consistency parameters of the intermediate nodes in the candidate reasoning path and the rationality parameters of the end nodes, the target reasoning path is determined from the candidate reasoning paths; finally, the visual language model is controlled to output the target response data corresponding to the data to be inferred according to the target reasoning path.

[0037] The server 102 may then send the inferred target response data to the terminal 101 so as to display the target response data to the user.

[0038] In this embodiment, a model reasoning method based on a visual language model is provided, such as Figure 2 As shown, the specific process of the model reasoning method based on the visual language model can be as follows:

[0039] S110: Obtain data to be inferred.

[0040] The data to be inferred is the data that needs to be inferred and input by the user. The data to be inferred may include text to be inferred and images to be inferred. The text to be inferred is data in text form. The user can directly input the text or convert the language input by the user into text. The image to be inferred may be a photo, picture, etc.

[0041] S120 , encoding the text to be inferred and the image to be inferred using a visual language model to generate state parameters of an initial node in an initial inference tree.

[0042] The text to be inferred and the image to be inferred are input into the visual language model, and the visual language model is used to encode them to obtain the text features corresponding to the text to be inferred and the image features corresponding to the image to be inferred. Specifically, the image to be inferred can be processed using the image encoder of the visual language model to obtain the initial image features, and then the initial image features are mapped to the semantic space of image and text alignment through the projection layer as the image features corresponding to the image to be inferred. The text segmenter is used to segment and vectorize the text to be inferred as the text features corresponding to the text to be inferred.

[0043] The image features and text features are spliced ​​together, and the spliced ​​features obtained can be used as the state parameters of the initial node in the initial inference tree. At this time, the initial inference tree obtained only contains the initial node. The initial inference tree is the basis for generating the target inference tree. The initial node can also be called the root node. The state parameter of the initial node is the spliced ​​features after splicing the image features and text features, which can be used to characterize the data to be inferred.

[0044] S130 , based on the feasibility parameters of the reasoning action in the image to be reasoned and the state parameters of the current node, the nodes of the initial reasoning tree are extended and updated to obtain a target reasoning tree.

[0045] In the process of generating the target inference tree based on the initial inference tree, it is necessary to continuously expand the nodes of the initial inference tree and select the current node from the expanded nodes. The current node can change in the expanded nodes, and each expansion and update process takes the initial node as the starting point of the change, that is, multiple current nodes can be generated in the expansion and update process, and the first current node is the initial node.

[0046] The target inference tree is the inference tree obtained after multiple expansion and update processes are performed on the basis of the initial inference tree. The target inference tree may include multiple nodes, each of which has its corresponding state parameters and rationality parameters. Among them, the nodes in the target inference tree may refer to a data state or a processing step, etc. If the node refers to the data state, its state parameter is the specific data state. If the node refers to the processing step, the state parameter can be considered as the action execution result obtained after the corresponding action is executed. The rationality parameter is a parameter obtained by the visual language model after evaluating the rationality of the reasoning action executed after each node is generated in order to improve the quality of the reasoning chain.

[0047] The reasoning action is an action pre-set based on the way humans solve problems. There may be multiple reasoning actions, which can be set according to actual needs. In the embodiment of the present invention, reasoning is performed on multimodal data, and the reasoning actions can be divided into thinking-level actions and action-level actions.

[0048] Among them, the action of the thinking level refers to the action of the thinking level that can simulate the human when solving complex reasoning tasks, such as decomposing problems, raising new sub-problems or restating problems, etc. These reasoning actions can be used to guide the visual language model to perform more advanced reasoning planning. In the implementation of the present invention, the action of the thinking level may include problem decomposition, next step reasoning, quick thinking and reflection.

[0049] Problem decomposition refers to breaking down a complex problem into a series of simpler sub-problems. The visual language model can generate a list of sub-problems or a sub-problem decomposition strategy based on the problem and the state parameters of the current node. For example, for a multi-step reasoning problem, the visual language model can be decomposed into "first identify A in the image", "determine the attributes of A", and "answer the question based on the attributes of A".

[0050] The next step of reasoning means that the visual language model can generate the next step of thinking for a given problem based on the existing reasoning steps, so that the model can make better decisions. The visual language model can generate the logical steps of the next step of reasoning based on the state parameters of the current node and the reasoning actions that have been executed. For example, if A in the image has been identified, the next step of reasoning can be "extracting the attributes of A", "associating A with the keyword B in the question", etc.

[0051] Quick thinking refers to giving a final answer directly to the current problem. Reflection means that the visual language model reflects on itself and re-evaluates its reasoning steps to determine if there are any problems and correct any errors. The visual language model can generate an evaluation and reflection on the reasoning process based on the state parameters of the current node and the reasoning actions that have been performed. For example, the visual language model can evaluate whether the current result meets expectations and determine whether it is necessary to re-plan the reasoning path or adjust the previous operation.

[0052] Actions at the action level are specific actions that are performed, and may include visual operations or language operations. Among them, visual operations may include operations such as identifying objects, locating objects, identifying attributes, and reasoning about relationships. Language operations may include operations such as extracting keywords, generating descriptions, and performing reasoning.

[0053] The feasibility parameter can be used to measure the feasibility of executing each reasoning action based on the image to be inferred. The larger the feasibility parameter, the more feasible it is to execute the reasoning action, so as to expand and update the initial reasoning tree and finally obtain the target reasoning tree. The feasibility parameter can be used to screen the reasoning actions, and the feasible reasoning actions can be screened out to generate the target reasoning tree, which can avoid introducing infeasible reasoning actions and increase the amount of data processing.

[0054] As an implementation mode, when generating a target reasoning tree, the current node may be selected from the candidate nodes based on the exploration parameters, the access parameters of the candidate nodes, and the reward parameters, wherein the exploration parameters are related to the current number of executions of the expansion and update processing; the extension node of the current node is generated using the feasibility parameters of the reasoning action in the image to be reasoned and the state parameters of the current node, and the rationality parameters of the extension node are calculated; the extension node is used as a new candidate node, and the steps of selecting the current node from the candidate nodes based on the exploration parameters, the access parameters of the candidate nodes, and the reward parameters and subsequent steps are returned to be executed until a specified condition is met, and a simulated reasoning path is generated; all model output values ​​related to the data to be reasoned generated by the visual language model in the process of generating the simulated reasoning path are used to calculate the simulated reward value of the simulated reasoning path; the reward parameters and access parameters of all nodes in the simulated reasoning path are updated with the simulated reward value to obtain a new initial reasoning tree, and the current number of executions is updated; the steps of selecting the current node from the candidate nodes based on the exploration parameters, the access parameters of the candidate nodes, and the reward parameters and subsequent steps are returned to be executed with the new initial reasoning tree until the current number of executions reaches a preset number, and the target reasoning tree is obtained.

[0055] The candidate node is the node that can be selected at present. When there is only the initial node in the initial reasoning tree, the initial node is the candidate node. When the initial reasoning tree includes other nodes except the initial node, the candidate node is the expanded node obtained after the current node is expanded.

[0056] For each candidate node, a corresponding value parameter may be calculated for the candidate node so that the current node can be selected from the candidate nodes using the value parameter. In some embodiments, the value parameter of the candidate node may be calculated by exploration parameters, access parameters of the candidate node, and reward parameters. Among them, the access parameters of the candidate node may include the number of visits to the candidate node and the number of visits to the parent node of the candidate node; the reward parameter may be used to measure the contribution of the node, and the larger the reward parameter, the greater the contribution; the exploration parameter is related to the current number of executions of the extended update process.

[0057] To expand and update the initial reasoning tree to the target reasoning tree, it is necessary to perform multiple expansion and update processes. A complete expansion and update process requires the execution of selection processing, expansion processing, simulation processing, and update processing. Among them, selection processing refers to selecting a node to use as the current node, and expansion processing refers to generating an expansion node for the current node according to the reasoning action; simulation processing refers to continuously executing selection processing and expansion processing until the simulated reasoning path is obtained when the specified conditions are met, and the simulated reward parameters corresponding to the simulated reasoning path are calculated; update processing refers to updating the reward parameters and access parameters of the nodes on the simulated reasoning path with the calculated simulation reward parameters to obtain the updated initial reasoning tree. Return to execute the expansion and update processing with the updated initial reasoning tree until the current execution number of the expansion and update processing reaches the preset number, and the target reasoning tree can be obtained.

[0058] For details, please refer to Figure 3 , showing a flow chart of generating a target reasoning tree. The selection process needs to select one from the candidate nodes as the current node. In the embodiment of the present invention, when selecting the current node from the candidate nodes, the first ratio can be calculated by using the reward parameter of the candidate node and the number of visits to the candidate node; the second ratio can be calculated by using the number of visits to the parent node of the candidate node and the number of visits to the candidate node; the ratio between the current number of executions of the extended update process and the preset number of times is calculated; the exploration parameter under the current number of executions is calculated by using the value range of the exploration weight and the number ratio; the second ratio is weighted by using the exploration parameter, and the weighted result and the first ratio are summed to obtain the value parameter of the candidate node; the current node is selected from the candidate nodes by using the value parameter of each candidate node.

[0059] In the selection process, two strategies are usually adopted, namely exploration and exploitation. Exploration means selecting nodes with smaller visit parameters each time to obtain more information, and exploitation means selecting nodes with higher reward parameters each time. The exploration parameter is a parameter used to balance the relationship between exploration and exploitation. When the value of the exploration parameter is large, the selection strategy tends to explore, and when the value of the exploration parameter is small, the selection strategy tends to exploit.

[0060] For each candidate node, the reward parameter of the candidate node can be divided by the number of visits to the candidate node to obtain a first ratio, and the number of visits to the candidate node's parent node and the number of visits to the candidate node can be used to calculate a second ratio. The second ratio is weighted using the exploration parameter, and the weighted result and the first ratio are summed to calculate the value parameter of the candidate node.

[0061] Specifically, the value parameter of the candidate node can be calculated according to the following formula:

[0062] ;

[0063] Among them, U represents the value parameter of the candidate node; w i Characterize the current reward parameters of the candidate node; n i Characterizes the number of visits to the candidate node; Then it is the first ratio; N i Represents the number of visits to the parent node of the candidate node. is the second ratio; C represents the exploration parameter.

[0064] As an implementation manner, the exploration parameter may be a preset empirical value.

[0065] As another implementation, in order to balance exploration and utilization, the exploration parameter is not a fixed value, but changes with the number of executions of the extended update process. That is, the current number of executions of the extended update process can be divided by the preset number of times to obtain the number ratio, and then the exploration parameter under the current number of executions can be calculated based on the value range of the exploration weight and the number ratio.

[0066] The value range of the exploration weight may include a maximum value range and a minimum value range, which may be set according to actual needs. For example, in an embodiment of the present invention, the maximum value range of the exploration weight may be 0.8-1, and the minimum value range may be 0.1-0.3. Optionally, in order to encourage exploration in the early stage of generating the target inference tree, a larger exploration parameter is required. As the number of executions of the extended update process increases, it gradually turns to utilization, and a smaller exploration parameter is required.

[0067] Specifically, the exploration parameter can be calculated by the following formula:

[0068] ;

[0069] Wherein, C represents the exploration parameter; t represents the current execution number of the extended update process; n represents the preset number; Characterization frequency ratio; w max The maximum value representing the exploration weight can be set to 0.8-1 when t is less than x1, where x1 is the initial critical execution number, that is, the current execution number is less than the initial critical execution number, indicating that it is in the initial stage; w min The minimum value representing the exploration weight can be set to 0.1-0.3 when t is greater than x2, where x2 is the final critical execution number, that is, when the current execution number is greater than the final critical number, it indicates that it is in the final stage, where x2>x1, and the specific values ​​of x1 and x2 can be set according to actual needs;α It is used to control the change rate of the exploration parameters. When the value is small, the change of the exploration parameters is relatively gentle. When the value is large, the change of the exploration parameters is relatively steep.

[0070] After calculating the value parameter of each candidate node, the candidate node with the maximum value parameter may be used as the current node.

[0071] After the selection process comes the expansion process, which uses reasoning actions to generate an expansion node of the current node and calculates the rationality parameters of the expansion node for subsequent processing.

[0072] In some embodiments, during the expansion processing, based on the state parameters of the current node and the feasibility parameters of the reasoning action in the image to be inferred, feasible reasoning actions can be selected and executed from the reasoning actions to generate expansion nodes, thereby avoiding the introduction of some infeasible expansion nodes, thereby reducing the subsequent data processing volume and improving the efficiency of generating the target reasoning tree.

[0073] For example, when generating an extended node, a visual language model may be used to generate a question summary corresponding to the text to be inferred, and an image description corresponding to the image to be inferred; a preceding node path from the initial node to the current node is obtained; for each of the reasoning actions, a feasibility parameter of the reasoning action is calculated using the image description, the question summary, the preceding node path, and the image to be inferred; based on the state parameters of the current node, a reasoning action whose feasibility parameter is not less than a preset parameter is executed to generate an extended node of the current node; for each of the extended nodes, a rationality parameter of the extended node is calculated using the preceding node path, the extended node, and the image to be inferred.

[0074] The data to be inferred may include text to be inferred and images to be inferred. The visual language model can generate a high-level summary explanation of the text to be inferred based on the text to be inferred, summarize the main aspects of the problem that the visual language model needs to deal with, and obtain the problem summary. For the image to be inferred, the visual language model can provide a concise summary of the visual elements related to the problem in the image to be inferred, that is, the image description, to assist in understanding the image to be inferred.

[0075] The previous node path refers to the path composed of each determined current node, that is, the path composed of multiple current nodes determined in sequence starting from the initial node. For example, see Figure 4 , shows a schematic diagram of the target reasoning tree, where the gray node is one of the reasoning paths. If the current node is node 8, the previous node path is node 1-node 3-node 8.

[0076] There are multiple reasoning actions. For each reasoning action, the visual language model can use visual information to determine whether the action is feasible. Specifically, the feasibility parameters of the reasoning action can be calculated to measure the feasibility.

[0077] When calculating the feasibility parameters, the image description, the problem summary, the preceding node path, the image to be inferred, and the inference action can be filled into a first prompt word template to generate a first prompt word; a visual language model is used to determine whether the inference action is feasible according to the first prompt word to obtain a first determination result; if the first determination result is a first preset result, a model output value corresponding to the first determination result is obtained as a feasibility parameter of the inference action.

[0078] The first prompt word template is a preset prompt word template for feasibility determination, and the first prompt word template may include multiple slots to be filled, such as a summary slot, a description slot, a current slot, an action slot, and an image slot. Specifically, the first prompt word template is as follows:

[0079] Prompt1 = f"""

[0080] You are a professional visual analysis assistant. I will show you a picture and provide you with a series of reasoning steps (COT). Your task is to carefully analyze the content of the picture and determine whether the current action is feasible based on the visual information in the picture.

[0081] ## Judgment Guide

[0082] 1. Can the current thinking-level actions significantly optimize the reasoning path?

[0083] 2. Can the execution at the current action level provide more clues and references for reasoning, or come up with the final answer?

[0084]

[0085] {image}

[0086]

[0087] ## Problem Summary:

[0088] { question_summary}

[0089] ## Image Description:

[0090] { image_summary}

[0091] ### Reasoning steps:

[0092] {COT}

[0093] ### Current Actions

[0094] {action}

[0095] Output requirements: Just answer Yes or No

[0096] """

[0097] Among them, {question_summry} is the summary slot, which is used to fill in the question summary; {image_summry} is the description slot, which is used to fill in the image description; {COT} is the current slot, which is used to fill in the previous node path; {action} is the action slot, which is used to fill in the current reasoning action that needs to be determined; {image} is the image slot, which is used to fill in the image to be inferred, or the image features of the image to be inferred, or the image features in the state parameters of the current node. After the first prompt word template is filled, the first prompt word can be obtained, and the first prompt word is input into the visual language model so that the visual language model can determine whether the reasoning action is feasible according to the guidance in the first prompt word, and give a first judgment result of "Yes" or "No".

[0098] If the first judgment result is "Yes" and the current reasoning action is an action at the thinking level, it indicates that the reasoning action can significantly optimize the reasoning path; if the first judgment result is "Yes" and the current reasoning action is an action at the action level, it indicates that the reasoning action can provide more clues for reasoning or obtain the final answer; if the first judgment result is "No" and the current reasoning action is an action at the thinking level, it indicates that the reasoning action cannot significantly optimize the reasoning path; if the first judgment result is "No" and the current reasoning action is an action at the action level, it indicates that the reasoning action cannot provide more clues for reasoning or obtain the final answer.

[0099] After the first prompt word is input into the visual language model, the output length of the visual language model can be set to 1. When the first judgment result is the first preset result, the model output value corresponding to the visual language model is used as the feasibility parameter of the reasoning action, wherein the first preset result can be set according to actual needs. In an embodiment of the present invention, the first preset result is "Yes". The model output value refers to the original output of the last layer of the visual language model, which has not been processed by the activation function, and is also called logits. It can be used to characterize the original confidence of each category. In an embodiment of the present invention, the model output value of the first judgment result of "Yes" is directly obtained as the feasibility parameter of the reasoning action.

[0100] According to the above method, a corresponding first prompt word can be generated for each reasoning action, and the first prompt word can be processed using the visual language model to obtain a feasibility parameter corresponding to the reasoning action. The preset parameter is a feasibility threshold set according to actual needs. In the embodiment of the present invention, the preset parameter can be set to 0.7. If the feasibility parameter of the reasoning action is not less than the preset parameter, it can be considered that the currently selected reasoning action is feasible under the existing visual information. If the feasibility parameter of the reasoning action is less than the preset parameter, it can be considered that the currently selected reasoning action is not feasible under the existing visual information. For example, a robot needs to be controlled to grab object A, but object A does not exist in the current visual information.

[0101] When generating an extension node, only the reasoning actions whose feasibility parameters are not less than the preset value are selected to generate the extension node of the current node. Specifically, the reasoning actions whose feasibility parameters are not less than the preset value can be regarded as extension nodes, and based on the state parameters of the current node, the visual language model is used to execute the reasoning actions corresponding to the extension node to generate the state parameters corresponding to the extension node. Each time the visual language model generates an extension node, the rationality parameters of the extension node can be calculated to measure the rationality of the corresponding action.

[0102] When calculating the rationality parameters of the extended node, for each extended node, the extended node and the preceding node path may be used as the designated node path; the designated node path is filled into the second prompt word template to obtain the second prompt word; the designated node path and the image to be inferred are filled into the third prompt word template to obtain the third prompt word; the logical consistency parameters of the extended node are calculated according to the second prompt word using the visual language model, and the visual association parameters of the extended node are calculated according to the third prompt word; and the rationality parameters of the extended node are calculated using the logical consistency parameters and the visual association parameters.

[0103] For each expansion node, the expansion node and the path of the previous node can be used as the designated node path. The designated node path represents the reasoning path up to the expansion node. The designated node path can be obtained by adding the expansion node to the end of its previous node path. For example, see Figure 4 , when the current node is node 3, the previous node path is node 1-node 3, and the expanded node is node 8, the specified node path is node 1-node 3-node 8.

[0104] The second prompt word template is a prompt word template used to calculate the logical consistency parameter, which may include a first specified slot, and the first specified slot can be used to fill the specified node path. Specifically, the second prompt word template is as follows:

[0105] prompt2 = f"""

[0106] You are an expert in logical reasoning evaluation. Please carefully analyze the given Chain of Thought (COT) reasoning steps and determine whether they are logically consistent. Your answer should be "Yes" or "No".

[0107] # Reasoning steps:

[0108] - step1: {cot_step1}

[0109] - step2: {cot_step2} ......

[0110] - stepN: {cot_stepn}

[0111] """

[0112] Among them, step1 to stepN are the first designated slots, and the designated node path can be filled into the first designated slot to obtain the second prompt word.

[0113] The third prompt word template is a prompt word template used to calculate visual association parameters. The third prompt word template may include a second designated slot and an image slot. The third prompt word template is specifically as follows:

[0114] Prompt3 = f"""

[0115] Please observe the given image carefully and evaluate whether the following reasoning steps are based on the provided image content. Determine whether the entire reasoning process can be supported by visual evidence in the image. If any step of reasoning introduces information that is not shown in the image, output "no", otherwise output "yes".

[0116]

[0117] {image}

[0118]

[0119] ### Reasoning steps:

[0120] {COT}

[0121] """

[0122] Among them, {COT} is the second designated slot, which can be used to fill the specified node path, {image} is the image slot, which can be used to fill the image to be inferred, or the image features of the image to be inferred, or the image features in the state parameters of the current node. After filling the corresponding content into the corresponding slot in the third prompt word template, the third prompt word can be obtained.

[0123] The generated second prompt word and third prompt word are respectively input into the visual language model, so that the visual language model can calculate the logical consistency parameter of the extended node according to the guidance of the second prompt word, and calculate the visual association parameter of the extended node according to the guidance of the third prompt word.

[0124] According to the guidance of the second prompt word, the answer output by the visual language model is Yes or No. The length of the visual language model can be set to 1, and the model output value corresponding to the answer Yes is obtained as the logical consistency parameter. According to the guidance of the third prompt word, the answer output by the visual language model is Yes or No. The length of the visual language model can be set to 1, and the model output value corresponding to the answer Yes is obtained as the visual association parameter.

[0125] Then, for an extended node, the average value between its corresponding logical consistency parameter and visual association parameter can be calculated to obtain the rationality parameter of the extended node. The rationality parameter is obtained by evaluating the extended node by the visual language model in the process of generating the target reasoning tree. It can dynamically affect the update process of MCTS, reduce the weight of unreasonable reasoning paths, and give priority to exploring more reasonable paths to improve the quality and reliability of the reasoning chain.

[0126] The extended node can be used as a new candidate node and returned to perform selection processing until the specified condition is met to generate a simulated reasoning path. The specified condition can be set according to actual needs. In the embodiment of the present invention, the specified condition can be that the response content corresponding to the data to be inferred has been obtained or the path has reached a specified length.

[0127] After the simulated reasoning path is generated, all model output values ​​related to the data to be inferred generated by the visual language model in the process of generating the simulated reasoning path can be obtained, and the simulated reward value of the simulated reasoning path can be calculated based on the obtained model output values.

[0128] Among them, the simulated reward value can be calculated from two parts of parameters, one part is the model output value of the visual language model, and the other part is the rationality parameter of the node. Specifically, when calculating the simulated reward value of the simulated reasoning path, it can be to calculate the average value of all model output values ​​related to the data to be inferred in the process of generating the simulated reasoning path by the visual language model as the first reward value; calculate the average value of the rationality parameters of all nodes in the simulated reasoning path to obtain the second reward value; sum the first reward value and the second reward value to obtain the simulated reward value of the simulated reasoning path.

[0129] It should be noted that in the process of generating the simulated reasoning path, multiple selection processes and expansion processes are performed. In each expansion process, a corresponding reasoning action may be performed based on the state parameters of the current node to generate an extended node and the state parameters corresponding to the extended node. In the process of generating state parameters, multiple reasoning tokens can be generated, and the model output values ​​corresponding to all reasoning tokens can be obtained, and the average model output value can be calculated as the first reward value.

[0130] The simulated reasoning path is a node path composed of the current nodes determined in sequence. The simulated reasoning path may contain multiple nodes. In the aforementioned expansion process, the rationality parameter is calculated for each expansion node, and each expansion node may be a node in the simulated reasoning path. Therefore, each node in the simulated reasoning path has a corresponding rationality parameter. The average value of the rationality parameters of all nodes in the simulated reasoning path can be calculated as the second reward value. Finally, the first reward value and the second reward value are summed to obtain the simulated reward value of the simulated reasoning path.

[0131] After calculating the simulated reward value, the simulated reward value can be used to update the reward parameters and access parameters of all nodes in the simulated reasoning path to obtain a new initial reasoning tree. Specifically, for each node in the simulated reasoning path, the simulated reward value can be added to the reward parameter of the node to obtain the updated reward parameter corresponding to the node, and 1 can be added to the number of visits of the node to obtain the updated number of visits. After updating the reward parameters and access parameters of each node in the simulated reasoning path, the current execution number of the extended update processing can be added by 1 to obtain the updated current execution number.

[0132] If the current execution times after update are less than the preset times, the process returns to the initial node and executes according to the above steps until the current execution times of the extended update process reach the preset times, thereby obtaining the target reasoning tree.

[0133] S140: Determine a candidate reasoning path from the reasoning paths of the target reasoning tree based on the rationality parameters of all end nodes in the target reasoning tree.

[0134] The generated target reasoning tree contains multiple nodes, and each node has its corresponding rationality parameters and state parameters. Among them, the end node refers to the node in the target reasoning tree that can obtain the response data corresponding to the data to be inferred, that is, the state parameter of the end node can include the response data corresponding to the data to be inferred. The reasoning path refers to the node path from the initial node to the end node in the target reasoning tree, which may include the initial node, the intermediate node and the end node. The intermediate node is the node in the reasoning path except the initial node and the end node. Using the rationality parameters of the end node, the candidate reasoning path can be determined from the reasoning path in the target reasoning tree for subsequent use.

[0135] As an implementation method, all end nodes in the target reasoning tree may be obtained; end nodes whose rationality parameters are greater than a rationality threshold are taken as candidate end nodes; in the target reasoning tree, all node paths from the initial node to the candidate end node are obtained to obtain candidate reasoning paths.

[0136] The rationality threshold can be set in advance according to actual needs. If the rationality parameter is greater than the rationality threshold, the end node can be considered to be more reasonable. Only the nodes from the initial node to the candidate end node are used as candidate reasoning paths. Unreasonable end nodes can be removed in advance to reduce the subsequent data processing volume.

[0137] S150: Determine a target reasoning path from the candidate reasoning paths according to the consistency parameters of the intermediate nodes in the candidate reasoning paths and the rationality parameters of the end nodes.

[0138] The candidate reasoning path can be considered as a node path from the initial node to the candidate end node. The other nodes in the candidate reasoning path except the initial node and the candidate end node can be recorded as intermediate nodes. Using the rationality parameters of the intermediate nodes and the end nodes in the candidate reasoning path, the target reasoning path can be screened out from the candidate reasoning paths.

[0139] In some implementations, the cosine similarity between any two intermediate nodes in all candidate reasoning paths may be calculated; all intermediate nodes may be clustered to obtain node clusters; for each of the node clusters, the consistency parameter of each intermediate node in the node cluster may be calculated based on the cosine similarity between any two intermediate nodes in the node cluster; for each of the candidate reasoning paths, the path parameter corresponding to the candidate reasoning path may be calculated using the consistency parameter of each intermediate node in the candidate reasoning path and the rationality parameter of the end node; and the candidate reasoning path corresponding to the largest path parameter may be used as the target reasoning path.

[0140] There may be multiple candidate reasoning paths, and each candidate reasoning path may contain multiple intermediate nodes. By comparing these intermediate nodes, if some intermediate nodes appear frequently in multiple candidate reasoning paths, then these nodes may be correct or key nodes leading to the correct answer. If the intermediate nodes in some candidate reasoning paths are unique or conflict with other paths, it indicates that these paths may be wrong or have low confidence. Therefore, reliable target reasoning paths can be screened out from the candidate reasoning paths by comparing and counting the consistency between all intermediate nodes.

[0141] All intermediate nodes in all candidate reasoning paths can be obtained, and all intermediate nodes can be vectorized using a vectorization model to obtain the node vectors corresponding to the intermediate nodes; then the node vectors are used to calculate the cosine similarity between any two intermediate nodes. All intermediate nodes can be clustered and divided into clusters. Similar intermediate nodes can be divided into the same cluster, and the divided clusters can be recorded as node clusters. Among them, the specific clustering algorithm used in the clustering process can be set according to actual needs, such as k-means clustering, DBSCAN clustering, etc.

[0142] For the intermediate nodes contained in the node cluster, multiple node groups can be obtained by combining two of them, and each node group contains two different intermediate nodes in the node cluster. For each node group, the cosine similarity between the two intermediate nodes can be obtained as the similarity value of the node group, and then the average similarity value of all node groups in the node cluster is calculated as the consistency parameter of each intermediate node in the node cluster.

[0143] For each candidate reasoning path, the consistency parameter of each intermediate node in the candidate reasoning path can be obtained. Using the consistency parameter of each intermediate node in the candidate reasoning path and the rationality parameter of the end node, the path parameter corresponding to the candidate reasoning path can be calculated. The path parameter can be used to measure the quality of the candidate reasoning path. The higher the path parameter, the higher the quality of the candidate reasoning path and the more reliable it is.

[0144] Optionally, when calculating the path parameters of a candidate reasoning path, for each candidate reasoning path, the sum of the consistency parameters of all intermediate nodes in the candidate reasoning path is divided by the number of intermediate nodes in the candidate reasoning path to obtain an average consistency parameter of the candidate reasoning path; the average consistency parameter is weighted with a first weight to obtain a first parameter; the rationality parameters of the end nodes in the candidate reasoning path are weighted with a second weight to obtain a second parameter; and the first parameter and the second parameter are summed to obtain the path parameter corresponding to the candidate reasoning path.

[0145] For each candidate reasoning path, the consistency parameters of all intermediate nodes in the candidate reasoning path can be obtained and summed, and the summed value is divided by the number of intermediate nodes in the candidate reasoning path to obtain the average consistency parameter of the candidate reasoning path. The first weight is a custom weight related to the consistency parameter, and the second weight is a custom weight related to the rationality parameter. The sum of the first weight and the second weight is 1. The first parameter can be obtained by weighting the average consistency parameter with the first weight; the second parameter can be obtained by weighting the rationality parameter with the second weight. Finally, the first parameter and the second parameter are summed to obtain the path parameter of the candidate reasoning path.

[0146] According to the above method, the path parameter corresponding to each reasoning path can be calculated, and the candidate reasoning path corresponding to the maximum path parameter can be determined as the target reasoning path.

[0147] S160, controlling the visual language model to output target response data corresponding to the data to be inferred according to the target reasoning path.

[0148] After determining the target reasoning path, the visual language model is directly controlled to reason about the data to be reasoned according to the target reasoning path to obtain the target response data corresponding to the data to be reasoned.

[0149] The model reasoning scheme based on the visual language model provided by the embodiment of the present invention can be applied to various scenarios for reasoning on multimodal data. For example, in a visual question-answering scenario, the visual language model can gradually reason and give answers based on images and questions, reduce model hallucinations, and improve the accuracy of answers. In a robot intelligent control scenario, the visual language model can plan the steps to achieve its task based on the captured visual information and task description and complete the task. In the image description generation task, the visual language model can generate a detailed description based on the image content and explain the reasoning process in detail.

[0150] The method provided by the embodiment of the present invention can introduce the MCTS algorithm in the reasoning of the visual language model so that it can adapt to tasks that require multi-step reasoning and complex logic. In the expansion stage of MCTS, the visual information is used to screen the reasoning actions, which can avoid introducing infeasible reasoning actions and reduce the data processing volume of subsequent processing. After obtaining the target reasoning tree, the rationality parameters of the end node can be used to select candidate reasoning paths with reasonable and reliable answers, and then based on the logical consistency of the reasoning steps in the candidate reasoning paths, the target reasoning path with higher reasoning quality is selected. Finally, the target reasoning path is used to output the target response data, which can effectively improve the reasoning efficiency and accuracy of the visual language model in complex reasoning tasks.

[0151] In order to better implement the above method, the embodiment of the present invention also provides a model reasoning device based on a visual language model, which can be integrated in an electronic device, and the electronic device can be a terminal, a server, etc. Among them, the terminal can be a mobile phone, a tablet computer, a smart Bluetooth device, a laptop, a personal computer, etc.; the server can be a single server or a server cluster composed of multiple servers.

[0152] For example, in this embodiment, the method of the embodiment of the present invention is described in detail by taking the specific integration of a model reasoning device based on a visual language model in a server as an example.

[0153] For example, Figure 5 As shown, the model reasoning device 200 based on the visual language model may include an acquisition module 210 , an initial module 220 , an expansion and update module 230 , a candidate determination module 240 , a target determination module 250 and a reasoning module 260 .

[0154] An acquisition module 210 is used to acquire data to be inferred, where the data to be inferred includes text to be inferred and images to be inferred;

[0155] An initial module 220, configured to encode the text to be inferred and the image to be inferred by using a visual language model to generate state parameters of an initial node in an initial inference tree;

[0156] An expansion and updating module 230 is used to perform expansion and updating processing on the nodes of the initial reasoning tree based on the feasibility parameters of the reasoning action in the image to be reasoned and the state parameters of the current node to obtain a target reasoning tree, wherein the current node takes the initial node as the starting point of the change, and each node in the target reasoning tree has corresponding state parameters and rationality parameters;

[0157] A candidate determination module 240, configured to determine a candidate reasoning path from the reasoning path of the target reasoning tree based on the rationality parameters of all the end nodes in the target reasoning tree, the state parameters of the end nodes including the response data corresponding to the data to be reasoned, and the reasoning path including the initial node, the intermediate node and the end node;

[0158] A target determination module 250, configured to determine a target reasoning path from the candidate reasoning paths according to consistency parameters of the intermediate nodes in the candidate reasoning paths and rationality parameters of the end nodes;

[0159] The reasoning module 260 is used to control the visual language model to output target response data corresponding to the data to be inferred according to the target reasoning path.

[0160] In some embodiments, the node also has corresponding access parameters and reward parameters, and the extended update module 230 is specifically used to:

[0161] selecting a current node from the candidate nodes based on an exploration parameter, a visit parameter of the candidate nodes, and a reward parameter, wherein the exploration parameter is related to a current number of executions of the extended update process;

[0162] Generate an extended node of the current node and calculate a rationality parameter of the extended node by using a feasibility parameter of the inference action in the image to be inferred and a state parameter of the current node;

[0163] The expanded node is used as a new candidate node, and execution is returned to explore parameters, access parameters of the candidate nodes, and reward parameters, and the step of selecting the current node and subsequent steps from the candidate nodes until the specified conditions are met, thereby generating a simulated reasoning path;

[0164] Calculating a simulated reward value of the simulated reasoning path by using all model output values ​​related to the data to be inferred generated by the visual language model in the process of generating the simulated reasoning path;

[0165] updating the reward parameters and access parameters of all nodes in the simulated reasoning path with the simulated reward value, obtaining a new initial reasoning tree, and updating the current execution count;

[0166] Return execution with the new initial inference tree to explore parameters, access parameters of candidate nodes and reward parameters, select the step of the current node and subsequent steps from the candidate nodes, until the current execution times reaches a preset number, to obtain a target inference tree.

[0167] In some embodiments, the extension update module 230 is specifically used to:

[0168] Generate a question summary corresponding to the text to be inferred and an image description corresponding to the image to be inferred using a visual language model;

[0169] Obtaining a preceding node path from the initial node to the current node;

[0170] For each of the reasoning actions, the feasibility parameter of the reasoning action is calculated based on the image description, the problem summary, the preceding node path, and the image to be reasoned;

[0171] Based on the state parameter of the current node, executing the reasoning action with the feasibility parameter not less than the preset parameter to generate an extended node of the current node;

[0172] For each of the extended nodes, the rationality parameter of the extended node is calculated using the preceding node path, the extended node and the image to be inferred.

[0173] In some embodiments, the extension update module 230 is specifically used to:

[0174] Fill the image description, the problem summary, the preceding node path, the image to be inferred, and the inference action into a first prompt word template to generate a first prompt word;

[0175] Using a visual language model to determine whether the reasoning action is feasible according to the first prompt word, to obtain a first determination result;

[0176] If the first determination result is a first preset result, a model output value corresponding to the first determination result is obtained as a feasibility parameter of the reasoning action.

[0177] In some embodiments, the extension update module 230 is specifically used to:

[0178] For each of the extended nodes, the extended node and the preceding node path are used as designated node paths;

[0179] Filling the designated node path into the second prompt word template to obtain a second prompt word;

[0180] Filling the designated node path and the image to be inferred into a third prompt word template to obtain a third prompt word;

[0181] Using a visual language model to calculate a logical consistency parameter of the extended node according to the second prompt word, and calculating a visual association parameter of the extended node according to the third prompt word;

[0182] The rationality parameter of the extended node is calculated using the logical consistency parameter and the visual association parameter.

[0183] In some embodiments, the access parameter of the candidate node includes the number of accesses to the candidate node and the number of accesses to the parent node of the candidate node. The extended update module 230 is specifically used to:

[0184] Calculating a first ratio using the reward parameter of the candidate node and the number of visits to the candidate node;

[0185] Calculating a second ratio based on the number of visits to the parent node of the candidate node and the number of visits to the candidate node;

[0186] Calculating a ratio between a current execution count of the extended update process and the preset count;

[0187] Calculate the exploration parameters under the current number of executions based on the value range of the exploration weight and the number ratio;

[0188] weighting the second ratio by using the exploration parameter, and summing the weighted result and the first ratio to obtain the value parameter of the candidate node;

[0189] The value parameter of each candidate node is used to select a current node from the candidate nodes.

[0190] In some embodiments, the extension update module 230 is specifically used to:

[0191] Calculating an average value of all model output values ​​related to the to-be-inferred data in a process of generating the simulated reasoning path by the visual language model as a first reward value;

[0192] Calculating an average value of rationality parameters of all nodes in the simulated reasoning path to obtain a second reward value;

[0193] The first reward value and the second reward value are summed to obtain a simulated reward value of the simulated reasoning path.

[0194] In some embodiments, the target determination module 250 is specifically configured to:

[0195] For all intermediate nodes in all candidate reasoning paths, calculate the cosine similarity between any two intermediate nodes;

[0196] Cluster all intermediate nodes to obtain node clusters;

[0197] For each of the node clusters, according to the cosine similarity between any two intermediate nodes in the node cluster, calculating the consistency parameter of each intermediate node in the node cluster;

[0198] For each of the candidate reasoning paths, using the consistency parameter of each intermediate node and the rationality parameter of the end node in the candidate reasoning path, calculate the path parameter corresponding to the candidate reasoning path;

[0199] The candidate reasoning path corresponding to the largest path parameter is taken as the target reasoning path.

[0200] In some embodiments, the target determination module 250 is specifically configured to:

[0201] For each of the candidate reasoning paths, the sum of the consistency parameters of all the intermediate nodes in the candidate reasoning path is divided by the number of the intermediate nodes in the candidate reasoning path to obtain an average consistency parameter of the candidate reasoning path;

[0202] Performing weighted processing on the average consistency parameter with a first weight to obtain a first parameter;

[0203] weighting the rationality parameters of the end nodes in the candidate reasoning path with a second weight to obtain a second parameter;

[0204] The first parameter and the second parameter are summed to obtain a path parameter corresponding to the candidate reasoning path.

[0205] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above modules can refer to the previous method embodiments, which will not be repeated here.

[0206] From the above, it can be seen that the model reasoning device based on the visual language model of this embodiment can use the feasibility parameters of the reasoning action in the image to be reasoned and the state of the current node to expand and update the nodes in the initial reasoning tree, which can avoid introducing unreasonable nodes in the expansion, reduce the amount of data processing, and quickly generate a target reasoning tree. In the target reasoning tree, based on the rationality parameters of the end node, the candidate reasoning paths with reliable results are screened out, and the consistency parameters of the intermediate nodes in the candidate reasoning paths and the rationality parameters of the end nodes are used to comprehensively evaluate the quality of the candidate reasoning paths, ensuring that the target reasoning path with a reliable reasoning process can be screened out for reasoning, thereby ensuring that the visual language model can perform reasoning efficiently and accurately in complex reasoning scenarios.

[0207] The embodiment of the present invention further provides an electronic device, which may be a terminal, a server, or the like. The terminal may be a mobile phone, a tablet computer, a smart Bluetooth device, a notebook computer, a personal computer, or the like; the server may be a single server or a server cluster composed of multiple servers, or the like.

[0208] In some embodiments, the model reasoning device based on the visual language model can also be integrated into multiple electronic devices. For example, the model reasoning device based on the visual language model can be integrated into multiple servers, and the model reasoning method based on the visual language model of the present invention can be implemented by multiple servers.

[0209] In this embodiment, the electronic device of this embodiment is a server as an example for detailed description, for example, Figure 6 As shown, it shows a schematic diagram of the structure of an electronic device involved in an embodiment of the present invention, specifically:

[0210] The electronic device may include components such as a processor 310 with one or more processing cores, a memory 320 with one or more computer-readable storage media, a power supply 330, an input module 340, and a communication module 350. Those skilled in the art will appreciate that Figure 6 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0211] The processor 310 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory 320, and calling data stored in the memory 320. In some embodiments, the processor 310 may include one or more processing cores; in some embodiments, the processor 310 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 310.

[0212] The memory 320 can be used to store software programs and modules. The processor 310 executes various functional applications and data processing by running the software programs and modules stored in the memory 320. The memory 320 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 320 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 320 may also include a memory controller to provide the processor 310 with access to the memory 320.

[0213] The electronic device also includes a power supply 330 for supplying power to various components. In some embodiments, the power supply 330 can be logically connected to the processor 310 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 330 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0214] The electronic device may further include an input module 340, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

[0215] The electronic device may further include a communication module 350. In some embodiments, the communication module 350 may include a wireless module. The electronic device may perform short-range wireless transmission through the wireless module of the communication module 350, thereby providing the user with wireless broadband Internet access. For example, the communication module 350 may be used to help the user send and receive emails, browse web pages, and access streaming media.

[0216] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 310 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 320 according to the following instructions, and the processor 310 will run the application programs stored in the memory 320, thereby implementing the steps in the methods of the embodiments of the present invention.

[0217] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0218] From the above, it can be seen that the electronic device provided by the embodiment of the present invention can use the feasibility parameters of the reasoning action in the image to be inferred and the state of the current node to expand and update the nodes in the initial reasoning tree, which can avoid introducing unreasonable nodes in the expansion, reduce the amount of data processing, and quickly generate a target reasoning tree. In the target reasoning tree, based on the rationality parameters of the end node, candidate reasoning paths with reliable results are screened out, and the consistency parameters of the intermediate nodes in the candidate reasoning paths and the rationality parameters of the end nodes are used to comprehensively evaluate the quality of the candidate reasoning paths, ensuring that a target reasoning path with a reliable reasoning process can be screened out for reasoning, thereby ensuring that the visual language model can perform reasoning efficiently and accurately in complex reasoning scenarios.

[0219] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0220] To this end, an embodiment of the present invention provides a computer-readable storage medium, in which a plurality of instructions are stored. The instructions can be loaded by a processor to execute the steps in any one of the visual language model-based model methods provided by the embodiments of the present invention.

[0221] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0222] According to one aspect of the present invention, a computer program product or computer program is provided, the computer program product or computer program including a computer program / instruction, the computer program / instruction being stored in a computer-readable storage medium. A processor of an electronic device reads the computer program / instruction from the computer-readable storage medium, and the processor executes the computer program / instruction, so that the electronic device executes the method provided in various optional implementations provided in the above embodiments.

[0223] Since the instructions stored in the storage medium can execute the steps in any one of the model methods based on the visual language model provided in the embodiments of the present invention, the beneficial effects that can be achieved by any one of the model methods based on the visual language model provided in the embodiments of the present invention can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0224] The above is a detailed introduction to a model method and device based on a visual language model provided by an embodiment of the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A model reasoning method based on a visual language model, characterized in that: The method comprises: Acquire data to be inferred, wherein the data to be inferred includes text to be inferred and images to be inferred; Encoding the text to be inferred and the image to be inferred by using a visual language model to generate state parameters of an initial node in an initial inference tree; Based on the feasibility parameter of the reasoning action in the image to be reasoned and the state parameter of the current node, the nodes of the initial reasoning tree are extended and updated to obtain a target reasoning tree, wherein the current node takes the initial node as the starting point of change, and each node in the target reasoning tree has a corresponding state parameter and a rationality parameter, wherein the rationality parameter is a parameter obtained after evaluating the rationality of executing the reasoning action, wherein the reasoning action includes a preset action of reasoning on multimodal data, and the feasibility parameter is used to measure the feasibility of executing the reasoning action based on the image to be reasoned; Based on the rationality parameters of all the end nodes in the target inference tree, determine a candidate inference path from the inference path of the target inference tree, the state parameter of the end node includes the response data corresponding to the data to be inferred, and the inference path includes the initial node, the intermediate node and the end node; Determine a target reasoning path from the candidate reasoning paths according to consistency parameters of the intermediate nodes in the candidate reasoning paths and rationality parameters of the end nodes, wherein the consistency parameters are calculated based on the cosine similarity between any two intermediate nodes in a node cluster, and the node cluster is obtained by clustering the intermediate nodes in all candidate reasoning paths; The visual language model is controlled to output target response data corresponding to the data to be inferred according to the target reasoning path.

2. The method according to claim 1, characterized in that The node also has corresponding access parameters and reward parameters. The feasibility parameters of the reasoning action in the image to be reasoned and the state parameters of the current node are used to expand and update the nodes of the initial reasoning tree to obtain a target reasoning tree, including: selecting a current node from the candidate nodes based on an exploration parameter, a visit parameter of the candidate nodes, and a reward parameter, wherein the exploration parameter is related to a current number of executions of the extended update process; Generate an extended node of the current node and calculate a rationality parameter of the extended node by using a feasibility parameter of the inference action in the image to be inferred and a state parameter of the current node; The expanded node is used as a new candidate node, and execution is returned to explore parameters, access parameters of the candidate nodes, and reward parameters, and the step of selecting the current node and subsequent steps from the candidate nodes until the specified conditions are met, thereby generating a simulated reasoning path; Calculating a simulated reward value of the simulated reasoning path by using all model output values ​​related to the data to be inferred generated by the visual language model in the process of generating the simulated reasoning path; updating the reward parameters and access parameters of all nodes in the simulated reasoning path with the simulated reward value, obtaining a new initial reasoning tree, and updating the current execution count; Return execution with the new initial inference tree to explore parameters, access parameters of candidate nodes and reward parameters, select the step of the current node and subsequent steps from the candidate nodes, until the current execution times reaches a preset number, to obtain a target inference tree.

3. The method according to claim 2, characterized in that The method of using the feasibility parameter of the reasoning action in the image to be reasoned and the state parameter of the current node to generate an extended node of the current node and calculate the rationality parameter of the extended node includes: Generate a question summary corresponding to the text to be inferred and an image description corresponding to the image to be inferred using a visual language model; Obtaining a preceding node path from the initial node to the current node; For each of the reasoning actions, the feasibility parameter of the reasoning action is calculated based on the image description, the problem summary, the preceding node path, and the image to be reasoned; Based on the state parameter of the current node, executing the reasoning action with the feasibility parameter not less than the preset parameter to generate an extended node of the current node; For each of the extended nodes, the rationality parameter of the extended node is calculated using the preceding node path, the extended node and the image to be inferred.

4. The method according to claim 3, characterized in that The calculating the feasibility parameter of the reasoning action based on the image description, the problem summary, the preceding node path and the image to be reasoned includes: Fill the image description, the problem summary, the preceding node path, the image to be inferred, and the inference action into a first prompt word template to generate a first prompt word; Using a visual language model to determine whether the reasoning action is feasible according to the first prompt word, to obtain a first determination result; If the first determination result is a first preset result, a model output value corresponding to the first determination result is obtained as a feasibility parameter of the reasoning action.

5. The method according to claim 3, characterized in that: The calculating the rationality parameter of the extended node by using the preceding node path, the extended node and the image to be inferred includes: For each of the extended nodes, the extended node and the preceding node path are used as designated node paths; Filling the designated node path into the second prompt word template to obtain a second prompt word; Filling the designated node path and the image to be inferred into a third prompt word template to obtain a third prompt word; Using a visual language model to calculate a logical consistency parameter of the extended node according to the second prompt word, and calculating a visual association parameter of the extended node according to the third prompt word; The rationality parameter of the extended node is calculated using the logical consistency parameter and the visual association parameter.

6. The method according to claim 2, characterized in that The access parameters of the candidate nodes include the number of visits to the candidate nodes and the number of visits to the parent node of the candidate nodes. The current node is selected from the candidate nodes based on the exploration parameters, the access parameters of the candidate nodes and the reward parameters, including: Calculating a first ratio using the reward parameter of the candidate node and the number of visits to the candidate node; Calculating a second ratio based on the number of visits to the parent node of the candidate node and the number of visits to the candidate node; Calculating a ratio between a current execution count of the extended update process and the preset count; Calculate the exploration parameters under the current number of executions based on the value range of the exploration weight and the number ratio; weighting the second ratio by using the exploration parameter, and summing the weighted result and the first ratio to obtain the value parameter of the candidate node; The value parameter of each candidate node is used to select a current node from the candidate nodes.

7. The method according to claim 2, characterized in that: The step of calculating the simulated reward value of the simulated reasoning path by using all model output values ​​related to the data to be inferred generated by the visual language model in the process of generating the simulated reasoning path includes: Calculating an average value of all model output values ​​related to the to-be-inferred data in a process of generating the simulated reasoning path by the visual language model as a first reward value; Calculating an average value of rationality parameters of all nodes in the simulated reasoning path to obtain a second reward value; The first reward value and the second reward value are summed to obtain a simulated reward value of the simulated reasoning path.

8. The method according to claim 1, characterized in that Determining a target reasoning path from the candidate reasoning paths according to the consistency parameters of the intermediate nodes in the candidate reasoning paths and the rationality parameters of the end nodes includes: For all intermediate nodes in all candidate reasoning paths, calculate the cosine similarity between any two intermediate nodes; Cluster all intermediate nodes to obtain node clusters; For each of the node clusters, according to the cosine similarity between any two intermediate nodes in the node cluster, calculating the consistency parameter of each intermediate node in the node cluster; For each of the candidate reasoning paths, using the consistency parameter of each intermediate node and the rationality parameter of the end node in the candidate reasoning path, calculate the path parameter corresponding to the candidate reasoning path; The candidate reasoning path corresponding to the largest path parameter is taken as the target reasoning path.

9. The method according to claim 8, characterized in that The step of calculating the path parameter corresponding to each candidate reasoning path by using the consistency parameter of each intermediate node and the rationality parameter of the end node in the candidate reasoning path for each candidate reasoning path includes: For each of the candidate reasoning paths, the sum of the consistency parameters of all the intermediate nodes in the candidate reasoning path is divided by the number of the intermediate nodes in the candidate reasoning path to obtain an average consistency parameter of the candidate reasoning path; Performing weighted processing on the average consistency parameter with a first weight to obtain a first parameter; weighting the rationality parameters of the end nodes in the candidate reasoning path with a second weight to obtain a second parameter; The first parameter and the second parameter are summed to obtain a path parameter corresponding to the candidate reasoning path.

10. A model reasoning device based on a visual language model, the device being used to implement the method according to any one of claims 1 to 9, characterized in that: The device comprises: An acquisition module, used to acquire data to be inferred, wherein the data to be inferred includes text to be inferred and images to be inferred; An initial module, used to encode the text to be inferred and the image to be inferred through a visual language model to generate state parameters of an initial node in an initial inference tree; An extended update module, used to perform extended update processing on the nodes of the initial reasoning tree based on the feasibility parameters of the reasoning action in the image to be reasoned and the state parameters of the current node to obtain a target reasoning tree, wherein the current node takes the initial node as the starting point of change, and each node in the target reasoning tree has a corresponding state parameter and a rationality parameter, wherein the rationality parameter is a parameter obtained after evaluating the rationality of executing the reasoning action, wherein the reasoning action includes a preset action of reasoning on multimodal data, and the feasibility parameter is used to measure the feasibility of executing the reasoning action based on the image to be reasoned; A candidate determination module, configured to determine a candidate reasoning path from the reasoning path of the target reasoning tree based on the rationality parameters of all the end nodes in the target reasoning tree, wherein the state parameters of the end nodes include the response data corresponding to the data to be inferred, and the reasoning path includes the initial node, the intermediate node and the end node; a target determination module, configured to determine a target reasoning path from the candidate reasoning paths according to consistency parameters of the intermediate nodes in the candidate reasoning paths and rationality parameters of the end nodes, wherein the consistency parameters are calculated based on the cosine similarity between any two intermediate nodes in a node cluster, and the node cluster is obtained by clustering the intermediate nodes in all candidate reasoning paths; The inference module is used to control the visual language model to output target response data corresponding to the data to be inferred according to the target inference path.

Images