Method for context problem reasoning based on large language model and related device

Abstract

A method for reasoning on situational problems based on a large language model and related equipment relate to the technical field of artificial intelligence, wherein the method comprises: obtaining a situational problem; inputting the situational problem to an agent to obtain a first answer output by the agent, the first answer being reasoned by a large language model integrated in the agent based on the situational problem; if it is determined that the first answer is incorrect, inputting the first answer to the agent to obtain a second answer output by the agent, the second answer being reasoned by the large language model after the agent reflects on the large language model based on the first answer or supplements knowledge of the large language model; and wherein the agent is trained based on sample situational problems. The technical solution provided by the present application can reflect and supplement the knowledge mechanism of the agent, thereby improving the accuracy and reliability of the answer to the situational problem.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a method for reasoning on situational problems based on a large language model and related equipment. BACKGROUND

[0002] A situational problem refers to a question raised in a specific environment or scenario, requiring the respondent to analyze and reason based on the given situational information to obtain an answer. Such problems are widely used in intelligent assistants, educational assessment, customer service, etc., aiming to test and improve the understanding and reasoning ability of artificial intelligence systems in complex scenarios.

[0003] Currently, the mainstream method for solving situational problems is to use a large language model. A large language model, through training on a large amount of text data, has obtained rich knowledge and strong language understanding ability. When facing a situational problem, a large language model can directly generate a corresponding answer based on the input question and situational description.

[0004] However, a large language model is prone to errors or inappropriate answers when facing complex or unfamiliar situations, and it is difficult to effectively self-correct, thereby affecting the accuracy of the answer to the situational problem. SUMMARY

[0005] The present application provides a method for reasoning on situational problems based on a large language model and related equipment, which can reflect on the agent's knowledge and supplement mechanism, thereby improving the accuracy and reliability of the answer to the situational problem.

[0006] In a first aspect of the present application, a method for reasoning on situational problems based on a large language model is provided, comprising:

[0007] Obtaining a situational problem;

[0008] Inputting the situational problem into an agent, obtaining a first answer output by the agent, the first answer being reasoned by a large language model integrated in the agent based on the situational problem;

[0009] If it is determined that the first answer is incorrect, inputting the first answer into the agent, obtaining a second answer output by the agent, the second answer being reasoned by the large language model after the agent reflects on the large language model or supplements knowledge based on the first answer;

[0010] Wherein, the agent is trained based on sample situational problems.

[0011] In a second aspect of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method for reasoning a situational question based on a large language model according to any of the above aspects when executing the computer program.

[0012] In a third aspect of the present application, a non-transitory computer-readable storage medium is provided, having stored thereon a computer program, wherein the computer program is executed by a processor to implement the method for reasoning a situational question based on a large language model according to any of the above aspects.

[0013] In a fourth aspect of the present application, a computer program product is provided, comprising a computer program, wherein the computer program is executed by a processor to implement the method for reasoning a situational question based on a large language model according to any of the above aspects.

[0014] In summary, the one or more technical solutions provided in the present application have at least the following technical effects or advantages:

[0015] By inputting the situational question into the agent, the large language model integrated in the agent is used to perform initial reasoning to obtain a first answer. When the first answer is determined to be incorrect, the agent can use the first answer as new input to perform reasoning reflection or knowledge supplement on the large language model, thereby generating a more accurate second answer.

[0016] The iterative reasoning process described above enables the agent to improve the accuracy of the answer by self-correction and knowledge updating when facing complex or unfamiliar situations. Compared with the method of directly generating an answer by a large language model in the prior art, the method of the present application can effectively deal with the case of incorrect initial answer, and significantly improves the accuracy and reliability of the answer to the situational question through the reflection and knowledge supplement mechanism of the agent. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings described below are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0018] Figure 1 is a flowchart of a method for reasoning a situational question based on a large language model provided by an embodiment of the present application.

[0019] Figure 2 is a flowchart of an agent training method provided by an embodiment of the present application.

[0020] Figure 3is a flowchart of a reflection module provided by an embodiment of the present application.

[0021] Figure 4 is a flowchart of a distillation module provided by an embodiment of the present application.

[0022] Figure 5 is a flowchart of a reasoning module provided by an embodiment of the present application.

[0023] Figure 6 is a context question answer result display diagram provided by an embodiment of the present application.

[0024] Figure 7 is a context question answer result display diagram provided by an embodiment of the present application.

[0025] Figure 8 is a structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the objectives, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below with reference to the drawings in the present application. Obviously, the described embodiments are some of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0027] The context question answering task requires an agent to understand the question and give a suitable answer in a specific environment context. This task is based on a scanned complex indoor scene in the real world, and the agent needs to understand and answer the question according to the layout of the environment, its own position, orientation and state, and other information. The difficulty of this task lies in the fact that the agent not only needs to understand semantic information, but also must combine visual and spatial information for comprehensive judgment.

[0028] At present, the common solution method usually relies on a Transformer-based backbone network, which can effectively encode visual information and text information and support an end-to-end training mode. In this method, the visual data and the text data are fused after being encoded, and finally an answer is generated through a prediction output head.

[0029] In addition, there are some methods that use pre-trained models to process context questions. Specifically, these methods use a large amount of multi-modal data in the pre-training phase, integrate three-dimensional environment data into the language model, and enable the model to have preliminary context understanding ability. After that, through fine-tuning, these pre-trained models can further adapt to specific tasks and generate more accurate answers.

[0030] Existing answering methods are mainly divided into two categories: closed-set answering and open-set answering. Closed-set answering methods require the agent to select the correct answer to the question from a predefined, finite set of answers. The advantage of this method lies in its controllability.

[0031] However, its limitation lies in the restricted range of answers, which may prevent it from handling questions beyond a predefined set. Open-ended answering methods, on the other hand, are not limited by a fixed list of answers, allowing agents to generate responses from an infinite or non-predefined answer space. This approach is more flexible but also more challenging because the process of generating answers does not involve explicit reasoning, resulting in weaker interpretability.

[0032] Based on the above, embodiments of the present invention provide a method for contextual problem reasoning based on a large language model to solve the problems existing in the prior art.

[0033] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating a method for contextual reasoning based on a large language model, provided by an embodiment of the present invention. This method can be implemented using a computer program, a microcontroller, or run on a von Neumann architecture-based system for contextual reasoning based on a large language model. The computer program can be integrated into an application or run as a standalone utility application. Specifically, the method may include the following steps:

[0034] S101, Obtaining the contextual question.

[0035] Contextual questions are those posed in a specific environment or scenario, requiring consideration of surrounding information to answer correctly. Contextual questions involve not only semantic understanding but also a comprehensive judgment combining spatial information and visual perception.

[0036] In this embodiment of the invention, the contextual problem includes a contextual scenario and a target problem. These two components together clearly define the specific task that the agent needs to solve, providing a clear target for the entire question-answering system.

[0037] Here, "contextual scenario" refers to a comprehensive description of the specific environment and state in which the intelligent agent is located. In the embodiments of this application, it can be understood as a detailed description of a three-dimensional indoor scene and a collection of state information such as the intelligent agent's position and orientation in that scene.

[0038] For example, a contextual scenario can include environmental information such as room layout, object distribution, and object attributes. This information collectively constitutes the background environment of the problem, providing the agent with the context needed to understand and answer the question. The contextual scenario serves to guide the perception module in focusing on key aspects and extracting and processing environmental information, while also providing the reasoning module with the foundational data for spatial reasoning and relational judgment.

[0039] Correspondingly, the target question refers to a specific inquiry that needs to be answered by the agent in a given situational scenario. In the embodiments of the present application, it can be understood as a specific question raised based on the current three-dimensional scene and the state of the agent, which may involve object recognition, spatial relationship judgment, attribute query and other aspects. The target question is usually presented in the form of natural language, and the agent needs to understand and answer it in combination with the situational scenario information.

[0040] S102, input the situational question to the agent, and obtain the first answer output by the agent, wherein the first answer is obtained by a large language model integrated in the agent based on the situational question reasoning, and the agent is trained based on sample situational questions.

[0041] Wherein, the agent refers to a computing system or program that can perceive the environment, make decisions and take actions to achieve specific goals. In the embodiments of the present application, it can be understood as a comprehensive artificial intelligence system integrating a large language model, a perception module, a reflection module, an extraction module and a reasoning module. This agent is designed to understand and answer complex situational questions, especially in three-dimensional indoor scene question answering tasks.

[0042] Further, the agent in the embodiments of the present application is a multi-module collaborative system. Its core is a ready-to-use large language model, which has a wide knowledge base and strong language understanding ability. Around this core, the agent also includes a specially designed perception module, a reflection module, an extraction module and a reasoning module. The perception module is responsible for analyzing and understanding the three-dimensional scene and extracting key environmental information. The reflection module can evaluate and improve the reasoning process and accumulate experience in solving problems. The extraction module extracts and summarizes general knowledge from historical data. The reasoning module integrates all the above information to generate the final answer.

[0043] On the basis of the above embodiments, as an optional embodiment, in step 102: input the situational question to the agent, and obtain the first answer output by the agent, this step can further include the following steps:

[0044] Step 201, input the situational scene to the subtitle model integrated in the agent, generate the text description of the key information of the object in the situational scene through the subtitle model, and input the text description to the large language model, wherein the subtitle model is obtained by training the agent based on sample situational questions.

[0045] The subtitle model refers to a machine learning model that can convert visual information into textual descriptions. In the embodiments of the present application, it can be understood as a neural network model specially designed for processing three-dimensional scene information. The subtitle model is usually based on deep learning techniques, such as convolutional neural networks, recurrent neural networks, or Transformer architecture. The core function of the subtitle model is to analyze complex spatial data, identify objects and their attributes in the scene, and generate structured text descriptions.

[0046] In the embodiments of the present application, the subtitle model first analyzes the input three-dimensional scene data, which may include point clouds, depth images or other forms of spatial representation. Through deep learning algorithms, the model can identify each object in the scene and extract their key features, such as color, shape, material and size, etc. At the same time, the subtitle model can also understand and describe the relative position and spatial relationship between objects, such as "on the left" or "above", etc.

[0047] Based on the extracted features and spatial relationship information, the subtitle model then generates accurate, concise and structured text descriptions. These descriptions not only contain the attribute information of the objects, but also include their position and relationship in the scene.

[0048] It is worth noting that since the subtitle model is trained based on sample context problems, it can extract and describe key information related to possible context problems, rather than simply describe all visible details.

[0049] The subtitle model plays a key role in bridging three-dimensional scene data and large language models in the present application. By converting complex spatial information into structured text descriptions, the subtitle model provides the necessary environmental information basis for subsequent problem understanding and answering, enabling large language models to better understand and handle context problems related to three-dimensional scenes.

[0050] The key information refers to the core data or features that are crucial for understanding and solving problems in a specific scene or problem context. In the embodiments of the present application, it can be understood as information such as scene elements, object attributes and spatial relationships that are closely related to context problems. These information is extracted from complex three-dimensional environments and plays a decisive role in accurately answering context problems.

[0051] Specifically, the key information can include the basic attributes of objects in the scene, such as color, shape, material, size, etc., as well as the spatial relationships between objects, such as relative position, direction, and distance. For example, in a living room scene, the color and position of the sofa, the size and orientation of the TV, and the relative position of the coffee table to the sofa can all be key information. The importance of the above information depends on the specific situational problem, such as asking "where is the red object in the room?" The color of the object becomes a key piece of information.

[0052] Step 202, input the target question to the large language model, and obtain a first answer by reasoning the target question based on the text description through the large language model.

[0053] In specific implementation, the system first converts the target question into a format suitable for processing by the large language model, which can be a piece of text. Then, this piece of text is provided as input to the large language model together with the scene text description generated in step S201. The large language model analyzes the semantics of the question, identifies keywords and query intent, and searches for relevant information in the scene description. Based on this information, the large language model activates its internal reasoning mechanism, which can include multi-step logical deduction, knowledge retrieval, and information integration.

[0054] For example, if the target question is "What is the color of the largest furniture in the room?", the large language model will first understand that the question is asking about the color of the furniture, and that it is specifically referring to the "largest" furniture. Then, it will search for information about the size and color of the furniture in the scene text description. By comparing the size information of different furniture, it determines the largest furniture, and then finds the color description of that furniture. Finally, the model generates an answer that meets the requirements of the question, i.e., the first answer.

[0055] S103, if it is determined that the first answer is incorrect, input the first answer to the intelligent agent, and obtain a second answer output by the intelligent agent, the second answer being obtained by the large language model based on the first answer reasoning and reflection or knowledge supplement of the large language model.

[0056] Specifically, when the large language model outputs "cannot answer" as the first answer, it is considered to be incorrect. The first answer error refers to the initial answer generated by the large language model that cannot correctly or completely answer the given situational question. In the embodiments of the present application, it can be understood that the answer output by the large language model does not match the expected correct answer, or the model directly indicates that it cannot answer the question. The first answer error is used to trigger the error correction mechanism of the intelligent agent, to start the reasoning reflection and knowledge supplement process to generate a more accurate second answer.

[0057] Further, the error can be caused by two main reasons: the error of the large language model in logical reasoning, or the insufficient amount of knowledge stored by the large language model.

[0058] The following two scenarios are illustrated by two practical examples:

[0059] (1) Logical reasoning error:

[0060] Suppose there is a living room scene, the question is: "If the sofa is red and the TV cabinet is brown, what color is the furniture closest to the window?" The scene description is: "In the living room, there is a red sofa placed against the wall, a brown TV cabinet opposite the sofa, and a white tea table between the sofa and the TV cabinet. Close to the window is a white bookshelf."

[0061] In the above case, the large language model may not be able to correctly handle the positional relationship and color information of multiple objects, leading to an incorrect conclusion. In fact, the correct reasoning should be: the bookshelf is closest to the window, and the bookshelf is white, so the answer should be "white".

[0062] (2) Lack of knowledge:

[0063] Suppose there is a kitchen scene, the question is: "What is the purpose of the device with a HEPA filter in the kitchen?" The scene description is: "In the corner of the kitchen, there is a square-shaped appliance with 'HEPA filter' marked on it. Next to it are a refrigerator and a microwave."

[0064] In the above case, the large language model lacks professional knowledge about HEPA filters. HEPA filters are usually used in air purifiers, but if the large language model does not have this knowledge, it cannot make a correct inference. The correct answer should be: "This device may be an air purifier, used to clean the air in the kitchen."

[0065] In implementation, the agent first analyzes the reasons for the first answer error, and the agent activates the corresponding error correction mechanism. If it is a reasoning logic error, the reflection module of the agent will be activated. The reflection module analyzes each step of the reasoning process, identifies the key points that may cause errors, and generates correction strategies. These strategies may include reorganizing the reasoning steps, adjusting the reasoning logic, or introducing additional intermediate reasoning steps. If it is a lack of knowledge, the agent will activate the refining module. The refining module retrieves relevant professional knowledge or common sense information from the pre-established knowledge database and integrates these information into the reasoning process.

[0066] After completing the reasoning reflection or knowledge supplement, the intelligent agent will generate a new, optimized prompt. This prompt contains the original situational question, the content of the first answer, the reasoning logic after reflection or the supplemented knowledge information. The intelligent agent then inputs this optimized prompt into the large language model, guiding the model to conduct more in-depth and accurate reasoning. Based on this optimized prompt, the large language model combines its inherent language understanding and generation capabilities to generate a second answer.

[0067] Based on the above embodiment, as an optional embodiment, in step S103: if it is determined that the first answer is incorrect, the first answer is input to the intelligent agent, and a second answer output by the intelligent agent is obtained. The step can further include the following steps:

[0068] Step 301: input the first answer to the intelligent agent, and match the first answer with the method database by calling the method database through the intelligent agent to determine whether there is a reasoning process with a similarity greater than a threshold, wherein the method database and the knowledge database are obtained by training the intelligent agent based on sample situational questions.

[0069] The method database refers to a structured storage system for saving and managing the reasoning methods and solution strategies used by the intelligent agent in processing situational questions. In the embodiment of the application, it can be understood as a data set containing a large number of reasoning process templates, each template corresponding to a specific type of situational question and its solution. These templates contain reasoning steps, logical structures, key decision points, etc., and are summaries of the intelligent agent's past successful problem-solving experiences. The method database is used to improve the efficiency and accuracy of the intelligent agent in processing new problems. By matching similar problem types, the intelligent agent can quickly call appropriate reasoning strategies, avoid repeated construction of reasoning processes, thereby speeding up problem solving and improving answer quality.

[0070] Correspondingly, the knowledge database refers to a comprehensive knowledge storage system containing a large amount of professional knowledge, common sense information and concept relationships related to situational questions. In the embodiment of the application, it can be understood as a multi-domain, multi-level structured knowledge graph covering object attributes, spatial relationships, causal relationships and other types of knowledge. This database not only stores static facts, but also contains dynamically updated rules and reasoning chains. The knowledge database is used to provide the intelligent agent with the background knowledge and professional information needed to answer situational questions. When the large language model faces a knowledge gap, the system can retrieve relevant information from the knowledge database to supplement the necessary knowledge, thereby supporting more accurate and comprehensive reasoning processes and improving the quality and reliability of the answers.

[0071] Notably, both the method database and the knowledge database are derived from training the agent on sample situational problems. As the system processes more problems, these databases will continue to grow and optimize. New successful reasoning processes will be added to the method database, while new domain knowledge will be integrated into the knowledge database. This mechanism of continuous learning enables the agent to continuously improve its ability to handle complex situational problems as it is used, demonstrating strong adaptability and scalability.

[0072] In implementation, the agent first converts the first answer into a feature vector. Then, the agent calls the method database, which stores a large number of reasoning processes trained based on sample situational problems. Each stored reasoning process is represented as a corresponding feature vector. The agent calculates the similarity between the feature vector of the first answer and each reasoning process in the database using cosine similarity or other appropriate similarity measurement methods. If a reasoning process with a similarity greater than a preset threshold is found, the agent will mark it as a potential reference method.

[0073] Step 302, if there is a reasoning process with a similarity greater than the threshold, input the reasoning process with a similarity greater than the threshold to the large language model.

[0074] In implementation, when the system finds a reasoning process with a similarity greater than the threshold, the system first extracts all reasoning processes with a similarity greater than the preset threshold from the method database. The system will integrate and adapt these reasoning processes to ensure they can match the current situational problem. The adjustment may include replacing specific entity names, adjusting the order of reasoning steps, or adding or deleting certain reasoning steps. Subsequently, the system will construct a new, more rich and structured input by combining these adjusted reasoning processes with the original situational problem and the first answer as additional context information. The system passes this input to the large language model, providing the large language model with a more explicit reasoning framework and reference.

[0075] Step 303, if there is no reasoning process with a similarity greater than the threshold, call the knowledge information corresponding to the first answer in the knowledge database, and input the knowledge information to the large language model.

[0076] When the system finds that there is no inference process with a similarity greater than the threshold, it first analyzes the content of the first answer, extracts the key concepts, entities, and relationships. Then, the system uses this information to perform a multi-dimensional search in the knowledge database, retrieving relevant knowledge information. This knowledge information may include concept definitions, attribute descriptions, causal relationships, spatial relationships, and other types. The system will filter and integrate the retrieved knowledge information. Subsequently, the system constructs a new, knowledge-enhanced input by combining the filtered and integrated knowledge information with the original context question and the first answer. The system passes this input to the large language model, providing the large language model with rich background knowledge and professional information, thereby supporting more in-depth and accurate reasoning.

[0077] Step 304, obtaining the second answer output by the large language model.

[0078] In specific implementation, the system first receives the second answer output by the large language model. If the second answer is still "cannot answer", the system considers it as an incorrect answer and initiates a loop mechanism. This may include readjusting the knowledge supplement strategy, fine-tuning the inference process, or further expanding the knowledge retrieval range, and then inputting the optimized information into the large language model again to try to generate a new answer. If the second answer contains "show reasoning + answer", the system considers it as a successful answer.

[0079] In a feasible implementation, the system can also verify the successful answer. The verification process includes consistency check, logical verification, completeness evaluation and knowledge matching. The system checks whether the reasoning process in the answer is consistent with the provided background information and knowledge, evaluates the logicality of the reasoning process, ensures that each reasoning step is reasonable, checks whether the answer completely answers all aspects of the original question, and compares the answer with the information in the knowledge database to ensure that there are no obvious knowledge errors. If the verification process finds any problems, the system may trigger additional optimization loops, such as re-adjusting the knowledge supplement strategy or fine-tuning the inference process, and then inputting the large language model again to obtain an improved answer.

[0080] The above embodiment describes the reasoning process of the agent for the context question. Based on the above embodiment, please refer to Figure 2 , Figure 2 is a flowchart of an agent training method provided by an embodiment of the present application. The training process of the agent will be described below. Specifically, the training process of the agent can include the following steps:

[0081] Step 401, obtaining a sample context question, the sample context question including a sample context scene and a sample target question.

[0082] The sample context question refers to a set of pre-prepared question sets used in the training process of the intelligent agent, which simulates various situations and queries that may be encountered in actual applications. In the embodiment of the present application, it can be understood as a series of paired data containing sample context scenes and sample target questions. The sample context scene is a detailed description of a three-dimensional indoor environment, including room layout, object distribution, object attributes, and other information. The sample target question is a specific inquiry based on these scenes.

[0083] The sample context question mainly provides rich visual scene data for the training of the caption model, helping the model learn how to accurately describe complex three-dimensional environments. Secondly, these questions provide diverse reasoning exercises for large language models, enabling them to learn how to perform logical reasoning in different contexts. Thirdly, through sample context questions, the system can build and optimize the method database and knowledge database, accumulating effective reasoning strategies and domain knowledge. Finally, sample questions are also used to evaluate and verify the performance of the entire intelligent agent system, ensuring its ability to accurately handle various types of context questions. By using a large number of and diverse sample context questions, the generalization ability of the intelligent agent can be significantly improved, enabling it to effectively deal with unknown situations in actual applications.

[0084] Step 402, training the caption model integrated in the intelligent agent through the sample context scene, and obtaining the sample key information output by the trained caption model.

[0085] During the training process of the intelligent agent, the training of the caption model aims to improve the model's understanding and description of three-dimensional environments. The caption model is responsible for converting complex three-dimensional scene information into structured text descriptions in the entire system, providing necessary input for subsequent reasoning processes.

[0086] Further, the embodiment of the present application uses the caption model as the perception module of the intelligent agent. Specifically, the perception module uses the caption model to locate all objects in the three-dimensional scene and generate descriptive sentences for each object. The reasoning module further answers questions based on the caption information. However, due to the training of the caption model, the evaluation criteria are based on text similarity indicators, resulting in generated captions with higher text similarity forms, but poor performance in key descriptive information that significantly affects reasoning judgments. There is a situation where the training evaluation criteria do not match the application context.

[0087] Based on the above embodiment, as an optional embodiment, to solve the above problem, the present application adopts self-criticism sequence training, and proposes a reinforcement learning method dedicated to key information by designing a reward function. Specifically, the process can further include the following steps:

[0088] Step 501, inputting the sample context scene into the subtitle model integrated in the agent to obtain a sample text description output by the subtitle model.

[0089] Step 502, evaluating the sample text description based on a loss function and a reward function to obtain an evaluation result, and adjusting the subtitle model according to the evaluation result, wherein the loss function and the reward function are used to adjust the proportion of key information about objects in the sample context scene in the sample text description.

[0090] The loss function and the reward function are two complementary evaluation tools that work together to optimize the performance of the subtitle model. The loss function refers to a mathematical function used to measure the difference between the model's predicted output and the actual target value. In the embodiments of the present application, it can be understood as a measurement tool for evaluating the gap between the sample text description generated by the subtitle model and the ideal description. It usually takes the form of cross-entropy loss or mean square error, which is used to guide the parameter adjustment in the model learning process, helping the model gradually reduce the prediction error and improve the overall accuracy and quality of the generated text.

[0091] Correspondingly, the reward function is a mathematical function used to evaluate and encourage specific behaviors or outputs in the reinforcement learning framework. In the embodiments of the present application, it can be understood as a specially designed scoring mechanism used to adjust the proportion of key information about objects in the sample context scene in the sample text description.

[0092] Further, the design of the reward function takes into account multiple aspects, including the accuracy of key attribute descriptions, the completeness of spatial relationship descriptions, information density, and the ability to capture rare but important information. In this way, the reward function guides the model to pay more attention to the accurate description of important object attributes and spatial relationships in the scene, thereby generating text descriptions that are more conducive to subsequent reasoning tasks.

[0093] For example, the loss function expression based on the self-criticism sequence training is:

[0094] ;

[0095] In the formula, represents the loss function of the subtitle candidate , represents the beam size of the beam search, i.e., the number of generated candidate subtitles, represents the i-th subtitle candidate generated by the beam search, represents the baseline subtitle generated by the greedy search, represents the reward function of the generated subtitle candidate , represents the reward function of the baseline subtitle, represents the length of the subtitle candidate , representing given scene information generating subtitles probability.

[0096] Specifically, the loss function is based on reinforcement learning principles, aiming to guide model optimization through a reward function R. The model generates multiple subtitle candidates through beam search and a baseline subtitle through greedy search.

[0097] wherein the reward function is the core of the entire self-critical sequence training, which considers both the text similarity of the subtitle and the reference subtitle and the generation of key adjectives in the subtitle, and its expression is:

[0098] ;

[0099] wherein, represents the score of the consensus-based image description evaluation index, represents the set of correct key adjectives contained in the subtitle, represents the set of correct adjectives contained in the subtitle, represents the number of correct adjectives in the subtitle, represents the number of incorrect adjectives in the subtitle, represents a weight parameter for balancing the contributions of CIDEr score and key adjective reward.

[0100] wherein the key information richness index is used to evaluate the generation quality of key information in the subtitle. This index reflects the model's performance in generating key descriptive information by comparing the number of correctly generated adjectives with the total number of adjectives, and its expression is:

[0101] ;

[0102] wherein n represents the number of subtitles, represents the total number of adjectives in the reference subtitle.

[0103] Further, the loss function ensures the basic quality and accuracy of the generated text, providing the model with overall optimization direction. The reward function further guides the model to focus on the most important information points for subsequent reasoning, achieving fine tuning of the subtitle model.

[0104] The combination of loss function and reward function not only improves the overall performance of the model, but also makes the generated scene description more consistent with the specific needs of the situational problem answering task, so that the subtitle model can generate scene descriptions that are both accurate and rich in key information, providing high-quality input for the subsequent reasoning process, thereby significantly improving the efficiency and accuracy of the entire intelligent agent system in handling complex situational problems.

[0105] Step 403, obtaining a sample first answer based on the sample key information and the sample target question through the large language model.

[0106] The implementation principle of step 403 is the same as that of step 102, and specific reference can be made to step 102, which will not be described in detail here.

[0107] Step 404, obtaining a method database by reasoning and reflecting on the large language model through the sample first answer.

[0108] Specifically, through the perception module, the agent obtains the information of the current environment, and can perform zero-shot reasoning under the prompt through the inherent knowledge of the large language model. The inherent ability of the large language model is sufficient to meet most cases, but there are still cases where effective reasoning cannot be formed. This is because the context question answer belongs to a vertical professional field, and the out-of-box large language model lacks method experience for solving professional problems, so it still needs to be taught the method for specific tasks and accumulate experience for solving problems in memory. In order to achieve this goal, the embodiment of the present application proposes a reflection module, which can continuously learn from past question and answer reasoning experience, summarize methods for solving specific professional tasks, and gradually improve its reasoning performance.

[0109] Please refer to Figure 3 , Figure 3 is a flowchart of a reflection module provided by an embodiment of the present application. Based on the above embodiment, as an optional embodiment, the process can further include the following steps:

[0110] Step 601, if it is determined that the sample first answer is incorrect, feeding back the result of the first answer error to the large language model to obtain a thought chain output by the large language model, the thought chain including a correct sample first answer and a corresponding reasoning process.

[0111] The thought chain refers to a structured reasoning process representation that records the entire thinking path from the question to the answer in detail. In the embodiment of the present application, it can be understood as a series of logical reasoning steps generated by the large language model when processing the context question, including the correct sample first answer and the detailed reasoning process to reach this answer. The thought chain is usually presented in a coherent narrative form, starting from question understanding, through information extraction, knowledge application and step-by-step reasoning, and finally reaching a conclusion and self-verification.

[0112] In the training process of the agent, when the system determines that the sample first answer is incorrect, the error result is fed back to the large language model and the thought chain output by the large language model is obtained, which is an effective self-correction and learning mechanism. It can make the system learn from mistakes, deeply understand the shortcomings in the reasoning process, and generate more accurate and interpretable answers.

[0113] In implementation, the system first constructs a special prompt input to the large language model. This prompt contains the original context question, the incorrect sample first answer, and a clear instruction asking the model to rethink the question and generate a detailed reasoning process.

[0114] After receiving the prompt, the large language model initiates a deep self-reflection and re-reasoning mechanism. In this process, the model first analyzes the reasons for the incorrect answer, which may be due to insufficient information understanding, incorrect reasoning logic, or improper knowledge application. Then, the model re-examines the key information in the question and designs a step-by-step reasoning process. This process usually includes clearly defining the premise conditions, setting intermediate reasoning steps, applying relevant knowledge, and finally drawing a correct conclusion. Each step of reasoning is clearly stated, forming a complete thought chain.

[0115] Step 602, generate a method database based on the thought chain.

[0116] In implementation, the system first performs in-depth analysis and structured processing on the acquired thought chain. This process involves identifying key components in the thought chain, such as question understanding, information extraction, knowledge application, reasoning steps, and conclusion generation. The system uses natural language processing techniques such as semantic analysis and information extraction algorithms to parse the content of the thought chain, extracting the core reasoning logic and key decision points.

[0117] Next, the system converts the extracted information into standardized reasoning templates. These templates not only contain the logical structure of reasoning, but also include applicable question types, required premise conditions, and possible variants of metadata.

[0118] In a feasible implementation, to improve retrieval efficiency and application flexibility, the system uses vector representation technology to encode each reasoning template into a high-dimensional vector. This representation method allows the system to quickly find the most relevant reasoning strategies when faced with new questions.

[0119] Through the above thought chain technology, the large language model can find the unexplained reasons and understand and summarize the correct method to answer the question. Finally, the correct reasoning process and correction results are saved in the method database memory of the reflection module, and are indexed by the embedded vector representation of the question, which is used for subsequent retrieval by the reasoning module based on the cosine similarity of the question.

[0120] Step 405, knowledge extraction from the large language model through the sample first answer, obtaining a knowledge database.

[0121] After being able to perceive scene information and learn to solve related problems, the intelligent agent has greatly improved in answering questions, but due to the inherent sparsity of point clouds and the chaotic distribution of objects in three-dimensional scenes, the subtitle generation model cannot generate information about all objects, and there is an unavoidable information loss situation, which also leads to reasoning failure due to information loss.

[0122] To address this situation, the embodiment of the present application designs an extraction module to extract prototype knowledge information of objects in the training set. This knowledge information is independent of specific scene conditions and has general properties, so it can be used as inherent knowledge information of the intelligent agent to supplement missing information and use it during reasoning.

[0123] Please refer to Figure 4 , Figure 4 is a flowchart of an extraction module provided by the embodiment of the present application. Specifically, the process can further include the following steps:

[0124] Step 701, the sample first answer, sample key information and sample target question corresponding to the same object in the sample situation scene are combined into a triple, and the feature vector of the triple is extracted.

[0125] In the training process of the intelligent agent, in order to solve the problem of reasoning failure caused by the lack of three-dimensional scene information, the embodiment of the present application combines the sample first answer, sample key information and sample target question corresponding to the same object in the sample situation scene into a triple, and extracts the feature vector of these triples. Thus, the multi-dimensional information of the object in different scenes and question situations is captured, laying a foundation for subsequent knowledge extraction and generalization.

[0126] In specific implementation, the system first identifies and extracts relevant information of the same object from the sample situation scene. The above-mentioned "same object" may appear in different scenes and have different attributes or states. For each identified object instance, the system will collect the sample first answer, sample key information and sample target question related to it. The sample first answer contains the initial understanding and description of the object by the large language model; the sample key information is the key description of the object in the scene extracted by the subtitle generation model; and the sample target question reflects the query type related to the object.

[0127] Combining the above three kinds of information into a triple provides a comprehensive description of the object, including its attributes, functions, and performance in different situations. Secondly, this combination captures the internal relationship between the object, the scene description and the question, which helps to understand the role and importance of the object in different situations.

[0128] Step 702, obtain the centroid obtained by clustering the feature vectors corresponding to the same object.

[0129] Specifically, during the agent training process, the centroid obtained by clustering the feature vectors corresponding to the same object is used to extract the core features and general attributes of the object from a large number of samples, thereby effectively addressing the incomplete information and noise problems caused by the sparsity of the point cloud of the three-dimensional scene and the chaotic distribution of the object, and providing a more reliable foundation for subsequent knowledge generalization and reasoning.

[0130] In specific implementation, the system first collects all feature vectors corresponding to the same object in different sample context scenarios. Each feature vector represents the comprehensive features of the object in a specific scenario and problem context. Next, the system performs clustering analysis on these feature vectors. The choice of clustering algorithm can be determined according to the characteristics and distribution of the data. Common methods include K-means clustering, hierarchical clustering, or density clustering, etc. In this embodiment, K-means clustering is used as an example for illustration. The core idea of K-means algorithm is to divide similar feature vectors into the same cluster and calculate the center point of each cluster, i.e. the centroid.

[0131] In a feasible implementation, the process of K-means clustering is as follows: First, the system needs to determine the number of clusters K. This K value can be determined by heuristic methods or cross-validation. Then, the algorithm randomly selects K initial centroids and assigns each feature vector to the cluster represented by the nearest centroid. Next, the algorithm recalculates the centroid of each cluster, i.e. the average of all vectors in the cluster, and reassigns the feature vectors to the nearest new centroid. This process is iterated until the centroid position no longer changes significantly or the preset number of iterations is reached.

[0132] Step 703: input the knowledge information corresponding to the centroid into the large language model, generalize the knowledge information corresponding to the centroid through the large language model, and obtain sample knowledge information.

[0133] Specifically, inputting the knowledge information corresponding to the centroid into the large language model and generalizing it to obtain sample knowledge information can convert the highly abstract numerical representation obtained by clustering in the previous steps into more general, understandable and applicable semantic knowledge.

[0134] In specific implementation, the system first needs to convert the numerical representation of the centroid back to a text form that can be understood. This process may involve reverse mapping or decoding techniques to convert high-dimensional vectors into a series of keywords, phrases or sentences. These text descriptions may include basic attributes of the object, common uses, typical scenarios, etc. For example, for the object "chair", the converted text may include "has four legs", "used for sitting", "commonly found in living rooms and offices", etc.

[0135] Next, the system provides these preliminary converted text descriptions as input to a large language model. The large language model receives these centroid-based preliminary descriptions, utilizes its vast pre-trained knowledge base and powerful language understanding capabilities, and performs deep processing and expansion on this information.

[0136] During processing, the large language model supplements and refines the information that may be missing in the initial descriptions. For example, if the initial description only mentions the basic structure of a chair, the model may add general information about the material, style, comfort, etc. Second, the model generates more abstract and general descriptions, so that this knowledge can be applied to a wider range of scenarios. Third, the model considers variations and special uses of objects in different contexts, thereby increasing the diversity and applicability of knowledge. Finally, the model may also establish connections between objects and other related concepts, forming a small knowledge network, i.e., sample knowledge information. Sample knowledge information is no longer limited to specific scenarios or specific types of problems, but contains the general characteristics, functions, variations of objects, and associations with other concepts.

[0137] For example, for "chair", the sample knowledge information may include: "A chair is a piece of furniture used for sitting, usually with four legs and a backrest. It can be made of wood, metal, plastic, etc. There are many types of chairs, including dining chairs, office chairs, and leisure chairs, suitable for different occasions. In some designs, chairs may not have legs but use other support structures. Chairs are often used with tables, sofas, and other furniture."

[0138] Step 704, generate a knowledge database based on sample knowledge information.

[0139] Step 406, integrate the method database and the knowledge database into the agent.

[0140] The above embodiments respectively describe the training process of the perception module, the reflection module, and the extraction module in the agent. Based on the above embodiments, please refer to Figure 5 , Figure 5 is a flowchart of a reasoning module provided by an embodiment of the present application. The application process of the reasoning module will be described below.

[0141] Specifically, in the reasoning process of the agent, the reasoning module integrates the outputs of the perception module, the reflection module, and the extraction module, and combines the powerful capabilities of the large language model, to realize a complex and efficient reasoning system.

[0142] The reasoning process starts with reading the scene information and converting it into its own perspective state. It provides the agent with an environment understanding centered on itself. The system reads in all the object captions and bounding box information generated by the perception module, and uses the rotation-translation seven-tuple of the scene position to generate the corresponding rotation matrix and translation matrix. Through this conversion, the position information of the object is mapped to the current perspective of the agent, so that the subsequent reasoning can be based on the "subjective" perception of the agent. This method simulates the cognitive process of humans in the actual environment, and helps to generate more natural and intuitive reasoning results.

[0143] Next, the system generates the prompts needed for reasoning. The design of the prompt contains three key components: system prompt, scene text description, and reasoning guide. The system prompt outlines the basic requirements of the situational question answering task, setting the framework for the large language model. The scene text description uses natural language to comprehensively present the attributes and location information of the objects, especially classifying the objects into four directions of front, back, left and right according to their relative positions. This classification method helps the large language model better understand the spatial relationship. The reasoning guide provides a basic explicit and interpretable reasoning process, or integrates the experience in the reflection module and the refinement module when necessary. This multi-level prompt design aims to provide sufficient context and guidance for the large language model, enabling it to reason more accurately and targetedly.

[0144] Inputting the generated prompt into the large language model takes into account the complexity of the situational question answering task. The system uses the thought chain prompting technique, thereby improving the transparency and interpretability of the reasoning process, so that each step of reasoning can be traced and understood. Secondly, it can alleviate the hallucination phenomenon that may occur in the large language model, because each step of reasoning needs to have logical support. Finally, when it is impossible to form an interpretable reasoning, the system will output "cannot answer" in time. This honest feedback mechanism increases the credibility of the system.

[0145] During the reasoning process, if an uninterpretable situation occurs, the agent will activate the reflection and refinement modules. The system embeds the current scene question into a vector for similarity query in the method database of the reflection module. If a situation with similarity higher than the threshold is found, the system will use it as a reference, combine the corresponding reasoning process generation method experience, and integrate it into the prompt generator. If the similarity is lower than the threshold, the system will turn to the knowledge database for supplementation, and integrate the relevant knowledge experience into the prompt. This dynamic calling of experience and knowledge greatly enhances the adaptability and reasoning ability of the system, enabling it to handle more diverse and complex situational questions.

[0146] Finally, if the agent still cannot form an effective reasoning, the system will enable a supervised learning tool trained end-to-end in the context of question answering tasks. This tool provides candidate answer options for the large language model to judge whether it can form an interpretable reasoning. This step serves as a last safeguard mechanism to ensure that the system can give reasonable responses in various situations while maintaining the interpretability of reasoning.

[0147] Through the above multi-level, closed-loop agent, it greatly improves the agent's ability to handle complex three-dimensional scene problems, even in the case of incomplete information or noise, it can give reasonable reasoning. Secondly, through the thought chain technology and experience recall mechanism, the system realizes high interpretability, making the reasoning process transparent and traceable. Thirdly, the design of dynamically calling the reflection and refinement module enhances the adaptability and learning ability of the system, enabling it to continuously accumulate experience from new situations and improve reasoning ability.

[0148] On this basis, please refer to Figure 6 and Figure 7 , Figure 6 a context question answer result display diagram provided by the embodiment of the present application, Figure 7 a context question answer result display diagram provided by the embodiment of the present application.

[0149] Figure 8 An example of an entity structure diagram of an electronic device is shown in Figure 8 , which can include a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 can call the logical instructions in the memory 830 to execute the method for reasoning the context question based on the large language model.

[0150] In addition, the logic instructions in the memory 830 described above can be implemented in the form of software functional units and sold or used as independent products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0151] In another aspect, the present application also provides a computer program product, which comprises a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program can be executed by a processor to enable a computer to execute the method for reasoning a situational problem based on a large language model provided by the above-mentioned methods.

[0152] In yet another aspect, the present application also provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method for reasoning a situational problem based on a large language model provided by the above-mentioned methods.

[0153] The device embodiments described above are only schematic, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., they can be located in one place, or distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment. Those skilled in the art can understand and implement without creative labor.

[0154] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be implemented by means of software plus necessary general hardware platforms, and of course can also be implemented by hardware. Based on such understanding, the above technical solutions essentially or the parts that contribute to the prior art can be embodied in the form of a software product. The computer software product can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments.

[0155] It should be pointed out finally that the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features therein can be replaced equivalently; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for context problem reasoning based on a large language model, characterized in that, The method comprises the following steps: acquiring a context question; inputting the context question into an agent to acquire a first answer output by the agent, the first answer being reasoned by a large language model integrated in the agent based on the context question; if it is determined that the first answer is incorrect, inputting the first answer into the agent to acquire a second answer output by the agent, the second answer being reasoned by the large language model after the agent reflects on or supplements knowledge of the large language model based on the first answer; wherein the agent is trained based on sample context questions; the step of inputting the first answer into the agent to acquire a second answer output by the agent comprises: inputting the first answer into the agent to match the first answer with a method database through the agent to determine whether there is a reasoning process with a similarity greater than a threshold; if there is a reasoning process with a similarity greater than a threshold, inputting the reasoning process with a similarity greater than a threshold into the large language model; if there is no reasoning process with a similarity greater than a threshold, calling knowledge information corresponding to the first answer in a knowledge database and inputting the knowledge information into the large language model; acquiring the second answer output by the large language model; wherein the method database and the knowledge database are obtained based on training the agent based on sample context questions.

2. The method for situational problem reasoning based on a large language model according to claim 1, characterized in that, The context question comprises a context scene and a target question, and the step of inputting the context question into an agent to acquire a first answer output by the agent comprises: inputting the context scene into a caption model integrated in the agent to generate a text description of key information of objects in the context scene through the caption model and inputting the text description into the large language model; inputting the target question into the large language model to reason the target question based on the text description through the large language model to obtain a first answer; wherein the caption model is obtained based on training the agent based on sample context questions.

3. The method for situational problem reasoning based on a large language model according to claim 1 or 2, characterized in that, The training process of the agent comprises: acquiring sample context questions, the sample context questions comprising sample context scenes and sample target questions; training a caption model integrated in the agent through sample context scenes to acquire sample key information output by the trained caption model; obtaining a sample first answer based on the sample key information and the sample target question through a large language model; reflecting on the large language model through the sample first answer to obtain a method database; extracting knowledge of the large language model through the sample first answer to obtain a knowledge database; integrating the method database and the knowledge database in the agent.

4. The method for situational problem reasoning based on a large language model according to claim 3, characterized in that, The step of training a caption model integrated in the agent through sample context scenes comprises: inputting sample context scenes into the caption model integrated in the agent to acquire sample text descriptions output by the caption model; evaluating the sample text descriptions based on a loss function and a reward function to obtain an evaluation result and adjusting the caption model according to the evaluation result; The loss function and the reward function are used to adjust the proportion of key information about objects in a sample context scene in the sample text description.

5. The method for situational problem reasoning based on a large language model according to claim 3, characterized in that, The method database is obtained by reasoning and reflecting the large language model through the sample first answer, and includes: If it is determined that the sample first answer is incorrect, the result of the incorrect first answer is fed back to the large language model to obtain a thought chain output by the large language model, and the thought chain includes a correct sample first answer and a corresponding reasoning process; The method database is generated based on the thought chain.

6. The method for situational problem reasoning based on a large language model according to claim 3, characterized in that, The knowledge database is obtained by knowledge extraction of the large language model through the sample first answer, and includes: The sample first answer, sample key information and sample target question corresponding to the same object in the sample context scene are taken as a triple, and a feature vector of the triple is extracted; A centroid obtained by clustering the feature vectors corresponding to the same object is obtained; Knowledge information corresponding to the centroid is input into the large language model, and the knowledge information corresponding to the centroid is generalized by the large language model to obtain sample knowledge information; The knowledge database is generated based on the sample knowledge information.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method for reasoning a context question based on a large language model according to any one of claims 1 to 6.

8. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method for reasoning a context question based on a large language model according to any one of claims 1 to 6.

9. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the method for reasoning a context question based on a large language model according to any one of claims 1 to 6.

Images