Large language model-based situational problem reasoning method and related equipment

Abstract

The invention discloses a large language model-based situational problem reasoning method and related equipment, and relates to the technical field of artificial intelligence. The method comprises the following steps of: obtaining a situational problem; the situational question is input into an intelligent agent, a first answer output by the intelligent agent is obtained, and the first answer is obtained through reasoning of a large language model integrated in the intelligent agent based on the situational question; if it is determined that the first answer is wrong, the first answer is input to the intelligent agent, a second answer output by the intelligent agent is obtained, and the second answer is obtained by reasoning of the large language model after the intelligent agent conducts reasoning reflection or knowledge supplementation on the large language model based on the first answer; wherein the intelligent agent is obtained based on sample situation problem training. By implementing the technical scheme provided by the invention, a mechanism can be supplemented to the reflection and knowledge of the intelligent agent, so that the accuracy and reliability of answering the situation question are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method for reasoning about situational issues based on a large language model and related equipment. Background Art

[0002] Situational questions are questions asked in a specific environment or scenario, requiring the respondent to analyze and reason based on the given situational information to come up with an answer. Such questions are widely used in fields such as intelligent assistants, educational assessment, and customer service, and are designed to test and improve the AI ​​system's ability to understand and reason about complex scenarios.

[0003] At present, the mainstream method to solve situational problems is to use large language models. Large language models have acquired rich knowledge and powerful language understanding capabilities by training on massive text data. When faced with situational problems, large language models can directly generate corresponding answers based on the input questions and situation descriptions.

[0004] However, large language models are prone to making incorrect or inappropriate responses when faced with complex or unfamiliar situations, and are difficult to self-correct effectively, thus affecting the accuracy of responses to situational questions. Summary of the invention

[0005] The present invention provides a method and related equipment for reasoning about situational questions based on a large language model, which can improve the accuracy and reliability of answers to situational questions by providing a reflection and knowledge supplement mechanism for an intelligent agent.

[0006] In a first aspect of the present invention, a method for reasoning about a situational problem based on a large language model is provided, comprising: Get contextual questions; Input the situational question into the intelligent agent, and obtain a first answer output by the intelligent agent, wherein the first answer is inferred based on the situational question by a large language model integrated in the intelligent agent; If it is determined that the first answer is wrong, the first answer is input to the agent to obtain a second answer output by the agent, where the second answer is inferred by the large language model after the agent performs reasoning reflection or knowledge supplement on the large language model based on the first answer; Wherein, the intelligent agent is trained based on sample situational problems.

[0007] In a second aspect of the present invention, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method for reasoning about situational issues based on a large language model as described in any one of the above is implemented.

[0008] In a third aspect of the present invention, a non-transitory computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the method for reasoning about situational issues based on a large language model as described in any one of the above is implemented.

[0009] In a fourth aspect of the present invention, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the method for reasoning about situational issues based on a large language model as described in any one of the above is implemented.

[0010] In summary, one or more technical solutions provided in the present invention have at least the following technical effects or advantages: By inputting the situational question into the intelligent agent, the first answer is obtained by initial reasoning using the large language model integrated in the intelligent agent. When the first answer is determined to be wrong, the intelligent agent can use the first answer as a new input to reflect on the reasoning or supplement the knowledge of the large language model, thereby generating a more accurate second answer.

[0011] The above iterative reasoning process enables the intelligent agent to improve the accuracy of its answers through self-correction and knowledge updating when faced with complex or unfamiliar situations. Compared with the method of directly generating answers with a large language model in the prior art, the method of the present invention can effectively deal with the situation where the initial answer is wrong, and through the intelligent agent's reflection and knowledge supplementation mechanism, it significantly improves the accuracy and reliability of the answers to situational questions. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0013] Figure 1 It is a flowchart of a method for reasoning about situational issues based on a large language model provided by an embodiment of the present invention.

[0014] Figure 2 It is a flowchart of an intelligent agent training method provided by an embodiment of the present invention.

[0015] Figure 3 It is a flow chart of a reflection module provided by an embodiment of the present invention.

[0016] Figure 4 It is a flow chart of a refining module provided by an embodiment of the present invention.

[0017] Figure 5It is a flowchart of an inference module provided by an embodiment of the present invention.

[0018] Figure 6 It is a situational question answer result display diagram provided by an embodiment of the present invention.

[0019] Figure 7 It is a situational question answer result display diagram provided by an embodiment of the present invention.

[0020] Figure 8 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] The situational question answering task requires the agent to understand questions and give appropriate answers in a specific environmental context. This task is based on complex indoor scenes in the real world. The agent needs to understand and answer questions based on the layout of the environment, its own position, orientation, and state. The difficulty of this task lies in the fact that the agent not only needs to understand semantic information, but also must make a comprehensive judgment based on visual and spatial information.

[0023] At present, common solutions usually rely on Transformer-based backbone networks, which can effectively encode visual and textual information and support end-to-end training. In this method, visual data and textual data are fused after encoding, and finally the answer is generated through the prediction output head.

[0024] In addition, there are some methods that use pre-trained models to handle contextual issues. Specifically, these methods use a large amount of multimodal data in the pre-training stage to integrate three-dimensional environmental data into the language model, so that the model has preliminary contextual understanding capabilities. Afterwards, through fine-tuning, these pre-trained models can be further adapted to specific tasks and generate more accurate answers.

[0025] Among the existing answering methods, there are two main categories: closed-set answering and open-set answering. The closed-set answering method requires the agent to select the correct answer to the question from a predefined, limited set of answers. The advantage of this method is its controllability.

[0026] However, its limitation is that the answer range is limited and it may not be able to handle questions beyond the predefined set. The open answer method is not restricted by a fixed list of answers and allows the agent to generate answers from an infinite or non-predefined answer space. This method is more flexible, but also more challenging because the process of generating answers does not perform explicit reasoning, resulting in weaker interpretability.

[0027] Based on the above, an embodiment of the present invention provides a method for reasoning about situational issues based on a large language model to solve the problems existing in the above-mentioned prior art.

[0028] Please refer to Figure 1 , Figure 1 This is a flow chart of a method for reasoning about a situational problem based on a large language model provided by an embodiment of the present invention. The method can be implemented by a computer program, can be implemented by a single-chip microcomputer, or can be run on a reasoning system for situational problems based on a large language model based on a von Neumann system. The computer program can be integrated into an application or run as an independent tool application. Specifically, the method can include the following steps: S101. Obtain situational questions.

[0029] Among them, situational questions refer to questions asked in a specific environment or scenario, which require consideration of the surrounding environment information to answer correctly. Situational questions not only involve semantic understanding, but also require a comprehensive judgment based on spatial information and visual perception.

[0030] In the embodiment of the present invention, the situational question includes a situational scenario and a target question. These two components together clearly define the specific task that the agent needs to solve and provide a clear goal for the entire question-answering system.

[0031] The situational scene refers to a comprehensive description of the specific environment and state that the agent is in. In the embodiment of the present application, it can be understood as a detailed description of a three-dimensional indoor scene and a collection of state information such as the position and orientation of the agent in the scene.

[0032] For example, the situational scene can include environmental information such as room layout, object distribution, and object attributes. This information together constitutes the background environment of the question, providing the agent with the context needed to understand and answer the question. The situational scene is used to provide a focus for the perception module, guide the extraction and processing of environmental information, and also provide the reasoning module with basic data for spatial reasoning and relationship judgment.

[0033] Correspondingly, the target question refers to a specific inquiry that the agent needs to answer in a given situation scenario. In the embodiment of the present invention, it can be understood as a specific question based on the current three-dimensional scene and the state of the agent, which may involve multiple aspects such as object recognition, spatial relationship judgment, and attribute query. The target question is usually presented in the form of natural language, and the agent needs to understand and answer it in combination with the situation scenario information.

[0034] S102. Input the situational question into the intelligent agent, and obtain a first answer output by the intelligent agent, wherein the first answer is inferred based on the situational question by a large language model integrated in the intelligent agent, wherein the intelligent agent is trained based on sample situational questions.

[0035] Among them, the intelligent agent refers to a computing system or program that can perceive the environment, make decisions and take actions to achieve specific goals. In the embodiment of the present invention, it can be understood as a comprehensive artificial intelligence system that integrates a large language model, a perception module, a reflection module, a refinement module and a reasoning module. This intelligent agent is designed to understand and answer complex situational questions, especially question-answering tasks in three-dimensional indoor scenes.

[0036] Furthermore, the intelligent agent in the embodiment of the present invention is a system in which multiple modules work together. Its core is a large language model that can be used out of the box. This model has a broad knowledge base and strong language understanding capabilities. Around this core, the intelligent agent also includes specially designed perception modules, reflection modules, refinement modules and reasoning modules. The perception module is responsible for analyzing and understanding the three-dimensional scene and extracting key environmental information. The reflection module can evaluate and improve the reasoning process and accumulate experience in solving problems. The refinement module extracts and summarizes general knowledge from historical data. The reasoning module integrates all the above information to generate the final answer.

[0037] Based on the above embodiment, as an optional embodiment, in step 102: inputting the situational question to the agent and obtaining the first answer output by the agent, the step may further include the following steps: Step 201: Input the situational scene into a subtitle model integrated in the intelligent agent, generate a text description of key information corresponding to the objects in the situational scene through the subtitle model, and input the text description into the large language model, wherein the subtitle model is obtained by training the intelligent agent based on sample situational problems.

[0038] The subtitle model refers to a machine learning model that can convert visual information into text descriptions. In the embodiment of the present invention, it can be understood as a neural network model specially designed to process three-dimensional scene information. The subtitle model is usually based on deep learning technology, such as convolutional neural networks, recurrent neural networks, or Transformer architecture. The core function of the subtitle model is to analyze complex spatial data, identify objects and their attributes in the scene, and generate structured text descriptions.

[0039] In an embodiment of the present invention, the subtitle model first parses the input three-dimensional scene data, which may include point clouds, depth images or other forms of spatial representation. Through deep learning algorithms, the model can identify each object in the scene and extract their key features, such as color, shape, material, size and other attribute information. At the same time, the subtitle model can also understand and describe the relative position and spatial relationship between objects, such as "to the left of" or "above".

[0040] Based on the extracted features and spatial relationship information, the caption model then generates accurate, concise and structured text descriptions. These descriptions contain not only the attribute information of the objects, but also their positions and relationships in the scene.

[0041] It is worth noting that since the subtitle model is trained based on sample situational questions, it can specifically extract and describe key information related to possible situational questions instead of simply describing all visible details.

[0042] The subtitle model plays a key role in bridging the 3D scene data and the large language model in the present invention. By converting complex spatial information into structured text descriptions, the subtitle model provides the necessary environmental information foundation for subsequent question understanding and answering, enabling the large language model to better understand and process situational questions related to the 3D scene.

[0043] Among them, key information refers to the core data or features that are crucial to understanding and solving problems in a specific scene or problem context. In the embodiments of the present invention, it can be understood as information such as scene elements, object attributes, and spatial relationships that are closely related to situational problems. This information is extracted from a complex three-dimensional environment and plays a decisive role in accurately answering situational questions.

[0044] Specifically, key information may include basic attributes of objects in the scene, such as color, shape, material, and size, as well as spatial relationships such as relative position, direction, and distance between objects. For example, in a living room scene, the color and position of the sofa, the size and orientation of the TV, and the relative position of the coffee table and the sofa may all become key information. The importance of the above information depends on the specific situational question. For example, when asking "Where is the red object in the room?", the color of the object becomes key information.

[0045] Step 202: Input the target question into the large language model, and use the large language model to infer the target question based on the text description to obtain a first answer.

[0046] In specific implementation, the system first converts the target question into a format suitable for processing by the large language model, which can be a piece of text. Then, this text and the scene text description generated in step S201 are provided as input to the large language model. The large language model analyzes the semantics of the question, identifies keywords and query intent, and searches for relevant information in the scene description. Based on this information, the large language model activates its internal reasoning mechanism, which may include multi-step logical deduction, knowledge retrieval, and information integration.

[0047] For example, if the target question is "What color is the largest piece of furniture in the room?", the large language model will first understand that the question is asking about the color of the furniture, and specifically refers to the "largest" piece of furniture. Then, it will look for information about the size and color of the furniture in the scene text description. By comparing the size information of different pieces of furniture, it determines the largest piece of furniture, and then looks for the color description of the furniture. Finally, the model generates an answer that meets the requirements of the question, which is the first answer.

[0048] S103. If it is determined that the first answer is wrong, the first answer is input into the intelligent agent to obtain a second answer output by the intelligent agent. The second answer is inferred by the large language model after the intelligent agent performs reasoning reflection or knowledge supplement on the large language model based on the first answer.

[0049] Specifically, when the large language model outputs "cannot answer" as the first answer, it is considered as a first answer error. A first answer error means that the initial answer generated by the large language model cannot correctly or completely answer the given situational question. In the embodiment of the present invention, it can be understood that the answer output by the large language model does not match the expected correct answer, or the model directly indicates that the question cannot be answered. The first answer error is used to trigger the error correction mechanism of the intelligent agent, start the reasoning reflection and knowledge supplement process, and generate a more accurate second answer.

[0050] Furthermore, errors may come from two main reasons: errors in the logical reasoning of the large language model, or insufficient knowledge stored in the large language model.

[0051] The following two examples of actual scenarios illustrate these two situations: (1) Logical reasoning errors: Suppose there is a living room scene, and the question is: "If the sofa is red and the TV cabinet is brown, what color is the furniture closest to the window?". Scenario description: "There is a red sofa against the wall in the living room, a brown TV cabinet opposite the sofa, and a white coffee table between the sofa and the TV cabinet. There is a white bookshelf near the window."

[0052] In the above case, the large language model may not be able to correctly process the positional relationship and color information of multiple objects, resulting in an inability to draw conclusions. In fact, the correct reasoning should be: the bookshelf is closest to the window, and the bookshelf is white, so the answer should be "white".

[0053] (2) Insufficient knowledge: Suppose there is a kitchen scene and the question is: "What is that appliance with a HEPA filter in the kitchen used for?". Scenario description: "There is a square appliance in the corner of the kitchen with 'HEPA filter' marked on it. Next to it is a refrigerator and a microwave."

[0054] In the above case, the large language model lacks expertise about HEPA filters. HEPA filters are commonly used in air purifiers, but if the large language model does not have this knowledge reserve, it will not be able to make the correct inference. The correct answer should be: "This device is probably an air purifier, used to clean the air in the kitchen."

[0055] In specific implementation, the agent will first analyze the reason why the first answer is wrong, and the agent will activate the corresponding error correction mechanism. If it is an error in reasoning logic, the agent's reflection module will be activated. The reflection module analyzes each step in the reasoning process, identifies the key points that may cause errors, and generates correction strategies. These strategies may include reorganizing the reasoning steps, adjusting the reasoning logic, or introducing additional intermediate reasoning steps. If the knowledge reserve is insufficient, the agent will activate the refinement module. The refinement module retrieves relevant expertise or common sense information from the pre-established knowledge database and integrates this information into the reasoning process.

[0056] After completing reasoning reflection or knowledge supplementation, the agent will generate a new, optimized prompt. This prompt contains the original situational question, the content of the first answer, the reasoning logic after reflection, or the supplemented knowledge information. The agent then inputs this optimized prompt into the large language model to guide the model to make more in-depth and accurate reasoning. Based on this optimized prompt, the large language model combines its inherent language understanding and generation capabilities to generate the second answer.

[0057] Based on the above embodiment, as an optional embodiment, in step S103: if it is determined that the first answer is wrong, the step of inputting the first answer to the agent and obtaining the second answer output by the agent may further include the following steps: Step 301: input the first answer to the intelligent agent, and the intelligent agent calls the method database to match the first answer to determine whether there is a reasoning process with a similarity greater than a threshold, wherein the method database and the knowledge database are obtained by training the intelligent agent based on sample situational problems.

[0058] Among them, the method database refers to a structured storage system for storing and managing the reasoning methods and solution strategies used by the agent when dealing with situational problems. In the embodiment of the present invention, it can be understood as a data set containing a large number of reasoning process templates, each template corresponding to a specific type of situational problem and its solution. These templates contain information such as reasoning steps, logical structure, key decision points, etc., and are a summary of the experience of the agent in successfully solving problems in the past. The method database is used to improve the efficiency and accuracy of the agent in dealing with new problems. By matching similar problem types, the agent can quickly call appropriate reasoning strategies and avoid repeatedly constructing reasoning processes, thereby accelerating problem solving and improving the quality of answers.

[0059] Correspondingly, the knowledge database refers to a comprehensive knowledge storage system that contains a large amount of professional knowledge, common sense information and conceptual relationships related to situational problems. In an embodiment of the present invention, it can be understood as a multi-domain, multi-level structured knowledge graph, covering various types of knowledge such as object attributes, spatial relationships, causal relationships, etc. This database not only stores static facts, but also contains dynamically updated rules and reasoning chains. The knowledge database is used to provide the intelligent agent with the background knowledge and professional information required to answer situational problems. When the large language model faces a knowledge gap, the system can retrieve relevant information from the knowledge database and supplement the necessary knowledge, thereby supporting a more accurate and comprehensive reasoning process and improving the quality and reliability of the answer.

[0060] It is worth noting that both the method database and the knowledge database are obtained by training the agent based on sample situational problems. As the number of problems handled by the system increases, these databases will continue to be enriched and optimized. New successful reasoning processes will be added to the method database, and new domain knowledge will be integrated into the knowledge database. This continuous learning mechanism enables the agent to continuously improve its ability to handle complex situational problems as it is used, showing strong adaptability and scalability.

[0061] In the specific implementation, the agent first converts the first answer into a feature vector. Then, the agent calls the method database, which stores a large number of reasoning processes trained based on sample situational questions. Each stored reasoning process is represented as a corresponding feature vector. The agent uses cosine similarity or other appropriate similarity measurement methods to calculate the similarity between the feature vector of the first answer and each reasoning process in the database. If a reasoning process with a similarity greater than a preset threshold is found, the agent will mark it as a potential reference method.

[0062] Step 302: If there is an inference process with a similarity greater than the threshold, the inference process with a similarity greater than the threshold is input into the large language model.

[0063] In specific implementation, when the system finds that there are reasoning processes with similarities greater than a threshold, the system first extracts all reasoning processes with similarities exceeding a preset threshold from the method database. The system will integrate and adaptively adjust these reasoning processes to ensure that they match the current situational questions. Adjustments may include replacing specific entity names, adjusting the order of reasoning steps, or adding or deleting certain reasoning links. Subsequently, the system will use these adjusted reasoning processes as additional contextual information, together with the original situational question and the first answer, to construct a new, richer and more structured input. The system passes this input to the large language model, providing it with a clearer reasoning framework and reference basis.

[0064] Step 303: If there is no reasoning process with a similarity greater than the threshold, the knowledge information corresponding to the first answer in the knowledge database is called and the knowledge information is input into the large language model.

[0065] When the system finds that there is no reasoning process with a similarity greater than the threshold, it will first analyze the content of the first answer and extract the key concepts, entities and relationships. Then, the system uses this information to conduct a multi-dimensional search in the knowledge database to retrieve relevant knowledge information. This knowledge information may include various types such as concept definitions, attribute descriptions, causal relationships, spatial relationships, etc. The system will filter and integrate the retrieved knowledge information. Subsequently, the system constructs a new, knowledge-enhanced input with this filtered and integrated knowledge information together with the original situational question and the first answer. The system passes this input to the large language model, providing it with rich background knowledge and professional information, thereby supporting deeper and more accurate reasoning.

[0066] Step 304: Obtain a second answer output by the large language model.

[0067] In specific implementation, the system first receives the second answer output by the large language model. If the second answer is still "unable to answer", the system considers it an incorrect answer and starts a loop mechanism. This may include readjusting the knowledge supplement strategy, fine-tuning the reasoning process, or further expanding the scope of knowledge retrieval, and then inputting the optimized information into the large language model again to try to generate a new answer. If the second answer contains "display reasoning + answer", the system considers it a successful answer.

[0068] In a feasible implementation, the system can also verify successful answers. The verification process includes consistency checking, logic verification, completeness assessment, and knowledge matching. The system checks whether the reasoning process in the answer is consistent with the background information and knowledge provided, evaluates the logic of the reasoning process, ensures that each reasoning step is reasonable, checks whether the answer fully answers all aspects of the original question, and compares the answer with the information in the knowledge database to ensure that there are no obvious knowledge errors. If the verification process finds any problems, the system may trigger additional optimization cycles, such as re-adjusting the knowledge supplementation strategy or fine-tuning the reasoning process, and then inputting the large language model again to obtain improved answers.

[0069] The above embodiment illustrates the reasoning process of the situation problem through the intelligent agent. Figure 2 , Figure 2 1 is a flow chart of an agent training method provided by an embodiment of the present invention. The training process of the agent is described below. Specifically, the training process of the agent may include the following steps: Step 401: Obtain sample situational questions, where the sample situational questions include sample situational scenarios and sample target questions.

[0070] The sample situation questions refer to a set of pre-prepared questions used in the agent training process, which simulate various situations and queries that may be encountered in actual applications. In the embodiment of the present invention, it can be understood as a series of paired data containing sample situation scenes and sample target questions. The sample situation scenes are detailed descriptions of the three-dimensional indoor environment, including room layout, object distribution, object attributes and other information; and the sample target questions are specific inquiries based on these scenes.

[0071] Sample situational problems mainly provide rich visual scene data for the training of the subtitle model, helping the model learn how to accurately describe complex three-dimensional environments. Secondly, these problems provide a variety of reasoning exercises for the large language model, enabling it to learn how to perform logical reasoning in different situations. Furthermore, through sample situational problems, the system can build and optimize method databases and knowledge databases, and accumulate effective reasoning strategies and domain knowledge. Finally, sample problems are also used to evaluate and verify the performance of the entire intelligent agent system to ensure that it can accurately handle various types of situational problems. By using a large and diverse number of sample situational problems, the generalization ability of the agent can be significantly improved, enabling it to effectively deal with unknown situations in practical applications.

[0072] Step 402: Train the subtitle model integrated in the agent through sample situation scenarios to obtain sample key information output by the trained subtitle model.

[0073] During the training of the agent, the training of the caption model aims to improve the model's ability to understand and describe the three-dimensional environment. The caption model is responsible for converting complex three-dimensional scene information into structured text descriptions in the entire system, providing necessary input for the subsequent reasoning process.

[0074] Furthermore, the embodiment of the present invention uses a subtitle model as a perception module of the intelligent agent. Specifically, the perception module uses the subtitle model to locate all objects in the three-dimensional scene and generates a descriptive sentence for each object. The reasoning module further answers questions based on the subtitle information. However, due to the training of the subtitle model, the text similarity index is used as the standard during evaluation, resulting in the generated subtitles having a higher text similarity form, but performing poorly in terms of key descriptive information that has an important impact on reasoning and judgment, and there is a situation where the training evaluation index is inconsistent with the application scenario.

[0075] Based on the above embodiment, as an optional embodiment, in order to solve the above problem, the present invention adopts self-criticism sequence training and proposes a reinforcement learning method dedicated to key information by designing a reward function. Specifically, the process may also include the following steps: Step 501: Input a sample situational scenario into a subtitle model integrated in the agent, and obtain a sample text description output by the subtitle model.

[0076] Step 502: Evaluate the sample text description based on the loss function and the reward function to obtain an evaluation result, and adjust the subtitle model according to the evaluation result, wherein the loss function and the reward function are used to adjust the proportion of key information about objects in the sample context scene in the sample text description.

[0077] Among them, the loss function and the reward function are two complementary evaluation tools, which are used together to optimize the performance of the subtitle model. The loss function refers to a mathematical function used to measure the difference between the model prediction output and the actual target value. In the embodiment of the present invention, it can be understood as a measurement tool for evaluating the gap between the sample text description generated by the subtitle model and the ideal description. It usually takes the form of cross entropy loss or mean square error, etc., to guide parameter adjustment in the model learning process, help the model gradually reduce the prediction error, and improve the overall accuracy and quality of the generated text.

[0078] Correspondingly, the reward function is a mathematical function used to evaluate and motivate specific behaviors or outputs in the reinforcement learning framework. In the embodiment of the present invention, it can be understood as a specially designed scoring mechanism for adjusting the proportion of key information about objects in the sample context scene in the sample text description.

[0079] Furthermore, the design of the reward function takes into account multiple aspects, including the accuracy of key attribute descriptions, the completeness of spatial relationship descriptions, information density, and the ability to capture rare but important information. In this way, the reward function guides the model to pay more attention to the accurate description of the attributes and spatial relationships of important objects in the scene, thereby generating text descriptions that are more conducive to subsequent reasoning tasks.

[0080] For example, the loss function expression based on self-critical sequence training is: ; In the formula, Indicates subtitle candidates The loss function is represents the beam size of the beam search, that is, the number of candidate subtitles generated, represents the i-th subtitle candidate generated by beam search, represents the baseline caption generated by greedy search, Indicates the generation of subtitle candidates The reward function is represents the reward function for the baseline caption, Indicates subtitle candidates Length, Represents a given scene information Generate subtitles probability.

[0081] Specifically, the above loss function is based on the principle of reinforcement learning, and its purpose is to guide model optimization through the reward function R. The model generates multiple subtitle candidates through beam search and generates baseline subtitles through greedy search.

[0082] Among them, the reward function is the core of the entire self-critical sequence training. It not only considers the text similarity between the subtitles and the reference subtitles, but also pays attention to the generation of key adjectives in the subtitles. Its expression is: ; In the formula, represents the consensus-based image description evaluation index score, Indicates the correct set of key adjectives contained in the subtitles. Indicates the correct set of adjectives contained in the subtitles. Indicates the number of correct adjectives in the subtitles, Indicates the number of incorrect adjectives in the subtitles, represents the weight parameter used to balance the contribution of CIDEr score and key adjective reward.

[0083] Among them, the key information richness index is used to finally evaluate the quality of key information generation in subtitles. This index reflects the performance of the model in generating key descriptive information by comparing the number of correctly generated adjectives with the total number of adjectives. Its expression is: ; In the formula, n represents the number of subtitles, Indicates the total number of adjectives in the reference subtitle.

[0084] Furthermore, the above loss function ensures the basic quality and accuracy of the generated text, providing an overall optimization direction for the model. On this basis, the reward function further guides the model to focus on the most important information points for subsequent reasoning, thus achieving fine-tuning of the subtitle model.

[0085] The combination of loss function and reward function not only improves the overall performance of the model, but also makes the generated scene description more in line with the specific requirements of the situational question answering task. As a result, the subtitle model can generate scene descriptions that are both accurate and rich in key information, providing high-quality input for the subsequent reasoning process, thereby significantly improving the efficiency and accuracy of the entire intelligent system in dealing with complex situational problems.

[0086] Step 403: Obtain the first answer of the sample based on the sample key information and the sample target question through the large language model.

[0087] The implementation principle of step 403 is the same as that of step 102, and specific details may be referred to step 102, which will not be described in detail here.

[0088] Step 404: Perform reasoning and reflection on the large language model through the first answer of the sample to obtain a method database.

[0089] Specifically, through the perception module, the intelligent agent obtains information about the current environment and can perform zero-sample reasoning under prompts through the inherent knowledge of the large language model. The inherent capabilities of the large language model are sufficient to meet most situations, but there are still situations where effective reasoning cannot be formed. This is because answering situational questions belongs to a vertical professional field, and the out-of-the-box large language model lacks experience in methods for solving professional problems. Therefore, it is also necessary to teach it how to perform specific tasks and accumulate experience in solving problems in memory. In order to achieve this goal, an embodiment of the present invention proposes a reflection module, which can continuously learn from past question-and-answer reasoning experience, summarize methods for solving specific professional tasks, and gradually improve its reasoning performance.

[0090] Please refer to Figure 3 , Figure 3 FIG. 1 is a flow chart of a reflection module provided by an embodiment of the present invention. Based on the above embodiment, as an optional embodiment, the process may further include the following steps: Step 601: If it is determined that the first answer of the sample is wrong, the result of the wrong first answer is fed back to the large language model to obtain the thinking chain output by the large language model, and the thinking chain includes the correct sample first answer and the corresponding reasoning process.

[0091] Among them, the thought chain refers to a structured representation of the reasoning process, which records in detail the entire thinking path from question to answer. In the embodiment of the present invention, it can be understood as a series of logical reasoning steps generated by the large language model when processing situational problems, including the correct sample first answer and the detailed reasoning process to reach this answer. The thought chain is usually presented in a coherent narrative form, starting from problem understanding, through information extraction, knowledge application and step-by-step reasoning, and finally reaching a conclusion and self-verification.

[0092] During the training of the agent, when the system determines that the first answer of the sample is wrong, the chain of thinking that feeds this wrong result back to the large language model and obtains its output is an effective self-correction and learning mechanism. It enables the system to learn from mistakes, deeply understand the deficiencies in the reasoning process, and generate more accurate and explainable answers.

[0093] In specific implementation, the system first constructs a special prompt to input into the large language model. This prompt contains the original situation question, the wrong sample first answer, and a clear instruction requiring the model to rethink the problem and generate a detailed reasoning process.

[0094] After receiving the prompt, the large language model will start a deep self-reflection and re-reasoning mechanism. In this process, the model first analyzes the reasons for the wrong answer, which may be insufficient understanding of the information, incorrect reasoning logic, or improper application of knowledge. Then, the model will re-examine the key information in the question and design a step-by-step reasoning process. This process usually includes clarifying the premise, setting the intermediate reasoning steps, applying relevant knowledge, and finally reaching the correct conclusion. Each step of reasoning will be clearly stated to form a complete chain of thinking.

[0095] Step 602: Generate a method database based on the thought chain.

[0096] In specific implementation, the system first conducts in-depth analysis and structural processing of the acquired thought chain. This process involves identifying the key components of the thought chain, such as problem understanding, information extraction, knowledge application, reasoning steps, and conclusion generation. The system will use natural language processing technology, such as semantic analysis and information extraction algorithms, to parse the content of the thought chain and extract the core reasoning logic and key decision points.

[0097] Next, the system converts the extracted information into standardized reasoning templates, which contain not only the logical structure of the reasoning, but also metadata such as the applicable question types, required prerequisites, possible variants, etc.

[0098] In a feasible implementation, in order to improve retrieval efficiency and application flexibility, the system uses vector representation technology to encode each reasoning template into a high-dimensional vector. This representation method allows the system to quickly find the most relevant reasoning strategy when facing new problems.

[0099] Through the above-mentioned thought chain technology, the large language model can find unexplained reasons and understand and summarize the correct way to answer questions. Finally, the correct reasoning process and correction results are saved in the method database memory of the reflection module and represented as the index by the embedding vector of the question, which is used for the subsequent retrieval by the reasoning module based on the cosine similarity of the question.

[0100] Step 405: extract knowledge from the large language model through the first answer of the sample to obtain a knowledge database.

[0101] After being able to perceive scene information and learn how to solve related problems, the intelligent agent has greatly improved in answering questions. However, due to the inherent point cloud sparsity and cluttered object distribution of three-dimensional scenes, the subtitle generation model cannot generate information about all objects, and there is inevitable information missing, which also leads to reasoning failure due to information missing.

[0102] To address this situation, an embodiment of the present invention designs an extraction module that attempts to extract the prototype knowledge information of the object in the training set. This knowledge information is independent of specific scenario situations and has universal properties. Therefore, it can be used as the inherent knowledge information of the intelligent agent to supplement the missing information and be used during reasoning.

[0103] Please refer to Figure 4 , Figure 4 Schematic diagram of a process of a refining module provided by an embodiment of the present invention. Specifically, the process may further include the following steps: Step 701: Take the sample first answer, sample key information and sample target question corresponding to the same object in the sample situation scene as a triple, and extract the feature vector of the triple.

[0104] In the training process of the intelligent agent, in order to solve the reasoning failure problem caused by the lack of three-dimensional scene information, the embodiment of the present invention combines the sample first answer, sample key information and sample target question corresponding to the same object in the sample context scene into triples, and extracts the feature vectors of these triples. In this way, the multi-dimensional information of the object in different scenes and problem contexts is captured, laying the foundation for subsequent knowledge extraction and generalization.

[0105] In specific implementation, the system first identifies and extracts relevant information about the same object from sample contextual scenes. The above-mentioned "same object" may appear in different scenes with different attributes or states. For each identified object instance, the system collects the sample first answer, sample key information, and sample target questions related to it. The sample first answer contains the initial understanding and description of the object by the large language model; the sample key information is the key description related to the object in the scene extracted by the subtitle generation model; and the sample target question reflects the query type related to the object.

[0106] Combining the above three types of information into a triple provides a comprehensive description of the object, including its attributes, functions, and performance in different contexts. Secondly, this combination captures the intrinsic relationship between objects, scene descriptions, and questions, which helps to understand the role and importance of objects in different contexts.

[0107] Step 702: Obtain the centroid of the feature vectors corresponding to the same object by clustering.

[0108] Specifically, during the agent training process, the centroid obtained by clustering the corresponding feature vectors of the same object is obtained, aiming to extract the core features and common properties of the object from a large number of samples, thereby effectively dealing with the incomplete information and noise problems caused by the sparsity of point clouds and the cluttered distribution of objects in three-dimensional scenes, thereby providing a more reliable foundation for subsequent knowledge generalization and reasoning.

[0109] During specific implementation, the system first collects all feature vectors corresponding to the same object in different sample scenario scenes. Among them, each feature vector represents the comprehensive characteristics of the object in a specific scenario and problem situation. Next, the system performs cluster analysis on these feature vectors. The choice of clustering algorithm can be determined according to the characteristics and distribution of the data. Commonly used methods include K-means clustering, hierarchical clustering or density clustering. In this embodiment, K-means clustering is taken as an example for explanation. The core idea of ​​the K-means algorithm is to group similar feature vectors into the same cluster and calculate the center point of each cluster, that is, the centroid.

[0110] In one possible implementation, the process of K-means clustering is as follows: First, the system needs to determine the number of clusters K. This K value can be determined by heuristic methods or cross-validation. Then, the algorithm randomly selects K initial centroids and assigns each feature vector to the cluster represented by the nearest centroid. Next, the algorithm recalculates the centroid of each cluster, which is the average of all vectors in the cluster, and reallocates the feature vector to the nearest new centroid. This process will be iterated until the centroid position no longer changes significantly or the preset number of iterations is reached.

[0111] Step 703: input the knowledge information corresponding to the centroid into the large language model, generalize the knowledge information corresponding to the centroid through the large language model, and obtain sample knowledge information.

[0112] Specifically, the knowledge information corresponding to the centroid is input into the large language model and the sample knowledge information is obtained through generalization. The highly abstract numerical representation obtained by clustering in the previous step can be converted into more general, easy-to-understand and applicable semantic knowledge.

[0113] In specific implementation, the system first needs to convert the numerical representation of the center of mass back into understandable text form. This process may involve reverse mapping or decoding technology to convert high-dimensional vectors into a series of keywords, phrases or sentences. These text descriptions may include information such as the basic properties of the object, common uses, typical scenarios, etc. For example, for the object "chair", the converted text may include descriptions such as "has four legs", "used for sitting", and "commonly found in living rooms and offices".

[0114] Next, the system provides these preliminary converted text descriptions as input to the large language model. The large language model receives these preliminary descriptions generated based on centroids and deeply processes and expands this information using its huge pre-trained knowledge base and strong language understanding capabilities.

[0115] During the processing, the large language model supplements and completes the information that may be missing in the initial description. For example, if the initial description only mentions the basic structure of the chair, the model may add general information about materials, style, comfort, etc. Secondly, the model generates more abstract and general descriptions so that this knowledge can be applied to a wider range of scenarios. Thirdly, the model considers the variants and special uses of objects in different situations, thereby increasing the diversity and applicability of knowledge. Finally, the model may also establish connections between objects and other related concepts to form a small knowledge network, namely sample knowledge information. Sample knowledge information is no longer limited to specific scenarios or specific types of questions, but includes the general characteristics, functions, variants of objects, and associations with other concepts.

[0116] For example, for "chair", sample knowledge information may include: "A chair is a piece of furniture used for sitting, usually with four legs and a backrest. It can be made of wood, metal, plastic and other materials. There are many types of chairs, including dining chairs, office chairs, leisure chairs, etc., suitable for different occasions. In some designs, chairs may not have legs but use other supporting structures. Chairs are often used in conjunction with furniture such as tables and sofas."

[0117] Step 704: Generate a knowledge database based on the sample knowledge information.

[0118] Step 406: Integrate the method database and the knowledge database into the intelligent agent.

[0119] The above embodiments respectively illustrate the training process of the perception module, reflection module and refinement module in the intelligent agent. Figure 5 , Figure 5 FIG. 1 is a flow chart of a reasoning module provided by an embodiment of the present invention. The application process of the reasoning module will be described below.

[0120] Specifically, in the agent's reasoning process, the reasoning module integrates the outputs of the perception module, reflection module, and refinement module, and combines the powerful capabilities of the large language model to realize a complex and efficient reasoning system.

[0121] The reasoning process starts with reading the scene information and converting it into the agent's own perspective state. It provides the agent with a self-centered understanding of the environment. The system reads in all the object captions and detection box information generated by the perception module, and uses the rotation-translation seven-tuple of the scene position to generate the corresponding rotation matrix and translation matrix. Through this conversion, the object's position information is mapped to the agent's current perspective, so that subsequent reasoning can be based on the agent's "subjective" perception. This method simulates the human cognitive process in the actual environment and helps to generate more natural and intuitive reasoning results.

[0122] Next, the system generates the prompts required for reasoning. The design of the prompts contains three key components: system prompts, text descriptions of the scene, and reasoning guidance. The system prompts outline the basic requirements of the situational question answering task and set the framework of the task for the large language model. The scene text description uses natural language to comprehensively present the attributes and location information of the object. In particular, the objects are divided into four direction categories according to their relative positions: front, back, left, and right. This classification method helps the large language model better understand spatial relationships. Reasoning guidance provides a basic explicit and interpretable reasoning process, or integrates the experience in the reflection module and the refinement module when necessary. This multi-level prompt design is designed to provide sufficient context and guidance for the large language model, enabling it to make more accurate and targeted reasoning.

[0123] Inputting the generated prompts into the large language model takes into account the complexity of the situational question answering task, and the system adopts the thought chain prompt technology. This improves the transparency and explainability of the reasoning process, so that each step of the reasoning can be tracked and understood. Secondly, it can alleviate the hallucination phenomenon that may be caused by the large language model, because each step of the reasoning needs to be supported by logic. Finally, when an explainable reasoning cannot be formed, the system will promptly output "unable to answer". This honest feedback mechanism increases the credibility of the system.

[0124] During the reasoning process, if an unexplainable situation occurs, the intelligent body will activate the reflection and refinement module. The system embeds the problem of the current scene into a vector for similarity query in the method database of the reflection module. If a situation with a similarity higher than the threshold is found, the system will use it as a reference, generate method experience in combination with the corresponding reasoning process, and integrate it into the prompt generator. If the similarity is lower than the threshold, the system will turn to the knowledge database for supplementation and integrate the relevant knowledge experience into the prompt. This method of dynamically calling experience and knowledge greatly enhances the adaptability and reasoning ability of the system, enabling it to handle more diverse and complex situational problems.

[0125] Finally, if the agent still cannot form a valid reasoning, the system will enable a supervised learning tool that is trained end-to-end under the contextual question answering task. This tool provides candidate answer options for the large language model to determine whether it can form an explainable reasoning. This step serves as the final safeguard mechanism to ensure that the system can give reasonable responses in various situations while maintaining the explainability of reasoning.

[0126] Through the above multi-level, closed-loop intelligent agent, it greatly improves the ability of the intelligent agent to handle complex three-dimensional scene problems, and can give reasonable reasoning even when the information is incomplete or noisy. Secondly, through the thinking chain technology and experience recall mechanism, the system achieves a high degree of interpretability, making the reasoning process transparent and traceable. Furthermore, the design of dynamically calling reflection and refinement modules enhances the adaptability and learning ability of the system, enabling it to continuously accumulate experience from new situations and improve its reasoning ability.

[0127] On this basis, please refer to Figure 6 and Figure 7 , Figure 6 A display diagram of the answer results of a situational question provided by an embodiment of the present invention, Figure 7 A diagram showing the results of answering a situational question provided in an embodiment of the present invention.

[0128] Figure 8 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 8 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the method for reasoning about situational issues based on a large language model.

[0129] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0130] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the methods for reasoning about situational problems based on a large language model provided by the above methods.

[0131] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the method for performing situational question reasoning based on a large language model provided by the above methods.

[0132] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0133] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for reasoning about situational issues based on a large language model, characterized in that: include: Get contextual questions; Input the situational question into the intelligent agent, and obtain a first answer output by the intelligent agent, wherein the first answer is inferred based on the situational question by a large language model integrated in the intelligent agent; If it is determined that the first answer is wrong, the first answer is input to the agent to obtain a second answer output by the agent, where the second answer is inferred by the large language model after the agent performs reasoning reflection or knowledge supplement on the large language model based on the first answer; Wherein, the intelligent agent is trained based on sample situational problems.

2. The method for reasoning about situational issues based on a large language model according to claim 1, characterized in that: The situational question includes a situational scenario and a target question, and the step of inputting the situational question into the intelligent agent and obtaining a first answer output by the intelligent agent includes: Inputting the situation scene into a caption model integrated in the intelligent agent, generating a text description of key information corresponding to the object in the situation scene through the caption model, and inputting the text description into the large language model; Inputting the target question into the large language model, and using the large language model to infer the target question based on the text description to obtain a first answer; The subtitle model is obtained by training the agent based on sample situational questions.

3. The method for reasoning about situational issues based on a large language model according to claim 1, characterized in that: The step of inputting the first answer to the agent and obtaining a second answer output by the agent comprises: Inputting the first answer to the agent, matching the first answer with the method database through the agent, and determining whether there is a reasoning process with a similarity greater than a threshold; If there is an inference process with a similarity greater than the threshold, input the inference process with a similarity greater than the threshold into the large language model; If there is no reasoning process with a similarity greater than a threshold, calling the knowledge information corresponding to the first answer in the knowledge database, and inputting the knowledge information into the large language model; Obtaining a second answer output by the large language model; Wherein, the method database and the knowledge database are obtained by training the intelligent agent based on sample situational problems.

4. The method for reasoning about situational issues based on a large language model according to any one of claims 1 to 3, characterized in that: The training process of the agent includes: Obtaining sample situational questions, wherein the sample situational questions include sample situational scenarios and sample target questions; Training the subtitle model integrated in the agent through sample situation scenarios to obtain sample key information output by the trained subtitle model; Obtaining a first answer to the sample based on the sample key information and the sample target question through a large language model; Performing reasoning and reflection on the large language model through the first answer of the sample to obtain a method database; Extracting knowledge from the large language model through the first answer of the sample to obtain a knowledge database; The method database and the knowledge database are integrated into the intelligent agent.

5. The method for reasoning about situational issues based on a large language model according to claim 4, characterized in that: The training of the subtitle model integrated in the agent through the sample situation scene includes: Inputting a sample situational scenario into a subtitle model integrated in the agent, and obtaining a sample text description output by the subtitle model; Evaluate the sample text description based on the loss function and the reward function to obtain an evaluation result, and adjust the subtitle model according to the evaluation result; The loss function and the reward function are used to adjust the proportion of key information about objects in the sample context scene in the sample text description.

6. The method for reasoning about situational issues based on a large language model according to claim 4, characterized in that: The method database is obtained by performing reasoning and reflection on the large language model through the first answer of the sample, including: If it is determined that the first answer of the sample is wrong, the result that the first answer is wrong is fed back to the large language model to obtain a thought chain output by the large language model, wherein the thought chain includes a correct first answer of the sample and a corresponding reasoning process; A method database is generated based on the thought chain.

7. The method for reasoning about situational issues based on a large language model according to claim 4, characterized in that: The step of extracting knowledge from the large language model through the first answer of the sample to obtain a knowledge database includes: The sample first answer, sample key information and sample target question corresponding to the same object in the sample situation scene are taken as a triple, and a feature vector of the triple is extracted; Get the centroid obtained by clustering the corresponding feature vectors of the same object; Inputting the knowledge information corresponding to the centroid into the large language model, generalizing the knowledge information corresponding to the centroid through the large language model, and obtaining sample knowledge information; A knowledge database is generated based on the sample knowledge information.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the method for reasoning about situational issues based on a large language model as described in any one of claims 1 to 7 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for reasoning about situational issues based on a large language model as described in any one of claims 1 to 7 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for reasoning about situational issues based on a large language model as described in any one of claims 1 to 7 is implemented.

Images