Method and system for enabling large model to have spatial reasoning and long-term planning capabilities

Abstract

The invention discloses a method and system for enabling a large model to have spatial reasoning and long-term planning capabilities, and belongs to the technical field of artificial intelligence, and the method is implemented by the following steps: space-relation-graph conversion: converting complex spatial description into a graph structure which can be efficiently processed by a computer; adaptive reasoning: combining the language understanding ability of the large language model and the structure perception ability of the graph neural network to realize a reasoning mechanism; curriculum reinforcement learning: through curriculum reinforcement learning, a progressive learning strategy is used, so that the system can gradually master spatial reasoning tasks from simple to complex; by means of meta-cognitive monitoring, the system can evaluate the decision-making quality of the system in real time and adjust the decision-making quality when necessary; training and optimization: adopting an end-to-end training mode, and combining supervised learning and reinforcement learning to realize optimization. According to the method, the performance and robustness of the model in a complex space task are remarkably improved, and the adaptability and interpretability of the model are enhanced.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and system for enabling a large model to have spatial reasoning and long-term planning capabilities. Background Art

[0002] Large language models (LLMs) have performed well in a variety of natural language processing tasks, but they still have obvious limitations in spatial reasoning and long-term planning. These limitations mainly stem from two problems: spatial hallucinations and context inconsistency hallucinations in long-term reasoning. Existing solutions include instruction fine-tuning, prompt engineering, and reinforcement learning, but these methods are still difficult to completely solve the above problems. For example, in maze pathfinding tasks, traditional LLMs often cannot accurately understand spatial relationships, resulting in unreasonable path planning. In long-term planning tasks, LLMs are prone to context inconsistency problems, making the generated plans inconsistent. These problems seriously limit the application of LLMs in space-related tasks.

[0003] For LLMs, the existing technology mainly has the following problems: 1. LLMs have difficulty in accurately understanding and representing complex spatial relationships, resulting in limited spatial reasoning ability.

[0004] 2. In long-term planning tasks, LLMs are prone to context inconsistency, which affects the coherence and rationality of planning.

[0005] 3. Existing methods find it difficult to effectively combine the language understanding capabilities of LLMs and the spatial representation capabilities of graph structures.

[0006] 4. There is a lack of adaptive learning mechanisms for spatial reasoning tasks, making it difficult to cope with tasks of different difficulty levels.

[0007] 5. Existing methods lack computational efficiency and scalability when dealing with large-scale and complex spatial reasoning tasks.

[0008] These problems lead to unsatisfactory performance of LLMs in spatial reasoning and long-term planning tasks, limiting their application in areas such as robot navigation and intelligent traffic planning. Summary of the invention

[0009] The technical task of the present invention is to address the above shortcomings and provide a method and system for enabling large models to have spatial reasoning and long-term planning capabilities, overcome the limitations of current large language models in spatial reasoning and long-term planning tasks, form a powerful and flexible system, and solve the challenges faced by existing large language models in spatial reasoning and long-term planning tasks. It not only significantly improves the performance and robustness of the model in complex spatial tasks, but also enhances its adaptability and interpretability.

[0010] The technical solution adopted by the present invention to solve the technical problem is: A method to enable large models to have spatial reasoning and long-term planning capabilities, the implementation of which includes: Space-relation-graph conversion: converting complex spatial descriptions into graph structures that can be efficiently processed by computers; Adaptive reasoning: Combining the language understanding ability of large language models and the structural perception ability of graph neural networks, a flexible and powerful reasoning mechanism is realized; Curriculum reinforcement learning, through curriculum reinforcement learning, using progressive learning strategies, enables the system to gradually master spatial reasoning tasks from simple to complex; Metacognitive monitoring, which enables the system to assess the quality of its own decisions in real time and make adjustments if necessary; Training and optimization, adopts an end-to-end training method, combining supervised learning and reinforcement learning to achieve optimization.

[0011] Furthermore, the space-relationship-graph conversion specifically includes: Spatial description analysis: Leverage the latest large language models (such as GPT-4 or Qwen2-72B) to understand and parse spatial descriptions in natural language; these advanced models have strong language understanding capabilities and can accurately capture complex spatial relationship descriptions; Relationship diagram construction: The parsed structured information is converted into a relationship graph ,in is a collection of entities, is a set of relationships; in this graph, each entity becomes a node , and the spatial relationships between them are represented as edges connecting these nodes. ; Graph Embedding: Encode the constructed relationship graph using advanced graph neural networks such as GraphSAGE or GAT; this process generates a graph-level embedding that represents the entire scene and node-level embedding ; The process of graph embedding can be expressed by the following mathematical formula: ; Among them, GNN represents graph neural network operation, is the input relationship diagram, is a graph-level embedding, is the embedding set of all nodes.

[0012] Furthermore, the adaptive reasoning is specifically implemented as follows: Status indicates: Construct a comprehensive state representation: , this state contains global scene information , information about the current entity of interest And task-related context information context; Action generation: In the action generation phase, we use the advantages of large language models and graph neural networks to achieve the following: ,in represents the weighted fusion operation, represents the action chosen by the system at time step t; Adaptive weight calculation: In order to dynamically balance the contributions of the language model and the graph model, an adaptive weight mechanism is introduced: ,in Represents the sigmoid function; The final action selection formula is: .

[0013] Furthermore, the course reinforcement learning specifically includes: Graph complexity calculation: First, define a graph complexity metric: ; in, is the number of nodes, is the number of edges, is the diameter of the graph; Dynamic course generation: Based on graph complexity , using a large language model to dynamically generate a series of subtasks of increasing difficulty; Hierarchical Reinforcement Learning: For each generated subtask, the Proximal Policy Optimization (PPO) algorithm is applied for training; the optimization objective function is: ; in, is the policy entropy, used to encourage exploration; is the discount factor, It’s an instant reward. is the entropy regularization coefficient.

[0014] Furthermore, the metacognitive monitoring specifically includes: Uncertainty estimation: Define the uncertainty estimation function as: ; in, is the action probability distribution; Reasoning Path Adjustment: When Uncertainty Exceeding the preset threshold , triggering a replanning process performed by the large language model; Experience accumulation: Continuously record high-quality decision sequences into an experience pool 𝐷; these high-quality experiences will be used to regularly update the strategy network and value network.

[0015] Furthermore, the total loss function of the training and optimization is: ,in: is the supervised learning loss, using cross entropy to measure the difference between the predicted output and the true label; is the reinforcement learning loss, that is, maximizing the objective function mentioned above ; is the metacognitive monitoring loss, which uses mean squared error to optimize the accuracy of uncertainty estimation; and is the coefficient that balances the different loss terms.

[0016] Furthermore, the specific implementation process of this method is as follows: Initialization phase: First, load the pre-trained large language model (such as GPT-4 or Qwen2-72B) and graph neural network model; at the same time, initialize the experience pool 𝐷 and course task queue to prepare for continuous learning and course learning; Task reception: Receive a spatial reasoning task, such as "find the shortest path from point A to point B in a given room layout"; this task is usually given in natural language, containing rich spatial information and target description; Spatial-relational-graph conversion: convert the task described in natural language into a graph structure that can be processed by computers; the specific steps are as follows: Use a large language model to parse the task description and extract key entities and relations; Constructing the initial relationship diagram ,in is a collection of entities, is a set of relations; Generating graph-level embeddings using graph neural networks and node-level embeddings , this process can be expressed as: ; The importance of this step lies in converting complex spatial information into structured data representation; Curriculum generation: designing a progressive learning strategy that enables the system to start with simple tasks and gradually master more complex spatial reasoning capabilities; First, calculate the graph complexity ; based on , using a large language model to generate a series of subtasks of increasing difficulty; this curriculum learning method significantly improves the learning efficiency and generalization ability of the system; Iterate each subtask: This is the core loop of the system to perform spatial reasoning, which includes the following steps: a. State initialization: set the initial state , including graph embedding, current node embedding and task context; b. For each time step Do the following: Adaptive reasoning: Combining large language models and graph neural networks to achieve adaptive reasoning: Calculate the LLM prediction: , Compute GNN predictions: , Calculate adaptive weights: , Generate Action: ; Metacognitive Monitoring: Calculate uncertainty: , if , triggering re-planning, Otherwise, perform the selected action ; Environment Interaction: Execute an action , get a new state and rewards , Will Deposit into experience pool 𝐷; Determine whether the subtask goal or maximum number of steps has been reached: If yes, end the current subtask. If no, , continue to the next time step; c. Training and optimization: Improve performance through continuous learning and optimization: Sample batch data from the experience pool 𝐷, Calculate the loss: , Back propagation, updating model parameters.

[0017] The present invention also claims a system that enables a large model to have spatial reasoning and long-term planning capabilities, comprising: The space-relationship-graph conversion module is used to convert complex spatial descriptions into graph structures that can be efficiently processed by computers; Adaptive reasoning module, which combines the language understanding ability of large language models with the structural perception ability of graph neural networks to achieve a flexible and powerful reasoning mechanism; The curriculum reinforcement learning module is used to enable the system to gradually master spatial reasoning tasks from simple to complex through curriculum reinforcement learning; A metacognitive monitoring module that enables the system to assess the quality of its own decisions in real time and make adjustments when necessary; The training and optimization module uses an end-to-end training method and combines supervised learning and reinforcement learning to achieve optimization; The system implements large model operations based on the above-mentioned method of enabling large models to have spatial reasoning and long-term planning capabilities.

[0018] The present invention also claims a device that enables a large model to have spatial reasoning and long-term planning capabilities, characterized in that it includes: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is used to call the machine-readable program to implement the above method.

[0019] The present invention also claims protection for a computer-readable medium having computer instructions stored thereon, which implement the above method when executed by a processor.

[0020] Compared with the prior art, the method and system of the present invention for enabling a large model to have spatial reasoning and long-term planning capabilities have the following beneficial effects: The adaptive spatial reasoning framework (ALGNN-SR) that integrates large language models and graph neural networks effectively solves the challenges faced by existing large language models in spatial reasoning and long-term planning tasks through its innovative core components such as space-relation-graph transformation, adaptive reasoning, curriculum reinforcement learning, and metacognitive monitoring. This approach not only significantly improves the performance and robustness of the model in complex spatial tasks, but also enhances its adaptability and interpretability. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic diagram showing the principle of a method provided by an embodiment of the present invention for enabling a large model to have spatial reasoning and long-term planning capabilities. DETAILED DESCRIPTION

[0022] The present invention will be further described below in conjunction with specific embodiments.

[0023] The embodiment of the present invention provides a method for enabling a large model to have spatial reasoning and long-term planning capabilities, and provides an adaptive spatial reasoning framework (ALGNN-SR) that integrates a large language model and a graph neural network. The implementation of the method includes: 1. Space-relationship-graph conversion: Convert complex spatial descriptions into graph structures that can be efficiently processed by computers; this process is divided into three key steps: 1.1、Space description analysis: Leverage the latest large language models (such as GPT-4 or Qwen2-72B) to understand and parse spatial descriptions in natural language; these advanced models have strong language understanding capabilities and can accurately capture complex spatial relationship descriptions.

[0024] 1.2. Relationship diagram construction: Convert the parsed structured information into a relationship graph ,in is a collection of entities, is a set of relationships. In this graph, each entity becomes a node , and the spatial relationships between them are represented as edges connecting these nodes. .

[0025] 1.3. Graph Embedding: Encode the constructed relationship graph using advanced graph neural networks such as GraphSAGE or GAT. This process generates a graph-level embedding that represents the entire scene. and node-level embedding ; The process of graph embedding can be expressed by the following mathematical formula: ; Among them, GNN represents graph neural network operation, is the input relationship diagram, is a graph-level embedding, is the embedding set of all nodes.

[0026] 2. Adaptive reasoning: Combining the language understanding ability of the large language model and the structural perception ability of the graph neural network, a flexible and powerful reasoning mechanism is realized. The specific implementation is as follows: 2.1, Status Representation: Construct a comprehensive state representation: , this state contains global scene information ( ), information about the current focused entity ( ) and task-related context information (context).

[0027] 2.2, Action Generation: In the action generation phase, we use the advantages of large language models and graph neural networks to achieve the following: ,in represents the weighted fusion operation, represents the action chosen by the system at time step t.

[0028] 2.3. Adaptive weight calculation: In order to dynamically balance the contributions of the language model and the graph model, an adaptive weight mechanism is introduced: ,in Represents the sigmoid function; The final action selection formula is: .

[0029] 3. Course reinforcement learning: Through course reinforcement learning, using progressive learning strategies, the system can gradually master spatial reasoning tasks from simple to complex. Specifically, it includes: 3.1. Graph complexity calculation: First, define a graph complexity metric: ; in, is the number of nodes, is the number of edges, is the diameter of the graph; 3.2. Dynamic course generation: Based on graph complexity , using a large language model to dynamically generate a series of subtasks of increasing difficulty; 3.3. Hierarchical reinforcement learning: For each generated subtask, the Proximal Policy Optimization (PPO) algorithm is applied for training; the optimization objective function is: ; in, is the objective function, which means that The overall expected return under It is in strategy Expected value under is a discount factor, which is used to balance the importance of immediate rewards and future rewards. Its value range is usually between [0,1]; is the immediate reward obtained at time step t; is the entropy regularization coefficient, which is used to control the degree of exploration; It’s a strategy Entropy, used to encourage exploration; are the parameters of the policy network; This formula is a commonly used objective function in the PPO (Proximal Policy Optimization) algorithm, which combines two main parts: Cumulative discount rewards: ; Entropy regularization term: ; The formula takes into account both reward maximization and sufficient exploration through the entropy regularization term.

[0030] 4. Metacognitive monitoring, which enables the system to evaluate its own decision quality in real time and make adjustments when necessary. Specifically, it includes: 4.1. Uncertainty estimation: The uncertainty estimation function is defined as: ; in, is the action probability distribution; 4.2. Reasoning Path Adjustment: When Uncertainty Exceeding the preset threshold , triggering a replanning process performed by the large language model; 4.3. Experience accumulation: Continuously record high-quality decision sequences into an experience pool 𝐷; these high-quality experiences will be used to regularly update the strategy network and value network.

[0031] 5. Training and optimization: end-to-end training is used, combining supervised learning and reinforcement learning to achieve optimization. The total loss function is: ; in: is the supervised learning loss, using cross entropy to measure the difference between the predicted output and the true label; is the reinforcement learning loss, that is, maximizing the objective function mentioned above ; is the metacognitive monitoring loss, which uses mean squared error to optimize the accuracy of uncertainty estimation; and is the coefficient that balances the different loss terms.

[0032] In order to fully understand how this method framework (ALGNN-SR) coordinates the various modules to work together, the following is the overall execution flow applied to the system. This flow shows the entire process from task reception to final completion.

[0033] 1. Initialization phase: The system first loads the pre-trained large language model (such as GPT-4 or Qwen2-72B) and graph neural network model; at the same time, the system initializes the experience pool 𝐷 and the course task queue to prepare for continuous learning and course learning.

[0034] 2. Task reception: The system receives a spatial reasoning task, such as "find the shortest path from point A to point B in a given room layout"; this task is usually given in natural language and contains rich spatial information and target description.

[0035] 3. Spatial-Relational-Graph Transformation (SRG-TM module): This key step converts the task described in natural language into a graph structure that can be processed by computers. The specific steps are as follows: Use a large language model to parse the task description and extract key entities and relations; Constructing the initial relationship diagram ,in is a collection of entities, is a set of relations; Generating graph-level embeddings using graph neural networks and node-level embeddings , this process can be expressed as: ; The importance of this step lies in converting complex spatial information into structured data representation.

[0036] Curriculum Generation (CRLM module): Design a progressive learning strategy that enables the system to start with simple tasks and gradually master more complex spatial reasoning capabilities: First, calculate the graph complexity ; based on , using a large language model to generate a series of subtasks of increasing difficulty.

[0037] This curriculum learning method significantly improves the learning efficiency and generalization ability of the system.

[0038] Iterate each subtask: This is the core loop of the system to perform spatial reasoning, which includes the following steps: a. State initialization: set the initial state , including graph embedding, current node embedding and task context.

[0039] b. For each time step Do the following: Adaptive reasoning (ARM module): Combining large language models and graph neural networks to achieve adaptive reasoning: Calculate the LLM prediction: , Compute GNN predictions: , Calculate adaptive weights: , Generate Action: ; Metacognitive Monitoring (MCM module): Calculate uncertainty: , if , triggering re-planning, Otherwise, perform the selected action ; Environment Interaction: Execute an action , get a new state and rewards , Will Deposit into experience pool 𝐷; Determine whether the subtask goal or maximum number of steps has been reached: If yes, end the current subtask. If no, , continue to the next time step; c. Training and optimization: The system improves performance through continuous learning and optimization: Sample batch data from the experience pool 𝐷, Calculate the loss: , Back propagation, updating model parameters.

[0040] The above process shows how the ALGNN-SR framework systematically solves complex spatial reasoning tasks. Each module has its own unique role: the SRG-TM module concretizes abstract tasks into computable graph structures; the CRLM module designs intelligent learning paths; the ARM module fuses multi-source information at each decision point; and the MCM module monitors the quality of decisions throughout the process. This modular design not only improves the performance and reliability of the system, but also enhances its interpretability and scalability.

[0041] The main advantages of the ALGNN-SR framework proposed in this method are: 1. Accurate spatial representation: Through the space-relationship-graph conversion module, efficient conversion of natural language spatial descriptions into computer-processable graph structures is achieved, laying the foundation for accurate spatial reasoning.

[0042] 2. Enhanced reasoning capabilities: The adaptive reasoning module cleverly combines the language understanding capabilities of the large language model and the structural perception capabilities of the graph neural network to achieve more flexible and accurate spatial reasoning.

[0043] 3. Progressive learning: The curriculum reinforcement learning module designs an intelligent learning path, enabling the system to gradually master spatial reasoning tasks from simple to complex, significantly improving learning efficiency and generalization ability.

[0044] 4. Self-monitoring and error correction: The metacognitive monitoring module provides the system with the ability of "self-reflection", which can timely identify and correct potential wrong decisions, thereby improving the reliability of the system.

[0045] 5. Enhanced explainability: Through clear modular design and visualization of the decision-making process, the explainability of system decisions is improved, which helps users understand and trust the output of the system.

[0046] The ALGNN-SR framework is applicable to a variety of complex spatial reasoning and long-term planning tasks, especially when faced with incomplete or ambiguous spatial information, such as robot navigation, intelligent traffic planning, virtual reality scene generation, and complex strategy game AI. This innovative approach opens up new possibilities for the advancement of artificial intelligence in spatial cognition and is expected to have an important impact in multiple fields.

[0047] An embodiment of the present invention also provides a system that enables a large model to have spatial reasoning and long-term planning capabilities. The system implements large model operations based on the adaptive spatial reasoning framework (ALGNN-SR) that integrates a large language model and a graph neural network as described in the above embodiment.

[0048] The system includes: 1. Space-Relation-Graph Transformation Module (SRG-TM), The SRG-TM module is the cornerstone of the ALGNN-SR framework. It is responsible for converting complex spatial descriptions into graph structures that can be efficiently processed by computers. This process is divided into three key steps: 1.1、Space description analysis: Leverage the latest large language models (such as GPT-4 or Qwen2-72B) to understand and parse spatial descriptions in natural language; these advanced models have strong language understanding capabilities and can accurately capture complex spatial relationship descriptions.

[0049] 1.2. Relationship diagram construction: Convert the parsed structured information into a relationship graph ,in is a collection of entities, is a set of relationships. In this graph, each entity becomes a node , and the spatial relationships between them are represented as edges connecting these nodes. .

[0050] 1.3. Graph Embedding: Encode the constructed relationship graph using advanced graph neural networks such as GraphSAGE or GAT. This process generates a graph-level embedding that represents the entire scene. and node-level embedding ; The process of graph embedding can be expressed by the following mathematical formula: ; Among them, GNN represents graph neural network operation, is the input relationship diagram, is a graph-level embedding, is the embedding set of all nodes.

[0051] 2. Adaptive Reasoning Module (ARM), The ARM module is the core of the ALGNN-SR framework, which cleverly combines the language understanding ability of the large language model and the structural perception ability of the graph neural network to achieve a flexible and powerful reasoning mechanism. It includes: 2.1, Status Representation: Construct a comprehensive state representation: , this state contains global scene information ( ), information about the current focused entity ( ) and task-related context information (context).

[0052] 2.2, Action Generation: In the action generation phase, we use the advantages of large language models and graph neural networks to achieve the following: ,in represents the weighted fusion operation, represents the action chosen by the system at time step t.

[0053] 2.3. Adaptive weight calculation: In order to dynamically balance the contributions of the language model and the graph model, an adaptive weight mechanism is introduced: ,in Represents the sigmoid function; The final action selection formula is: .

[0054] 3. Course Reinforcement Learning Module (CRLM), The CRLM module introduces the concept of curriculum learning, enabling the system to gradually master spatial reasoning tasks from simple to complex. Specifically, it includes: 3.1. Graph complexity calculation: First, define a graph complexity metric: ; in, is the number of nodes, is the number of edges, is the diameter of the graph; 3.2. Dynamic course generation: Based on graph complexity , using a large language model to dynamically generate a series of subtasks of increasing difficulty; 3.3. Hierarchical reinforcement learning: For each generated subtask, the Proximal Policy Optimization (PPO) algorithm is applied for training; the optimization objective function is: ; in, is the policy entropy, used to encourage exploration; is the discount factor, It’s an instant reward. is the entropy regularization coefficient.

[0055] 4. Metacognitive Monitoring Module (MCM), The MCM module adds a layer of "self-awareness" to the ALGNN-SR framework, enabling the system to evaluate the quality of its own decisions in real time and make adjustments when necessary. It includes: 4.1. Uncertainty estimation: The uncertainty estimation function is defined as: ; in, is the action probability distribution; 4.2. Reasoning Path Adjustment: When Uncertainty Exceeding the preset threshold , triggering a replanning process performed by the large language model; 4.3. Experience accumulation: Continuously record high-quality decision sequences into an experience pool 𝐷; these high-quality experiences will be used to regularly update the strategy network and value network.

[0056] 5. Training and optimization module, End-to-end training is adopted, combining supervised learning and reinforcement learning to achieve optimization; the total loss function is: ; in: is the supervised learning loss, using cross entropy to measure the difference between the predicted output and the true label; is the reinforcement learning loss, that is, maximizing the objective function mentioned above ; is the metacognitive monitoring loss, which uses mean squared error to optimize the accuracy of uncertainty estimation; and is the coefficient that balances the different loss terms.

[0057] The specific implementation process of the system is as follows: 1. Initialization phase: The system first loads the pre-trained large language model (such as GPT-4 or Qwen2-72B) and graph neural network model; at the same time, the system initializes the experience pool 𝐷 and the course task queue to prepare for continuous learning and course learning.

[0058] 2. Task reception: The system receives a spatial reasoning task, such as "find the shortest path from point A to point B in a given room layout"; this task is usually given in natural language and contains rich spatial information and target description.

[0059] 3. Spatial-Relational-Graph Transformation (SRG-TM module): This key step converts the task described in natural language into a graph structure that can be processed by computers. The specific steps are as follows: Use a large language model to parse the task description and extract key entities and relations; Constructing the initial relationship diagram ,in is a collection of entities, is a set of relations; Generating graph-level embeddings using graph neural networks and node-level embeddings , this process can be expressed as: ; The importance of this step lies in converting complex spatial information into structured data representation.

[0060] Curriculum Generation (CRLM Module): The CRLM module designs a progressive learning strategy that enables the system to start with simple tasks and gradually master more complex spatial reasoning capabilities: First, calculate the graph complexity ; based on , using a large language model to generate a series of subtasks of increasing difficulty.

[0061] This curriculum learning method significantly improves the learning efficiency and generalization ability of the system.

[0062] Iterate each subtask: This is the core loop of the system to perform spatial reasoning, which includes the following steps: a. State initialization: set the initial state , including graph embedding, current node embedding and task context.

[0063] b. For each time step Do the following: Adaptive Reasoning (ARM module): The ARM module is the decision-making core of the system. It combines a large language model and a graph neural network to achieve adaptive reasoning: Calculate the LLM prediction: , Compute GNN predictions: , Calculate adaptive weights: , Generate Action: ; Metacognitive monitoring (MCM module), the MCM module provides the system with the ability of "self-reflection" to ensure the reliability of decision-making: Calculate uncertainty: , if , triggering re-planning, Otherwise, perform the selected action ; Environment Interaction: Execute an action , get a new state and rewards , Will Deposit into experience pool 𝐷; Determine whether the subtask goal or maximum number of steps has been reached: If yes, end the current subtask. If no, , continue to the next time step; c. Training and optimization: The system improves performance through continuous learning and optimization: Sample batch data from the experience pool 𝐷, Calculate the loss: , Back propagation, updating model parameters.

[0064] An embodiment of the present invention further provides a device for enabling a large model to have spatial reasoning and long-term planning capabilities, characterized in that it includes: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is used to call the machine-readable program to implement the method described in the above embodiment to enable the large model to have spatial reasoning and long-term planning capabilities.

[0065] The embodiment of the present invention further provides a computer-readable medium, on which computer instructions are stored, and when the computer instructions are executed by a processor, the processor executes the method described in the above embodiment for enabling a large model to have spatial reasoning and long-term planning capabilities. Specifically, a system or device equipped with a storage medium can be provided, on which a software program code for implementing the functions of any of the above embodiments is stored, and a computer (or CPU or MPU) of the system or device is enabled to read and execute the program code stored in the storage medium.

[0066] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0067] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer via a communication network.

[0068] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0069] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or written to a memory provided in an expansion unit connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or the expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0070] The present invention is shown and described in detail above through the accompanying drawings and preferred embodiments. However, the present invention is not limited to these disclosed embodiments. Based on the above multiple embodiments, those skilled in the art can know that the code review methods in the above different embodiments can be combined to obtain more embodiments of the present invention, and these embodiments are also within the protection scope of the present invention.

Claims

1. A method for enabling large models to have spatial reasoning and long-term planning capabilities, characterized in that: The implementation of this method includes: Space-relation-graph conversion: converting complex spatial descriptions into graph structures that can be efficiently processed by computers; Adaptive reasoning: Combining the language understanding ability of large language models and the structural perception ability of graph neural networks to realize the reasoning mechanism; Curriculum reinforcement learning, through curriculum reinforcement learning, using progressive learning strategies, enables the system to gradually master spatial reasoning tasks from simple to complex; Metacognitive monitoring, which enables the system to assess the quality of its own decisions in real time and make adjustments if necessary; Training and optimization, adopts an end-to-end training method, combining supervised learning and reinforcement learning to achieve optimization.

2. A method for enabling a large model to have spatial reasoning and long-term planning capabilities according to claim 1, characterized in that: The space-relationship-graph conversion specifically includes: Spatial description analysis: Use large language models to understand and parse spatial descriptions in natural language and accurately capture complex spatial relationship descriptions; Relationship diagram construction: The parsed structured information is converted into a relationship graph ,in is a collection of entities, is a set of relationships; in this graph, each entity becomes a node , and the spatial relationship between them is represented by the edges connecting these nodes ; Graph Embedding: Use graph neural networks to encode the constructed relationship graph; generate graph-level embeddings that represent the entire scene and node-level embedding ; The process of graph embedding is expressed by the following mathematical formula: ； Among them, GNN represents graph neural network operation, is the input relationship diagram, is a graph-level embedding, is the embedding set of all nodes.

3. A method for enabling a large model to have spatial reasoning and long-term planning capabilities according to claim 2, characterized in that: The adaptive reasoning is specifically implemented as follows: Status indicates: Construct a comprehensive state representation: ,in, is the global scene information, is the information of the current focused entity, and context is the contextual information related to the task; Action generation: Action generation based on large language model and graph neural network: ,in represents the weighted fusion operation, represents the action chosen by the system at time step t; Adaptive weight calculation: Introducing adaptive weight mechanism: ,in Represents the sigmoid function; The final action selection formula is: 。 4. A method for enabling a large model to have spatial reasoning and long-term planning capabilities according to claim 3, characterized in that: The course intensive learning specifically includes: Graph complexity calculation: First, define a graph complexity metric: ； in, is the number of nodes, is the number of edges, is the diameter of the graph; Dynamic course generation: Based on graph complexity , using a large language model to dynamically generate a series of subtasks of increasing difficulty; Hierarchical Reinforcement Learning: For each generated subtask, the Proximal Policy Optimization algorithm is applied for training; the optimization objective function is: ； in, is the policy entropy, used to encourage exploration; is the discount factor, It’s an instant reward. is the entropy regularization coefficient.

5. The method of enabling a large model to have spatial reasoning and long-term planning capabilities according to claim 1, characterized in that: The metacognitive monitoring specifically includes: Uncertainty estimation: Define the uncertainty estimation function as: ； in, is the action probability distribution; Reasoning Path Adjustment: When Uncertainty Exceeding the preset threshold , triggering a replanning process performed by the large language model; Experience accumulation: Continuously record high-quality decision sequences into an experience pool; these high-quality experiences will be used to regularly update the policy network and value network.

6. A method for enabling a large model to have spatial reasoning and long-term planning capabilities according to claim 1, characterized in that: The total loss function of the training and optimization is: ； in, is the supervised learning loss, using cross entropy to measure the difference between the predicted output and the true label; is the reinforcement learning loss, i.e., maximizing the objective function; is the metacognitive monitoring loss, which uses mean squared error to optimize the accuracy of uncertainty estimation; and is the coefficient that balances the different loss terms.

7. A method for enabling a large model to have spatial reasoning and long-term planning capabilities according to claim 1, characterized in that: The specific implementation process of this method is as follows: Initialization phase: First, load the pre-trained large language model and graph neural network model; at the same time, initialize the experience pool 𝐷 and course task queue to prepare for continuous learning and course learning; Task reception: Receive a spatial reasoning task, which contains rich spatial information and target description; Spatial-relational-graph conversion: Convert the task described in natural language into a graph structure that can be processed by computers. The specific steps are as follows: Use a large language model to parse the task description and extract key entities and relations; Constructing the initial relationship diagram ,in is a collection of entities, is a set of relations; Generating graph-level embeddings using graph neural networks and node-level embeddings , this process is expressed as: ; Curriculum Generation: Design a progressive learning strategy that enables the system to start with simple tasks and gradually master more complex spatial reasoning capabilities: First, calculate the graph complexity ; based on , using a large language model to generate a series of subtasks of increasing difficulty; Iterating over each subtask involves the following steps: a. State initialization: set the initial state , including graph embedding, current node embedding and task context; b. For each time step Do the following: Adaptive reasoning: Combining large language models and graph neural networks to achieve adaptive reasoning: Calculate the LLM prediction: , Compute GNN predictions: , Calculate adaptive weights: , Generate Action: ; Metacognitive Monitoring: Calculate uncertainty: , if , triggering re-planning, Otherwise, perform the selected action ; Environment Interaction: Execute an action , get a new state and rewards , Will Deposit into experience pool 𝐷; Determine whether the subtask goal or maximum number of steps has been reached: If yes, end the current subtask. If no, , continue to the next time step; c. Training and optimization: Improve performance through continuous learning and optimization: Sample batch data from the experience pool 𝐷, Calculate the loss: , Back propagation, updating model parameters.

8. A system that enables large models to have spatial reasoning and long-term planning capabilities, characterized in that: include: The space-relationship-graph conversion module is used to convert complex spatial descriptions into graph structures that can be efficiently processed by computers; Adaptive reasoning module, which is used to combine the language understanding ability of large language models and the structural perception ability of graph neural networks to realize the reasoning mechanism; The curriculum reinforcement learning module is used to enable the system to gradually master spatial reasoning tasks from simple to complex through curriculum reinforcement learning; A metacognitive monitoring module that enables the system to assess the quality of its own decisions in real time and make adjustments when necessary; The training and optimization module uses an end-to-end training method and combines supervised learning and reinforcement learning to achieve optimization; The system realizes large model operation based on the method of enabling the large model to have spatial reasoning and long-term planning capabilities as described in any one of claims 1 to 7.

9. A device that enables large models to have spatial reasoning and long-term planning capabilities, characterized in that: include: at least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is used to call the machine-readable program to implement the method described in any one of claims 1 to 7.

10. A computer-readable medium, characterized in that The computer readable medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images