Large model-based multi-level ownership cognition system

Abstract

The invention particularly relates to a multi-level self-cognition system based on a large model, and relates to the technical field of large models. A neural symbol world model module; a large language model cognition core module; and a hierarchical decision planning system module. According to the method, deep integration of perception, cognition and decision making is achieved through the hierarchical fusion architecture, and compared with the prior art, the method has remarkable advantages; the multi-modal perception encoder adopts layered encoding and a cross-modal attention mechanism, so that the semantic alignment problem of multi-source perception data is effectively solved, and the understanding ability of the system to a complex scene is greatly improved; according to the neural symbol world model, the neural network and symbol reasoning are combined, the limitation of a pure neural network method in physical modeling is overcome, meanwhile, the calculation complexity of a pure symbol system is avoided, and efficient and accurate environment characterization and prediction are achieved.

Description

Technical Field

[0001] This invention relates to the field of large model technology, and in particular to a multi-level embodied cognition system based on large models. Background Technology

[0002] Existing embodied question answering systems are mainly built on visual language models (VLMs), which learn the correspondence between vision and language by pre-training on large-scale image-text pair data.

[0003] Representative works include OpenAI's CLIP, which achieves cross-modal matching of images and text through contrastive learning; BLIP, which improves visual-language pre-training through a bootstrapping mechanism; and LLaVA, which combines large language models with visual encoders to achieve more complex visual understanding and reasoning capabilities. Existing embodied intelligent robot systems mainly adopt a modular architecture, including perception, planning, and execution modules, each operating relatively independently. Decision-making systems often use predefined rules and state machines to handle environmental interactions, such as the behavior tree framework in ROS, or reinforcement learning methods to train robot policies, such as Google's RT-1 and RT-2 systems. Existing neurosymbolic systems typically combine the learning capabilities of neural networks with the reasoning capabilities of symbolic systems. For example, NS-VQA parses the scene into symbolic representations and then uses a program execution engine to answer questions; DeepProbLog and NeurosymbolicAI combine neural networks with logical reasoning systems; and the CLEVR-Hans system improves the interpretability and robustness of the model by combining neural networks and symbolic reasoning.

[0004] Existing embodied systems suffer from a perception-cognition separation problem, treating visual processing and language understanding as independent modules, leading to the loss of semantic information during information flow. Visual features lack semantic understanding, making it difficult for language models to directly process raw visual information. The system cannot understand implicit relationships and contextual information within visual scenes, resulting in limited cross-modal reasoning capabilities. Another key issue is insufficient reasoning depth; existing systems perform poorly when handling complex problems requiring multi-step reasoning, causal analysis, and counterfactual thinking. End-to-end trained models tend to learn simple statistical associations rather than deep causal relationships, lacking explicit decomposition of reasoning steps and intermediate result verification mechanisms. This makes them ill-suited for scenarios requiring an understanding of physical laws and unable to effectively explain issues involving causality. Furthermore, existing systems have limitations in environmental representation, typically presenting static and fragmented representations that lack the ability to model dynamic environmental changes. Most visual-language models are trained on static images, lacking temporal understanding and explicit structure, making it difficult to predict environmental changes, effectively track changes in object states within a scene, and build long-term memory models of the environment. At the same time, it lacks self-correction ability, effective error detection and self-correction mechanisms, and the ability to monitor and evaluate its own cognitive processes. The reasoning process is usually unidirectional, the uncertainty quantification method is imperfect, it is difficult to identify potential errors, and it is unable to effectively resolve conflicts when faced with contradictory information.

[0005] The multi-level embodied cognition system based on a large model proposed in this invention effectively solves the above-mentioned problems and difficulties by deeply integrating perception and cognition, realizing multi-level reasoning, introducing an active exploration mechanism, enhancing self-correction capabilities, and improving knowledge transfer efficiency, thus providing an innovative technical path for the field of embodied artificial intelligence. Summary of the Invention

[0006] The purpose of this invention is to propose a multi-level embodied cognition system based on a large model in order to solve the above-mentioned problems.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: A multi-level embodied cognition system based on a large model, comprising: The multimodal perception module is configured to process multi-source data from UAV sensors in the AirSim environment and output unified multimodal features containing visual, spatial, and temporal information. The Neural Symbolic World Model module is configured to overcome the limitations of pure neural network physical modeling, enabling dynamic tracking of environmental states, prediction of object interactions, and causal relationship analysis, providing structured environmental information for cognition and decision-making. The core cognitive module of the large language model is configured to bridge the knowledge gap between the language space and the physical world, enabling flexible reasoning from rapid intuition to in-depth analysis, and outputting structured cognitive results with confidence. The hierarchical decision planning system module is configured to address the consistency and adaptability issues in decision-making under complex environments. Through goal decomposition, plan generation, and execution optimization, it enables the implementation of abstract goals into concrete actions.

[0008] Preferably, the multimodal sensing module specifically includes: Visual encoding submodule: processes RGB image data; Spatial coding submodule: Constructs scene graph representation using depth map and LiDAR data; Temporal coding submodule: Maintains the visual feature sequence of the most recent 10 frames at a frame rate of 10fps, forming a 1-second temporal window; By fusing three features through a cross-modal attention mechanism, visual, spatial, and temporal features are first mapped to a common representation space through a linear projection layer; then, a gated cross-attention mechanism is applied to calculate the attention weights between different modal features. The fused features are further integrated through residual connections and layer normalization, ultimately outputting a unified multimodal representation vector.

[0009] Preferably, the neural symbolic world model module specifically includes: In the AirSim environment, the neural symbolic world model maintains a structured representation of the urban environment; Construct a scene graph based on the output of the multimodal perception encoder. ; Node set Represents objects in a scene; edge set Indicates the relationship between objects; The scene graph construction process includes four steps: object detection and localization, relation reasoning, graph structure optimization, and temporal consistency maintenance. Add physical properties to each node in the scene graph; The analytical physics model is based on Newtonian mechanics, enabling rigid body dynamics, collision detection, and contact force calculation. The final state prediction is achieved by adaptively weighting and fusing the two prediction results; Construct a structural causal model to support three-level causal reasoning; Maintaining the cause-effect graph ,node Represents variables, edges It indicates a cause-and-effect relationship.

[0010] Preferably, the large language model cognitive core module specifically includes: Convert input information from different sources into a format that the language model can process; A memory compression algorithm is applied to compress long-term interaction history into a summary representation; Integrate multi-source inputs into a structured contextual representation; The context optimization algorithm optimizes the context content through three steps: relevance scoring, information compression, and structural reconstruction. The relevance scoring uses cosine similarity calculation. The final structured prompt consists of four parts: scene description, task description, reasoning guidance, and output format specification. Knowledge retrieval employs a hybrid retrieval strategy, combining sparse and dense retrieval. The search results are optimized using relevance ranking and deduplication algorithms, and the top K results are ultimately selected. Knowledge injection employs three strategies: immediate injection, which dynamically retrieves relevant knowledge during the reasoning process; pre-injection, which injects key knowledge in advance during the context building stage; and adaptive injection, which dynamically adjusts the injection strategy based on the difficulty of reasoning, with difficulty assessment based on problem type, number of entities, and relational complexity. The results of the fusion processing are used to generate the final cognitive reasoning result.

[0011] Preferably, the hierarchical decision-making and planning system module specifically includes: Receive and integrate three inputs to generate a high-level program; Receive the results of cognitive reasoning, extract the core objectives and constraints, and apply an objective decomposition algorithm to decompose the complex objective into a sub-objective network; The goal decomposition employs a hierarchical task network approach, recursively breaking down the top-level goal into sub-goals until a directly executable atomic goal is reached. The dependencies between sub-goals are represented using a directed acyclic graph. Goal parsing is used to transform task objectives into a formal representation, and a goal prioritization algorithm is applied to determine the priority order of sub-objectives; The outputs of the strategic planning layer are prioritized and distributed to parallel planning; the information value assessment algorithm calculates information entropy, information gain, and resource cost to comprehensively evaluate the information value. Sub-objective sequence planning applies a task planning algorithm to generate a sub-objective execution sequence; The planning process finds the initial state through a forward search. To the target state The least cost path; The information acquisition plan identifies key information gaps and designs information acquisition strategies. The system sorts information based on the size of the gaps and designs acquisition strategies, including proactive exploration and querying; Generate a preliminary execution plan and the optimal path.

[0012] Preferably, the method further includes: The system defines five basic actions: movement, rotation, hovering, shooting, and interaction; Action sequence generation employs model predictive control methods; The Monte Carlo simulation generates 100 possible scenario variations to identify high-risk events. For each abnormal situation, the system is designed with a three-level response strategy: primary strategy, backup strategy, and emergency strategy; Fine-tune the execution plan and output the final execution action; The security verification check ensures that the execution plan complies with security constraints, including physical constraints, environmental constraints, and task constraints. The verification process employs a formal method, representing constraints as logical predicates to verify whether the execution plan satisfies all constraints; for the parts that do not satisfy the constraints, the system automatically adjusts the parameters or replans. Adaptively adjust design feedback checkpoints to monitor execution progress.

[0013] Preferably, the metacognitive monitoring and correction mechanism module is configured to improve system reliability and robustness, correct errors and optimize models through closed-loop feedback, and provide assurance for secure deployment, specifically including: Integrate reasoning processes, decision-making plans, and environmental feedback information to conduct uncertainty analysis; The confidence estimation implements an integrated confidence assessment method, combining the model's internal confidence and external validation results; a calibration algorithm is applied to ensure the accuracy of the confidence assessment, and the calibration method is based on temperature scaling. Contradiction detection enables logical consistency checks, identifying logical contradictions in the reasoning process; Knowledge gap identification enables knowledge coverage analysis, identifying blind spots in the reasoning process; For each reasoning step, check whether the required knowledge points are in the coverage graph and calculate the coverage rate.

[0014] Preferably, the method further includes: A coordinated verification mechanism is established to verify the reliability of the inference results; Hypothesis generation implements diverse hypothesis generation algorithms to construct verifiable alternatives; based on the original reasoning results, multiple alternative hypotheses are generated by transforming key hypotheses. Transformation operations include condition substitution, reasoning path alteration, and conclusion strength adjustment; The system generates alternative hypotheses and determines the verification order by prioritizing the hypotheses. Priority calculation is based on the verifiability of the hypothesis, the degree of difference from the original hypothesis, and the potential impact. Validation experiment design involves designing validation experiments to verify key hypotheses. For physical hypotheses, design sensory experiments; for reasoning hypotheses, design predictive experiments; for decision-making hypotheses, design comparative experiments. The experimental design was validated to consider resource efficiency, and the validation cost was minimized through algorithm optimization. Counterfactual reasoning enables the construction of counterfactual scenarios and explores the boundaries of decision-making. Multiple counterfactual scenarios are generated to evaluate the performance of decisions under different conditions; sensitivity analysis assesses the impact of perturbation on key factors.

[0015] Preferably, the method further includes: Integrate the validation results to generate metacognitive-corrected output; The results integration algorithm summarizes the verification results and evaluates the reliability of the original decision; the system calculates the verification score for each hypothesis. A weighted fusion method is applied to integrate the original decision and validation results; Provide a transparent explanation of the generative correction process; Uncertainty propagation clearly marks residual uncertainty; the system calculates the confidence interval for correction decisions; and the uncertainty quantification module adjusts its parameters through a model update mechanism.

[0016] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are: 1. This invention achieves deep integration of perception, cognition and decision-making through a layered fusion architecture, which has significant advantages over existing technologies; the multimodal perception encoder adopts layered encoding and cross-modal attention mechanism, which effectively solves the semantic alignment problem of multi-source perception data and greatly improves the system's ability to understand complex scenes; the neural symbolic world model combines neural networks with symbolic reasoning, overcomes the limitations of pure neural network methods in physical modeling, and avoids the computational complexity of pure symbolic systems, thus achieving efficient and accurate environmental representation and prediction.

[0017] 2. This invention, through a large language model cognitive core and a visual-language interaction layer and embodied knowledge base retrieval mechanism, successfully bridges the alignment problem between physical world knowledge and language space. Furthermore, it achieves flexible switching from rapid intuitive judgment to deep analytical reasoning through a parallel reasoning processing unit. The hierarchical decision-making and planning system realizes hierarchical decomposition and integration of decisions. Through top-down goal decomposition and bottom-up execution optimization, it effectively solves the consistency and adaptability problems of planning decisions in complex environments. It breaks through the technical bottlenecks of traditional embodied systems in multimodal fusion, physical understanding, decision-making and planning, and self-correction, providing a novel solution for intelligent interaction in complex environments and laying a solid foundation for next-generation artificial intelligence systems. Attached Figure Description

[0018] Further details, features, and advantages of this application are disclosed in the following description of exemplary embodiments in conjunction with the accompanying drawings, in which: Figure 1 This is a system structure diagram of the present invention; Figure 2This is a schematic diagram illustrating the implementation path of the multimodal sensing encoder technology of the present invention. Figure 3 This is a technical roadmap for the neural symbolic world model of the present invention; Figure 4 This is a diagram of the core architecture of the large language model cognition of the present invention; Figure 5 This is a diagram of the hierarchical decision planning system of the present invention; Figure 6 This is a flowchart of the metacognitive monitoring and correction mechanism of the present invention. Detailed Implementation

[0019] Several embodiments of this application will now be described in more detail with reference to the accompanying drawings to enable those skilled in the art to implement this application. This application may be embodied in many different forms and for various purposes and should not be limited to the embodiments set forth herein. These embodiments are provided to make this application thorough and complete, and to fully convey the scope of this application to those skilled in the art. The embodiments described do not limit this application.

[0020] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It will be further understood that terms such as those defined in commonly used dictionaries shall be interpreted as having a meaning consistent with their meaning in the relevant field and / or the context of this specification, and shall not be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0021] Example 1 Its specific implementation method is combined with the appendix Figure 1 To be continued Figure 6 Please provide a detailed explanation.

[0022] Appendix Figure 1 This invention provides a block diagram of a multi-level embodied cognition system based on a large model, which shows the connection relationship between the multimodal perception module and the hierarchical decision planning system module, and marks the main functional interaction flow of each module.

[0023] In this embodiment, it includes: The implementation was based on the following hardware and software environment: the computing platform was a high-performance workstation equipped with an NVIDIA RTX 4090 GPU (24GB VRAM), an Intel Core i9-13900K processor, and 32GB DDR5 memory; the development framework included Python 3.10, PyTorch 2.1.0, and AirSim 1.8.1; the large language model used was Qwen 2.5-14B-Instruct; the simulation environment was the AirSim Neighborhood environment, which included buildings, roads, vehicles, and pedestrians.

[0024] The AirSim environment provides multimodal sensing data via API, including RGB images (1920×1080 resolution), depth maps (640×480 resolution), LiDAR point clouds (32 lines, 10Hz frequency), and IMU data (100Hz frequency). Communication between the system and AirSim uses a standardized interface, with data transmission latency controlled within 10ms. The communication protocol is based on TCP / IP, ensuring reliable data transmission.

[0025] The multimodal perception module is configured to process multi-source data from UAV sensors in the AirSim environment and output unified multimodal features containing visual, spatial, and temporal information. Specifically, it includes: The visual encoding submodule, based on the CLIPViT-L / 14 architecture, processes RGB image data. Images are first preprocessed, including resizing to 224×224 pixels, normalization, and data augmentation (random cropping, horizontal flipping, and color dithering). The preprocessed image is segmented into 16×16 pixel patches, each mapped to a 768-dimensional embedding space via a linear projection layer. The system adds learnable positional encodings to preserve spatial information, and then processes the data through a Transformer encoder, with each layer containing a 12-head self-attention mechanism and a feedforward neural network.

[0026] The formula for calculating self-attention is: ; in, These represent the query, key, and value matrices, respectively. The feature dimension is set to 64. The output features are semantically enhanced by an Adaptive Feature Enhancement Network (AFE), which includes convolutional layers with residual connections and channel attention mechanisms, ultimately generating a 1024-dimensional visual feature vector.

[0027] The spatial coding submodule constructs a scene graph representation using depth maps and LiDAR data. First, a pre-trained YOLOv8 model is used to identify objects in the scene, with a detection threshold set to 0.65, supporting 80 common object categories. The system combines depth information to calculate the 3D position of objects with centimeter-level accuracy. Then, an initial scene graph is constructed, with objects as nodes and spatial relationships between objects as edges.

[0028] Spatial relationships are calculated as follows: ; in, , Represents the 3D position of an object. Indicates the type of object. Function It achieves the recognition of 15 basic spatial relationships, including above, below, left, right, inside, containment, and contact. The scene graph is processed by a graph convolutional network (GCN), with each GCN layer having a hidden dimension of 256 and the activation function being ReLU, ultimately generating a structured scene representation.

[0029] The temporal coding submodule maintains the visual feature sequence of the most recent 10 frames at a frame rate of 10fps, forming a 1-second temporal window; each frame's features are marked with different time steps using temporal position coding, employing sinusoidal position coding. ; ; in, Indicates time and location. Indicates a dimension index. The model dimension is set to 512. The encoded temporal features are processed through a 4-layer bidirectional LSTM network with a hidden state dimension of 512 and a dropout rate of 0.1 to capture temporal dependencies. ; in, Indicates the features of the current frame. This indicates the hidden state at the previous moment. The system outputs the current state representation and state change prediction, supporting dynamic object trajectory prediction and event detection.

[0030] By fusing three features through a cross-modal attention mechanism, visual, spatial, and temporal features are first mapped to a common representation space (1024 dimensions) through a linear projection layer; then, a gated cross-attention mechanism is applied to calculate the attention weights between features of different modalities. ; in, The attention weights for each modality are represented and normalized using the softmax function: ; Weight vector Learned through end-to-end training; The fused features are further integrated through residual connections and layer normalization, ultimately outputting a unified multimodal representation vector with a dimension of 1024.

[0031] The Neural Symbolic World Model module is configured to overcome the limitations of pure neural network physical modeling, enabling dynamic tracking of environmental states, prediction of object interactions, and causal relationship analysis, providing structured environmental information for cognition and decision-making. Specifically, it includes: In the AirSim environment, the neural symbolic world model maintains a structured representation of the urban environment; Scene Graph Builder: Constructs scene graphs based on the output of a multimodal perceptual encoder. ; Node set Represents objects in the scene, with each node containing the following attributes: unique identifier ID, category. (Select from 30 predefined categories), 3D location 3D size ,speed acceleration ,direction and state (e.g., stationary, moving, or interactive); Edge set Represents the relationship between objects. Each edge contains the following attributes: source node ID, target node ID, relationship type Relation (selected from 15 predefined spatial relationships and 10 functional relationships), and relationship strength (a real number between 0 and 1). The scene graph construction process includes four steps: object detection and localization, relation reasoning, graph structure optimization, and temporal consistency maintenance. Relational reasoning employs a message-passing neural network, performing three rounds of message-passing updates: ; ; ; in Representing node characteristics, Indicates the node position. For edge feature extraction function, Let i represent the set of neighbors of node i; Add physical properties to each node in the scene graph, including mass (kilograms), friction coefficient (a real number between 0 and 1), elasticity coefficient (a real number between 0 and 1), and material type (choose from 8 predefined materials). The physics simulation employs a hybrid approach, combining analytical physics models with learned predictors; The analytical physics model is based on Newtonian mechanics and enables rigid body dynamics, collision detection, and contact force calculation. ; ; in , , These represent position, velocity, and acceleration, respectively. Indicates the time step (set to 0.033 seconds, corresponding to an update frequency of 30Hz); The collision detection method combines hierarchical bounding box (AABB) and precise collision detection, supporting collision calculation for complex shapes; Learning-based predictors employ graph neural network structures to learn physical laws by observing historical data. ; in, This represents the current scene graph. Indicates the current action. The graph neural network is implemented as a 5-layer GraphSAGE with hidden layer dimensions of [256, 256, 128, 128, 64]; The final state prediction is achieved by adaptively weighting and fusing the two prediction results: ; Where α is the adaptive weight, with an initial value of 0.7, which is dynamically adjusted based on historical prediction accuracy, with an adjustment step size of 0.05; Causal reasoning engine: Constructs structural causal models (SCM) and supports three levels of causal reasoning; The system maintains a cause-effect graph. ,node Representing variables (such as object state, environmental conditions, actions, etc.), edges To represent a causal relationship, each edge contains a weight representing the strength of the causal relationship; Level 3 causal reasoning: calculating conditional probability through associative reasoning. Given the current traffic conditions, will pedestrians cross the road? Intervention reasoning assessment of intervention effectiveness How would pedestrians react if the drone lowered its altitude to 5 meters? Counterfactual reasoning evaluates hypothetical scenarios If the northbound route had been chosen instead of the eastbound route, would the destination have been reached faster? The causal reasoning process employs a neurosymbolic approach, combining neural network prediction with symbolic reasoning: ; ; in This represents an intermediate variable. The system uses Monte Carlo sampling to perform integral calculations, with 1000 samples taken to ensure the stability of the inference results.

[0032] The system maintains the environmental state through a hierarchical memory system. The system implements a three-level memory architecture: working memory stores the current scene state and short-term history; contextual memory stores the medium-term event sequence and causal relationships; and semantic memory stores long-term patterns and physical knowledge (persistent storage, including pre-trained physical models and navigation rules).

[0033] The core cognitive module of the large language model is configured to bridge the knowledge gap between the language space and the physical world, enabling flexible reasoning from rapid intuition to in-depth analysis, and outputting structured cognitive results with confidence. Specifically, it includes: Based on the Qwen2.5-14B-Instruct model, the model service API is integrated with the system via local vLLM mounting: Convert input information from different sources into a format that the language model can process; The visual-language interaction layer receives the scene graph representation and converts the graph structure into a structured text description. The conversion process includes three steps: node description generation, relationship description generation, and overall scene summarization. The node description uses the template: "[Category] is located at ([x],[y],[z]), State: [State]"; the relationship description uses the template: "[Source Object][Relationship][Target Object]"; the scene summary generates a high-level description by extracting key information.

[0034] By analyzing user questions using engineering techniques, the system identifies question types (categorized into four types: fact queries, reasoning questions, planning requests, and counterfactual analysis) and extracts key entities and relationships. The question analysis employs a hybrid approach combining rule-based and learning methods, achieving an accuracy rate exceeding 85%. A memory compression algorithm is applied to compress long-term interaction history into a summary representation. The compression algorithm is based on importance sampling and hierarchical clustering, with a compression ratio of 10:1, while retaining key decision points and erroneous experiences. Integrate multi-source inputs into a structured contextual representation; The system constructs a three-layer context structure: the core layer contains the current problem and directly related scenario information; the support layer contains relevant background knowledge and historical information; and the meta-information layer contains task objectives and reasoning strategies. The context optimization algorithm optimizes the context content through three steps: relevance scoring, information compression, and structural reconstruction. The relevance scoring uses cosine similarity calculation. ;in, Represents an information vector. Represents the query vector; The final generated structured hints consist of four parts: scene description (30% of the total length), task description (20%), inference guidance (20%), and output format specification (30%). The total length of the hints is controlled within 8,000 tags to ensure that it does not exceed the model context window limit. Core language model: Based on the Qwen2.5-14B-Instruct model (14B parameters), the model service API is mounted via a local vLLM; the system implements an embodied knowledge base retrieval mechanism (RAG technology), constructing three types of knowledge bases: a physical knowledge base (containing physical laws, material properties, and dynamic models), a common sense reasoning base (containing the functions of everyday objects, typical scene behaviors, and safety rules), and a task-related knowledge base (containing navigation strategies, target recognition methods, and interaction protocols).

[0035] Knowledge retrieval employs a hybrid retrieval strategy, combining sparse retrieval (based on the BM25 algorithm) and dense retrieval (based on vector similarity): ;in, Indicates a query. Represents a document. and These represent the embedding vectors of the query and the document, respectively. This is the weighting coefficient (set to 0.3); The search results are optimized through relevance ranking and deduplication algorithms, and the top K results are finally selected (the value of K is dynamically adjusted according to the query complexity, ranging from 3 to 10). Knowledge injection employs three strategies: immediate injection, which dynamically retrieves relevant knowledge during the reasoning process; pre-injection, which injects key knowledge in advance during the context building stage; and adaptive injection, which dynamically adjusts the injection strategy based on the difficulty of reasoning, with difficulty assessment based on problem type, number of entities, and relational complexity. The multi-path reasoning module implements three reasoning strategies: fast intuitive reasoning directly generates the answer, suitable for simple problems, taking <0.5s; standard analytical reasoning generates the answer through a 5-step reasoning process, suitable for medium-complexity problems, taking <3s; and deep exploration reasoning generates the answer through a 10+ step detailed reasoning process, suitable for high-complexity problems, taking <5s. The module automatically selects the appropriate reasoning strategy based on the complexity of the problem, with complexity assessment based on the number of entities, the number of relationships, and the depth of reasoning. The self-verification mechanism implements three verification methods: consistency check verifies the logical consistency of the reasoning process and detects contradictions and circular arguments; fact verification verifies whether the facts used in the reasoning are accurate by querying the knowledge base and comparing the environmental state; and counterexample generation attempts to build counterexamples to test the robustness of the conclusion by changing the conditions and testing the applicability of the conclusion under different circumstances. The results from the three processing paths are combined to generate the final cognitive reasoning result; a weighted voting mechanism allocates weights based on the confidence level of each reasoning path. ;in, Indicates the first The confidence level of each path is determined by both internal model evaluation and external validation.

[0036] The result synthesis algorithm integrates the advantages of multi-path reasoning, employing paragraph-level selection and merging. The output is formatted to generate structured cognitive reasoning results, including a core conclusion (a concise answer), a summary of the reasoning process (key reasoning steps), a confidence assessment (numerical and qualitative descriptions between 0 and 1), and potential alternative explanations (provided when the confidence level is below 0.8).

[0037] The hierarchical decision planning system module is configured to solve the problems of consistency and adaptability in decision-making under complex environments. Through goal decomposition, plan generation, and execution optimization, it realizes the implementation from abstract goals to concrete actions. Specifically, it includes: Transforming cognitive reasoning results into concrete implementation plans: Receive and integrate three inputs to generate a high-level program; Receive the results of cognitive reasoning, extract the core objectives and constraints, and apply an objective decomposition algorithm to decompose the complex objective into a sub-objective network; The goal decomposition employs a hierarchical task network (HTN) method, recursively decomposing the top-level goal into sub-goals until a directly executable atomic goal is reached; the dependencies between sub-goals are represented by a directed acyclic graph (DAG): ;in, Represents a set of sub-targets. Represents a set of dependencies; Objective resolution is used to transform the task objective into a formal representation, and an objective prioritization algorithm is applied to determine the priority order of sub-objectives; the priority calculation formula is: ;in, Indicate the importance of the goal. Indicates the level of urgency. This represents the execution cost, where α, β, and γ are weighting coefficients (set to 0.5, 0.3, and 0.2 respectively). The outputs of the strategic planning layer are prioritized and distributed to parallel planning; the information value assessment algorithm calculates information entropy, information gain, and resource cost to comprehensively evaluate the information value. ;in, Information gain is used to measure the contribution of information to reducing uncertainty. Information entropy represents the uncertainty of information; This represents the acquisition cost, including time cost, energy cost, and risk cost; Sub-objective sequence planning applies a task planning algorithm to generate a sub-objective execution sequence. The planning algorithm is based on the STRIPS framework, defining an action set A, where each action a∈A includes preconditions. ,Effect and cost ; The planning process finds the initial state through a forward search. To the target state The least cost path; The information acquisition plan identifies key information gaps and designs information acquisition strategies; the system maintains the information status matrix. Each element Indicates the first The first variable The level of understanding of each attribute (a real number between 0 and 1). An information gap is defined as: ; The system sorts information gaps based on their size and designs acquisition strategies, including proactive exploration (such as changing perspectives and approaching the target) and querying (such as retrieving from the knowledge base and confirming with the user). Generate a preliminary execution plan; receive the sub-target sequence, convert it into a state transition diagram, and apply the hybrid A* algorithm to generate the optimal path: ;in, Indicates the distance from the starting point to the node. The actual cost, Indicates from node Estimated cost to the target; heuristic function Using a weighted sum of Euclidean distance and risk factors: ;in, Represents a node coordinates Indicates the target coordinates. Represents a node The risk value, The risk weight is set to 0.5.

[0038] The system defines five basic actions: movement (parameters: target position, speed), rotation (parameters: target orientation, angular velocity), hovering (parameters: duration, altitude), shooting (parameters: focal length, exposure), and interaction (parameters: interaction object, interaction type). Action sequence generation employs the Model Predictive Control (MPC) method: ;in, Indicates the predicted state. Indicates the reference trajectory. Indicates control input, and For the weight matrix ( Set the diagonal elements to [1.0, 1.0, 2.0]. The diagonal elements are set to [0.1, 0.1, 0.2]. The prediction time domain N is set to 20, corresponding to a prediction window of 0.66 seconds. The Monte Carlo simulation generates 100 possible scene variations and identifies high-risk events such as the sudden appearance of obstacles, sensor malfunctions, and communication interruptions. For each abnormal situation, the system is designed with a three-level response strategy: primary strategy (executed under normal circumstances), backup strategy (executed when the primary strategy fails), and emergency strategy (executed under high-risk circumstances). The response strategy is organized into a decision tree structure, where nodes represent states, edges represent triggering conditions, and leaf nodes represent specific actions. The execution plan is fine-tuned to output the final execution action; the execution efficiency optimization analysis of the action sequence identifies optimization opportunities, including action merging (combining consecutive small actions into a single action), parallel execution (executing actions that do not interfere with each other simultaneously), and preloading (preparing resources needed for the next step in advance). The optimization algorithm adopts a greedy strategy, selecting the optimization operation with the greatest efficiency improvement each time, until no further optimization is possible; Safety verification checks whether the execution plan complies with safety constraints, including physical constraints (such as maximum speed and acceleration limits), environmental constraints (such as no-fly zones and minimum obstacle distances), and mission constraints (such as maximum mission time and energy limits). The verification process employs a formal method, representing constraints as logical predicates to verify whether the execution plan satisfies all constraints; for the parts that do not satisfy the constraints, the system automatically adjusts the parameters or replans. Adaptive adjustments are made to design feedback checkpoints to monitor execution progress; the system evaluates the deviation between the current state and the expected state every 100ms during execution. ;in, Indicates the actual state. This indicates the expected state. When the deviation exceeds the threshold (set to 0.5), a dynamic adjustment mechanism is triggered to adjust the plan based on real-time feedback.

[0039] The adjustment strategies include parameter fine-tuning (for small deviations), local replanning (for medium deviations), and global replanning (for large deviations). The system generates execution plans with adaptive branches, sets condition checks at each key point, and selects different execution paths based on the condition results.

[0040] The metacognitive monitoring and correction mechanism module is configured to improve system reliability and robustness by correcting errors and optimizing models through closed-loop feedback, providing assurance for secure deployment. Specifically, it includes: Assess the reliability of the decision and make necessary corrections: Integrate reasoning processes, decision-making plans, and environmental feedback information to conduct uncertainty analysis; Confidence estimation integrates confidence assessment methods, combining internal model confidence and external validation results: ;in, This represents the internal confidence level of the model (calculated from the probability distribution output by the language model). This represents the logical consistency score (calculated by checking the consistency between reasoning steps). The historical performance score (based on the historical performance of similar decisions) is represented by α, β, and γ, which are weighting coefficients (set to 0.4, 0.4, and 0.2, respectively).

[0041] The system employs a calibration algorithm to ensure the accuracy of confidence assessment; the calibration method is based on temperature scaling. ;in This represents the sigmoid function. This represents the temperature parameter (adjusted using the validation set, set to 1.5). Contradiction detection enables logical consistency checks, identifying logical contradictions in the reasoning process; The system represents the reasoning process as a logical predicate network and checks for contradictory predicate pairs. The contradiction detection algorithm is based on the resolution principle and finds contradictions through proof by contradiction. The system also applies a knowledge consistency check to identify conflicts with known facts by comparing the facts used in the reasoning with standard facts in the knowledge base. Knowledge gap identification enables knowledge coverage analysis, identifying blind spots in the reasoning process; System maintenance knowledge coverage diagram Among them, nodes Representing knowledge points, edges This represents the relationships between knowledge points; for each reasoning step, it checks whether the required knowledge point is in the coverage graph and calculates the coverage rate. ;in, This represents the set of knowledge points required for reasoning. It represents the set of available knowledge points; the system also applies incompleteness detection to identify missing links in the reasoning chain by checking whether the premise of each step of reasoning is supported by the conclusions or known facts of the previous steps.

[0042] A coordinated verification mechanism is established to verify the reliability of the inference results; Hypothesis generation implements diverse hypothesis generation algorithms to construct verifiable alternatives; based on the original inference results, multiple alternative hypotheses are generated by transforming key hypotheses: ;in, Indicates the original hypothesis, Indicates the first Transformation operations, Indicates the transformation parameters; Transformation operations include condition substitution (changing preconditions), reasoning path alteration (using different reasoning methods), and conclusion strength adjustment (enhancing or weakening the certainty of the conclusion). The system generates 10 alternative hypotheses and determines the verification order by prioritizing the hypotheses. Priority calculation is based on the verifiability of the hypothesis, the degree of difference from the original hypothesis, and the potential impact. Validation experiment design involves designing validation experiments to verify key hypotheses. For physical assumptions (such as "obstacles can be bypassed"), the system designs perception experiments, such as acquiring images of obstacles from different angles; for reasoning assumptions (such as "pedestrians will continue moving in the current direction"), the system designs prediction experiments, such as short-term tracking of pedestrian trajectories; for decision assumptions (such as "the current path is optimal"), the system designs comparison experiments, such as simulating the execution of multiple candidate paths.

[0043] The experimental design was validated by considering resource efficiency, and the validation cost was minimized through an optimization algorithm. ;in, This represents the i-th experiment. Indicates the cost of the experiment. This represents the assumed coverage rate. This represents the target coverage threshold (set to 0.9). Counterfactual reasoning enables the construction of counterfactual scenarios and the exploration of decision boundaries; the system generates counterfactual scenarios through reverse reasoning. ;in, Representing a real-world scenario, This indicates a modification operation. Indicates the amount of modification; The system generates multiple counterfactual scenarios to evaluate the performance of decisions under different conditions; sensitivity analysis assesses the impact by perturbing key factors. ;in, Indicates key factors, Represents the original scene. This indicates the scene after the disturbance. The decision function is represented. The system generates a robustness report that evaluates the performance of the decision under different conditions, including a robustness score (a real number between 0 and 1), a list of sensitive factors, and a description of failure conditions.

[0044] Integrate the validation results to generate metacognitive-corrected output; The results integration algorithm summarizes the validation results and evaluates the reliability of the original decision; the system calculates the validation score for each hypothesis: ;in, Indicates the verification of the hypothesis The experimental set, Indicates the experimental weights. Indicates the experimental results (1 indicates support, -1 indicates opposition, 0 indicates neutrality); The system applies a weighted fusion method to integrate the original decision and validation results: ; Indicates the original decision. This indicates a decision based on the verification results. This represents the fusion weight (dynamically adjusted based on the confidence level of the original decision, ranging from 0.3 to 0.7). The system generates a transparent explanation of the correction process; it produces three parts: the rationale for correction (explaining why the original decision needs to be corrected), supporting evidence (listing key evidence supporting the correction), and an explanation of uncertainties (indicating the uncertainties that remain after correction). The explanation uses a structured template to ensure clarity and conciseness. The system compares the differences between the original and corrected decisions, highlighting key changes.

[0045] Uncertainty propagation explicitly marks residual uncertainty; the system calculates the confidence interval for the correction decision: ;in, Indicates the confidence level corresponding to Value (z=1.96 at a 95% confidence level). The standard deviation represents the uncertainty. The system provides multiple possible options and their confidence levels, sorted in descending order of confidence level. Decision recommendations include a risk assessment, clearly indicating the potential risks, their severity, and their probability of occurrence. The uncertainty quantification module's parameters are adjusted through a model update mechanism; error analysis collects correction history and identifies systematic error patterns. The system maintains an error database, recording detailed information for each correction, including the original decision, correction decision, correction reason, and actual result. The error analysis algorithm identifies common error patterns through clustering and applies root cause analysis to determine the causes of errors. The system generates error type classifications, including four main categories: perceptual errors, reasoning errors, knowledge errors, and execution errors.

[0046] The parameter update implements the incremental learning algorithm, updating the parameters of the uncertainty quantization module. The update formula is: ;in, Indicates the current parameter. This represents the learning rate (set to 0.01). The loss function is defined as the cross-entropy between the prediction confidence and the actual correctness.

[0047] The system employs regularization techniques to prevent overfitting, specifically L2 regularization: ;in, This represents the regularization coefficient (set to 0.001). The system implements parameter version control, supports rollback operations, saves a parameter snapshot after each update, and records performance metrics.

[0048] The knowledge base update adds newly discovered knowledge to the knowledge base. The system implements three update operations: adding new knowledge (when unrecorded knowledge is discovered), updating erroneous knowledge (when a knowledge entry is found to be inconsistent with reality), and enhancing existing knowledge (adding new examples or details). The knowledge update process includes a verification step to ensure the accuracy and consistency of the new knowledge. The system optimizes the knowledge retrieval mechanism, updates the index structure, and improves retrieval efficiency.

[0049] AirSim's specific application scenarios: urban environment navigation and interaction; In an urban environment, AirSim achieved the following specific tasks: the drone needed to navigate between buildings, identify a specific target (a red car), and plan the optimal path to approach the target while avoiding dynamic obstacles and non-flying areas.

[0050] Environmental settings: The AirSim Neighborhood environment includes various buildings (residential and commercial), road networks, vehicles (stationary and moving), pedestrians, and vegetation. The environment size is 500 x 500 meters, and building heights range from 5 to 30 meters. Weather conditions are set to sunny with good visibility, a wind speed of 2 m / s, and a northeast wind direction. Time is set to 3 PM with moderate lighting conditions.

[0051] Drone configuration: A quadcopter drone model is used, with a maximum flight speed of 10 m / s and a maximum acceleration of 2 m / s. The maximum flight altitude is 120 meters, and the battery capacity is 5000mAh (supporting approximately 30 minutes of flight time). The drone is equipped with an RGB camera (1920×1080 resolution, 90° field of view), a depth camera (640×480 resolution, measurement range 0.1-100 meters), a 32-line LiDAR (10Hz scanning frequency, 100m measurement range), and an IMU sensor (100Hz sampling rate).

[0052] Task Flow: The detailed process of the system executing tasks is as follows: The system receives the user's instruction: "Locate the red car in the city center, approach it to within 5 meters and take a high-resolution photo, avoiding all obstacles and pedestrians, prioritizing safety."

[0053] The multimodal perception coding module processes the drone camera input. The visual coding submodule inputs RGB images into the CLIPViT-L / 14 model to extract visual feature vectors; the spatial coding submodule detects objects and constructs a scene representation by combining depth information; the temporal coding submodule processes the most recent 10 frames of data to capture dynamic changes. The three features are fused through a cross-modal attention mechanism to generate a unified scene representation.

[0054] The neural symbolic world model constructs a structured representation of the urban environment. The scene graph builder generates a scene graph containing buildings, roads, vehicles, and pedestrians. Each node contains location, category, and state information, and edges represent spatial and functional relationships between objects. The physics and dynamics simulator predicts the trajectories of moving objects (such as vehicles and pedestrians), taking into account physical constraints and traffic rules. The causal reasoning engine analyzes causal relationships in the environment, such as "a red light causes a vehicle to stop" or "a pedestrian approaching the road may be preparing to cross the road." The temporal update module maintains the dynamic changes in the environmental state, recording key events and state transitions.

[0055] The core cognitive analysis of the large language model considers the task objective and environmental state. The multi-source input processing module converts the scene graph into a text description: "The current scene contains 12 buildings, 8 vehicles (2 of which are red), 5 pedestrians, and a drone located on the northern edge of the city at a height of 50 meters." The context builder integrates the scene description, task objective, and background knowledge to generate structured prompts. The core language model (Qwen2.5-14B-Instruct) analyzes the prompts and generates the reasoning process: "First, we need to obtain a global perspective, identify the possible locations of the red vehicles, then plan a safe path to approach the target, considering building heights and dynamic obstacles." The parallel reasoning processing unit simultaneously executes three reasoning strategies (fast, standard, and deep) and checks the consistency of the reasoning through a self-verification mechanism. The result integration module merges the three reasoning results to generate the final cognitive result.

[0056] The hierarchical decision-making and planning system generates a three-stage execution plan: Phase 1 (Exploration Phase): Ascend to a height of 80 meters, obtain a global view, scan the city center area, and identify potential red vehicles.

[0057] Phase 2 (Planning Phase): After confirming the location of the red vehicle, plan the optimal approach path, taking into account buildings, dynamic obstacles, and safe distances.

[0058] Phase 3 (Execution Phase): Fly along the planned path, descend to an altitude of 15 meters, approach the target to a distance of 5 meters, adjust the angle, and take high-resolution photos.

[0059] The strategic planning layer breaks down the task into a network of sub-objectives, including "acquiring a global view," "identifying red vehicles," "planning a safe path," "approaching the target," and "taking photos." The information value assessment module calculates the value score of each piece of information and identifies "red vehicle location" and "obstacle distribution" as key information. The tactical execution layer generates detailed paths and action sequences, including atomic actions such as takeoff, cruising, hovering, and taking photos. The behavior optimization module optimizes the execution plan to ensure energy efficiency and safety.

[0060] The metacognitive monitoring mechanism assesses the reliability of the plan and identifies potential risks. The uncertainty quantification module calculates the plan's confidence level to be 0.87 and identifies two key uncertainties: "there may be navigation blind spots between buildings" and "the red vehicle may be moving." The active verification system designs and verifies experiments, including scanning gaps between buildings from different angles and short-term tracking of vehicle movement. The correction decision generator modifies the original plan based on the verification results, adding two sub-tasks: "reducing altitude before the blind spot" and "predicting the vehicle's trajectory." The learning update module records the decision-making process and updates the uncertainty assessment parameters.

[0061] The system executes the revised plan: The drone first ascended to a height of 80 meters to capture a panoramic view of the city.

[0062] The vision system detected three red vehicles in the city center and identified the target vehicles through shape and location analysis.

[0063] The system plans a path from the current location to the target vehicle, avoiding tall buildings and detected dynamic obstacles.

[0064] Before approaching densely built-up areas, the drone lowered its altitude to 30 meters, slowed to 5 m / s, and increased the sensor scanning frequency.

[0065] The system detects that the target vehicle is moving slowly, predicts its trajectory, and dynamically adjusts its approach path.

[0066] The drone successfully approached the target vehicle to a distance of 5 meters, adjusted its angle to obtain the best shooting perspective, and took high-definition photos.

[0067] After completing the mission, the drone returned to its starting position, taking a total of 142 seconds, with 68% battery capacity remaining.

[0068] System performance metrics: In 50 test tasks, the system achieved the following performance metrics: Target recognition accuracy: 94.3% (the percentage of red cars successfully identified); Path planning success rate: 92.7% (the percentage of successfully planned feasible paths); Obstacle avoidance rate: 99.8% (the percentage of obstacles successfully avoided); Task completion time: Average 78 seconds (from receiving instructions to completing the task); Energy efficiency: 35% higher than the baseline system (energy consumption to complete the same task); Exception handling success rate: 87.5% (the percentage of unexpected situations successfully handled); Decision correction frequency: 2.3 times per task on average (number of times the system self-corrects).

[0069] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0070] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

[0071] It should be noted that, in this document, the use of relational terms such as "first" and "second" is merely for distinguishing one entity or operation from another, and does not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0072] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0073] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0074] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0075] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0076] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0077] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0078] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A multi-level embodied cognition system based on large models, characterized in that, Comprise: A multi-modal perception module configured to process multi-source data from a UAV sensor in an AirSim environment, outputting unified multi-modal features containing visual, spatial, and temporal information; A neural-symbolic world model module configured to overcome the limitations of pure neural network physical modeling, enabling environment state dynamic tracking, object interaction prediction, and causal relationship analysis, providing structured environment information for cognition and decision-making; A large language model cognitive core module configured to bridge the knowledge gap between the language space and the physical world, enabling flexible reasoning from quick intuition to deep analysis, and outputting structured cognitive results with confidence; A hierarchical decision-making and planning system module configured to solve the consistency and adaptability of decision-making in complex environments, achieving target decomposition, plan generation, and execution optimization, and realizing the landing from abstract goals to specific actions.

2. The multi-level embodied cognition system based on large models according to claim 1, wherein, The multi-modal perception module specifically comprises: A visual encoding submodule: processing RGB image data; A spatial encoding submodule: constructing a scene graph representation using depth map and lidar data; A temporal encoding submodule: maintaining a sequence of visual features for the last 10 frames, with a frame rate of 10fps, forming a 1-second temporal window; Through a cross-modal attention mechanism, the three features are first mapped to a common representation space through linear projection layers; then the attention weights between different modal features are calculated using a gated cross-attention mechanism; The fused features are further integrated through residual connection and layer normalization, and finally output a unified multi-modal feature vector.

3. The multi-level embodied cognition system based on large models according to claim 1, wherein, The neural-symbolic world model module specifically comprises: In the AirSim environment, the neural-symbolic world model maintains a structured representation of the urban environment; Constructing a scene graph based on multi-modal perception encoder outputs ; a set of nodes representing objects in a scene; a set of edges representing relationships between objects; The scene graph construction process includes four steps: object detection and localization, relationship reasoning, graph structure optimization, and temporal consistency maintenance; Physical attributes are added to each node in the scene graph; The physical model is based on Newtonian mechanics, implementing rigid body dynamics, collision detection, and contact force calculation; The final state prediction is obtained by adaptively weighting the two prediction results; A structural causal model is constructed to support three-level causal reasoning; Maintaining a causal graph , nodes representing variables, edges representing causal relationships.

4. The multi-level embodied cognition system based on large models according to claim 1, wherein, The large language model cognitive core module specifically comprises: Convert different sources of input information into a format that can be processed by the language model; Apply a memory compression algorithm to compress long-term interaction history into a summary representation; Integrate multi-source inputs into structured context representations; The context optimization algorithm optimizes the context content through three steps: correlation scoring, information compression, and structure reorganization, using cosine similarity to calculate correlation scoring: The final generated structured prompt contains four parts: scene description, task explanation, reasoning guidance, and output format specification A hybrid retrieval strategy is used for knowledge retrieval, combining sparse retrieval and dense retrieval; The retrieval results are optimized through relevance ranking and deduplication algorithms, and the top K results are finally selected; Three strategies are used for knowledge injection: immediate injection dynamically retrieves relevant knowledge during reasoning; pre-injection injects key knowledge in the context construction stage; adaptive injection dynamically adjusts the injection strategy based on the difficulty of reasoning, with difficulty assessment based on question type, entity number, and relationship complexity; Fusion processing results, generate final cognitive inference results.

5. The multi-level embodied cognition system based on large models according to claim 1, wherein, Hierarchical decision planning system module, specifically includes: Receive and integrate three-way input, generate high-level planning; Receive cognitive inference results, extract core goals and constraints, apply goal decomposition algorithm to decompose complex goals into subgoal network; Goal decomposition uses hierarchical task network method to recursively decompose top-level goals into subgoals until atomic goals that can be directly executed are reached; the dependency relationship between subgoals is represented by a directed acyclic graph: Goal analysis is used to convert task goals into formal representations and apply goal priority algorithms to determine the priority order of subgoals; Prioritize and distribute the output of the strategic planning layer to parallel planning; information value evaluation algorithm calculates information entropy, information gain and resource cost to evaluate the value of information comprehensively: Subgoal sequence planning applies task planning algorithms to generate subgoal execution sequences; The planning process finds a minimum cost path from the initial state to the goal state by forward search. Information acquisition plan identifies key information gaps and designs information acquisition strategies; The system sorts information gaps by size and designs acquisition strategies, including active exploration and query; Generate a preliminary execution plan and generate the optimal path.

6. The multi-level embodied cognition system based on large models according to claim 5, wherein, Also includes: The system defines five basic actions: movement, rotation, hovering, shooting, and interaction; Action sequence generation uses model predictive control methods; Generate 100 possible scenario changes through Monte Carlo simulation to identify high-risk events; For each abnormal situation, the system designs a three-level response strategy: main strategy, backup strategy, and emergency strategy; Fine-tune the execution plan and output the final execution action; Safety verification checks whether the execution plan meets safety constraints, including physical constraints, environmental constraints, and task constraints; The verification process uses formal methods to represent constraints as logical predicates and verify whether the execution plan meets all constraints; for parts that do not meet the constraints, the system automatically adjusts parameters or re-plans; Adaptive adjustment designs feedback checkpoints to monitor execution progress.

7. The multi-level embodied cognition system based on large models according to claim 1, wherein, Also includes: The meta-cognitive monitoring and correction mechanism module is configured to improve system reliability and robustness, correct errors through closed-loop feedback, optimize models, and provide security for deployment, specifically including: Integrate reasoning process, decision plan and environmental feedback information for uncertainty analysis; Confidence estimation implements an integrated confidence evaluation method that combines internal model confidence and external validation results; apply calibration algorithms to ensure the accuracy of confidence evaluation, and the calibration method is based on temperature scaling; Contradiction detection implements logical consistency checks to identify logical contradictions in the reasoning process; Knowledge gap identification implements knowledge coverage analysis to identify knowledge gaps in the reasoning process; For each reasoning step, check whether the required knowledge points are in the coverage graph and calculate the coverage rate.

8. The multi-level embodied cognition system based on large models according to claim 7, wherein, Also includes: Coordination verification mechanism to verify the reliability of the reasoning results; Hypothesis generation implements a diversified hypothesis generation algorithm to construct verifiable alternative solutions; Based on the original inference results, generate multiple alternative hypotheses by transforming key assumptions; Transformation operations include conditional replacement, reasoning path change, and conclusion strength adjustment; The system generates alternative hypotheses and determines the verification order through hypothesis priority ranking, with priority calculation based on the verifiability of the hypothesis, the degree of difference from the original hypothesis, and the potential impact; Verification experiment design aims to design verification experiments for key hypotheses; For physical hypotheses, perceptual experiments are designed; for reasoning hypotheses, prediction experiments are designed; for decision hypotheses, comparison experiments are designed; Verification experiment design considers resource efficiency and minimizes verification cost through optimization algorithms; Counterfactual reasoning realizes counterfactual scenario construction and explores the decision boundary; Multiple counterfactual scenarios are generated to evaluate the performance of the decision under different conditions; Sensitivity analysis assesses the impact of key factors by perturbing them.

9. The multi-level embodied cognition system based on large models according to claim 8, wherein, Also included: Integrate verification results to produce output corrected by metacognition; The result integration algorithm summarizes the verification results to evaluate the reliability of the original decision; The system calculates the verification score of each hypothesis; Apply a weighted fusion method to integrate the original decision and the verification results; Generate a transparent explanation of the correction process; Uncertainty propagation explicitly labels residual uncertainty; the system calculates the confidence interval of the corrected decision; and the uncertainty quantification module is adjusted through a model updating mechanism.

Images