Real-time environment action landing method based on neural-symbolic large language model planning

Abstract

The application discloses a method for planning real-time environment action landing based on a neural symbolic large language model, and relates to the technical field of artificial intelligence. The application comprises the following steps: S1. obtaining a task target to be executed, state information of a real-time interactive environment, and background knowledge related to the task; S2. based on a pre-trained large language model, generating a hierarchical natural language plan according to the task target, the environment state information and the background knowledge; S3. compiling the hierarchical natural language plan into a neural symbolic logic program with hierarchical task decomposition and explicit control flow, and dividing predicates in the program into definitional predicates that can be recursively expanded and atomic predicates that need to be directly mapped into environment actions. The application introduces a neural symbolic logic program as a unified intermediate representation between the natural language plan generated by the large language model and the environment bottom layer action, realizes automatic conversion of high-level planning to low-level action control, and solves the problem that the natural language plan is difficult to be directly landed and executed.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, and in particular relates to a method for implementing real-time environmental actions based on a neural symbolic large language model. Background Technology

[0002] With the development of artificial intelligence technology, large language models, due to their strong capabilities in knowledge understanding, semantic reasoning, task decomposition, and text generation, have been gradually applied to scenarios such as agent decision-making, task planning, and open environment interaction. In complex tasks, large language models can generate relatively complete high-level plans based on target descriptions and background knowledge, providing agents with task execution order, sub-task relationships, and policy hints, thus demonstrating good planning potential.

[0003] However, existing large language models mostly generate plans in the form of natural language, code snippets, or informal sequence of steps. Their outputs typically remain at a high-level semantic level, making it difficult to directly map to the low-level action control processes in real-time interactive environments. Especially in environments that only provide raw perceptual input and limited action interfaces, agents need to respond in real time based on continuously changing environmental states, and there is a significant representational gap between high-level language planning and low-level action control. Due to the lack of a unified formal intermediate representation and an executable implementation mechanism, existing solutions often fail to guarantee the stability, correctness, and real-time performance of task execution.

[0004] On the other hand, reinforcement learning can directly learn the mapping relationship from perceived state to action output through interaction with the environment, making it suitable for real-time decision-making and low-level control tasks. However, traditional reinforcement learning usually relies on manually designed reward functions and extensive environmental exploration, limiting its ability to plan long-term tasks, decompose complex tasks, and utilize prior knowledge. In open environments, with sparse rewards and partially observable conditions, relying solely on reinforcement learning often results in high training costs, convergence difficulties, and insufficient generalization ability, making it difficult to independently complete tasks with complex logical dependencies.

[0005] To address these shortcomings, some existing methods attempt to combine large language models with controllers, skill libraries, or pre-trained strategies. The large language model handles high-level decision-making, then calls predefined action modules to complete task execution. While these methods improve task completion efficiency to some extent, they typically rely on manually constructed skill sets, fixed control interfaces, or frequent online calls to the large language model. On the one hand, this limits the system's ability to automatically generate new atomic skills in new task scenarios; on the other hand, online calls to the large language model introduce high latency, making it difficult to meet the continuous interaction requirements of real-time environments. Furthermore, existing solutions generally lack explicit control flow expressions for task planning, making it difficult to support complex execution mechanisms such as branching decisions, loop execution, failure backtracking, and skill combination reuse.

[0006] Therefore, existing technologies still lack a technical solution capable of automatically converting high-level natural language planning generated by large language models into a formalized, executable, verifiable, and scalable action control process for real-time interactive environments. Especially in the absence of a complete predefined skill library, how to construct a unified method that integrates task planning, skill generation, policy learning, and execution verification capabilities remains a pressing technical problem to be solved in this field.

[0007] The purpose of this invention is to address the problems that existing high-level planning generated by large language models cannot be directly mapped to low-level actions in real-time interactive environments, the lack of long-term planning capabilities in traditional reinforcement learning, and the insufficient real-time performance and generalization of existing combination methods due to their reliance on predefined skills or online invocation of large language models. This invention proposes a real-time environment planning method based on large language models and neural symbolic predicate invocation.

[0008] This invention improves the reliability, real-time performance, and generalization ability of task execution in complex real-time environments by compiling natural language programming into a neural symbolic logic program with hierarchical decomposition relationships and explicit control flow, and by rendering the atomic predicates in the program into executable neural predicates. It combines inverse learning and reinforcement learning to achieve the mapping from raw perceptual input to environmental action output. Summary of the Invention

[0009] This invention provides a method for planning and implementing real-time environmental actions based on a neural symbolic large language model, comprising the following steps: S1. Obtain the task objective to be executed, the status information of the real-time interactive environment, and the background knowledge related to the task; The environmental state information includes environmental observation information, action interface information, and execution feedback information. The background knowledge includes task-related common sense, environmental rule knowledge, object dependency relationships, and operation rules, which are used to support subsequent planning generation, logic program construction, and execution result verification.

[0010] S2. Based on the pre-trained large language model, a hierarchical natural language plan is generated according to the task objective, environmental state information and background knowledge. The natural language plan includes the subtask dependencies and execution order information required to complete the task, which is used to represent the high-level solution path of the target task.

[0011] S3. Compile the hierarchical natural language program into a neural symbolic logic program with hierarchical task decomposition relationship and explicit control flow structure. At the same time, divide the predicates in the neural symbolic logic program into two categories: definition predicates that can be further expanded by logical rules, and atomic predicates that cannot be further decomposed by symbolic rules and need to be directly mapped to environmental executable actions. The neural symbolic logic program is used to transform high-level planning in natural language form into a task structure that can be formalized, reasoned, and scheduled. Its explicit control flow structure supports task decomposition, branch selection, loop execution, and failure backtracking.

[0012] S4. Perform a predicate library matching verification on the atomic predicate. If there is a trained skill in the predicate library that corresponds to the atomic predicate, then directly bind the trained skill to the atomic predicate as the execution implementation. If there is no corresponding trained skill in the predicate library, then the atomic predicate is determined as the neural predicate to be generated.

[0013] S5. For the neural predicate to be generated, based on the inverse learning framework, and combining the task objective, background knowledge, the constraint relationship of the neural symbolic logic program and the environmental state change information, construct the training objective, success judgment condition and reward function corresponding to the neural predicate to form a trainable reinforcement learning task.

[0014] S6. Based on the training objective, success criteria, and reward function, construct a corresponding reinforcement learning task. With the original environmental perception state as input and the environmental action interface as output, obtain the low-level action policy corresponding to the neural predicate through reinforcement learning training, establish a mapping relationship from the original environmental perception state to the action output, and after training, encapsulate the low-level action policy into a neural predicate and store it in the predicate library.

[0015] S7. Using the top-level predicate corresponding to the target task as the query entry point, the task scheduling is executed based on the neural symbolic logic program, the predicate is recursively expanded, and the trained skills or neural predicates corresponding to the atomic predicates are called to execute environmental action control. Simultaneously, the environmental state before and after the execution of the atomic predicate is verified, and subsequent predicate scheduling, execution retry, path backtracking or branch switching are performed based on the verification results until the target task is completed.

[0016] The present invention is further configured such that the neural symbolic logic program consists of multiple logic rules, which are constructed based on chained meta-rules, and the expressions of the chained meta-rules are: ; in, , , These are the predicates corresponding to the target task or subtask, respectively. , , These are the state variables in the state space, corresponding to the start state, end state, and intermediate state of the task execution, respectively. The predicates in the logical rules all adopt the form of binary predicates with state input and state output, which are used to characterize the completion state of the corresponding subtask when the agent transitions from the input state to the output state.

[0017] The present invention is further configured such that the defined predicate is a high-level task predicate that appears as the head of at least one logical rule in a neural symbolic logic program and can be decomposed through recursive expansion of logical rules; The atomic predicates are skill predicates that have no further symbolic definition in the current neural symbolic logic program and need to be directly mapped to executable actions at the underlying environment level.

[0018] The present invention is further configured such that, in step S5, the process of constructing the training objective, success determination condition and reward function based on inverse learning is specifically as follows: taking the neural predicate to be generated as the inverse term, and combining the task objective, background knowledge and the constraint relationship of the neural symbolic logic program, the target state change condition that the neural predicate must satisfy to execute successfully is determined; Based on the target state change conditions, corresponding success determination conditions are generated, and then a corresponding reward function is generated based on the success determination conditions and environmental state change information.

[0019] The present invention is further configured such that the success determination condition is determined by comparing the changes in the number of target objects, the state of target tools, key attribute markers, or task-related state variables in the environment state before and after the execution of the atomic predicate.

[0020] The present invention is further configured such that the reward function adopts a reward shaping method based on the difference in the amount of target objects held, and the specific process is as follows: execute the current action and obtain the observation state, basic reward value, termination signal and additional information returned by the environment; If the current action does not belong to the preset set of legal actions, then an illegal action penalty is applied to the base reward value; Obtain the current holding amount of the target object from the additional information, compare it with the holding amount of the target object at the previous time step, and if the current holding amount is greater than the holding amount at the previous time step, apply the target achievement reward to the base reward value and set the termination signal to true; The final output includes the observation status, the updated reward value, the termination signal, and additional information.

[0021] The present invention is further configured such that, in step S6, the specific process of reinforcement learning training is as follows: constructing a reinforcement learning task corresponding to the neural predicate to be generated, wherein the reinforcement learning task includes a state space, an action space, an environmental state transition process, a reward function, and a discount factor; During training, the agent continuously interacts with the environment and iteratively updates the policy parameters according to the reward function until the success criteria are met. After training, the low-level action strategies are encapsulated into neural predicates that correspond one-to-one with the atomic predicates and stored in the predicate library to achieve reusable storage.

[0022] The present invention is further configured such that, in step S7, a predicate dynamic selection mechanism guided by a large language model is adopted during the task scheduling process, specifically: determining the set of candidate predicates that can be executed at present based on the neural symbolic logic program and the current environment state; The current environment state, task objectives, historical execution trajectory, and structural constraints of the neural symbolic logic program are input into the large language model. The large language model evaluates the priority of the predicates in the candidate predicate set and selects the predicate to be called first. The large language model selects predicates only from the set of candidate predicates.

[0023] The present invention is further configured such that, in step S7, the specific process of result verification and control flow adjustment is as follows: obtain the first environment state before the execution of the atomic predicate and the second environment state after the execution, and compare the first environment state and the second environment state based on the success determination condition corresponding to the atomic predicate; If the comparison result meets the success criteria, then continue with the subsequent predicate scheduling. If the comparison result does not meet the success criteria, then according to the explicit control flow structure of the neural symbolic logic program, the execution of the atomic predicate is retried, backtracked to the upper-level defined predicate, or switched to other executable branches.

[0024] The present invention is further configured such that the trained skills and encapsulated neural predicates stored in the predicate library can be directly called and combined for reuse in subsequent tasks without repeated training.

[0025] The present invention has the following beneficial effects.

[0026] 1. This invention introduces a neural symbolic logic program as a unified intermediate representation between the natural language planning generated by the large language model and the underlying actions of the environment, thereby realizing the automatic conversion from high-level planning to low-level action control and solving the problem that natural language planning is difficult to implement directly.

[0027] 2. This invention represents tasks as a hierarchical structure composed of definition predicates and atomic predicates, enabling target tasks to be decomposed layer by layer according to logical rules, while being compatible with existing skill invocation and automatic generation of new skills, thereby improving the structured representation capability and execution organization efficiency of complex tasks.

[0028] 3. This invention addresses atomic predicates lacking execution implementation by introducing inverse learning to automatically construct their corresponding training objectives, success criteria, and reward signals. This eliminates the need for manual design of reward functions and combines reinforcement learning to complete low-level policy training. As a result, the system can autonomously generate new executable skills even in the absence of a predefined skill library, thereby improving the autonomy of task implementation and the scalability of scenarios.

[0029] 4. This invention enables the task execution process to have explicit control flow and verifiable results through logical reasoning scheduling, execution result verification, and failure backtracking mechanisms. It can retry, backtrack, or reselect branches when atomic predicates fail, thereby improving the reliability and stability of task execution in complex real-time environments.

[0030] 5. This invention stores the trained neural predicates in a predicate library, enabling the skills to be reused, combined, and transferred without repeated training, thus improving the system's generalization ability and task execution efficiency for unseen tasks.

[0031] 6. This invention deeply integrates the high-level planning capability of large language models, the formal constraint capability of neural symbolic logic programs, and the low-level action learning capability of reinforcement learning, enabling intelligent agents to complete complex tasks in a real-time interactive environment that only provides raw perceptual input and action interfaces. It has excellent real-time performance, scalability, and engineering application value. Attached Figure Description

[0032] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below.

[0033] Figure 1 This is the main flowchart of the method of the present invention.

[0034] Figure 2 Build a flowchart for the corresponding reward function.

[0035] Figure 3 Flowchart of dynamic predicate selection mechanism guided by large language models.

[0036] Figure 4 Flowchart for verifying the execution results of atomic predicates and adjusting the control flow.

[0037] Figure 5 This is a chain-like meta-rule graph for neural symbolic logic programs. Detailed Implementation

[0038] The technical solutions of the present invention will be described below with reference to the accompanying drawings. The described embodiments are only some embodiments of the present invention, and not all embodiments.

[0039] In this embodiment, the present invention is applied to a real-time interactive environment that only provides raw perceptual input and action interfaces, so as to automatically transform high-level natural language planning generated by a large language model into an executable action control process in the environment. The method generally includes processes such as task goal acquisition, natural language planning generation, neural symbolic logic program construction, atomic predicate implementation, neural predicate training, logical reasoning execution, result verification, and control flow adjustment.

[0040] Step S1 Information Acquisition In this embodiment, the target description of the task to be completed is first obtained, and the original perception state information in the current real-time interactive environment is received. At the same time, background knowledge related to the task is introduced as input for subsequent natural language planning and generation, neural symbolic logic program construction, neural predicate training and execution result verification.

[0041] Specifically, the task objective is used to characterize the target task that the agent needs to complete, and the task objective can be input into the system in the form of natural language text. Environmental state information is a low-level perceptual input directly provided by the environment, including environmental observation information, action interface information, and execution feedback information. Among them, environmental observation information is used to characterize the current environmental state, action interface information is used to limit the set of actions that the agent can perform, and execution feedback information is used to reflect the environmental response and state changes after the action is performed.

[0042] Background knowledge is used to assist the system in understanding task semantics, establishing task decomposition relationships, and supporting subsequent execution result verification. Background knowledge may include task-related common sense, environmental rule knowledge, object dependencies, and operational rules. After obtaining the task objective, environmental state information, and background knowledge, these serve as the basic input for generating natural language planning using a large language model in step S2.

[0043] Step S2 Natural Language Planning and Generation In this embodiment, after obtaining the task objective, environmental state information and background knowledge in step S1, a natural language plan oriented towards the target task is generated using a large language model. The natural language plan is used to describe the sub-task sequence, task dependencies and execution order required to complete the target task, and serves as the direct input for the subsequent construction of the neural symbolic logic program.

[0044] Specifically, the task objective, environmental state information, and background knowledge obtained in step S1 are input into the large language model. The large language model, combining its pre-trained knowledge and semantic understanding capabilities, performs a high-level decomposition of the target task to obtain the natural language planning corresponding to the current task. The natural language planning is preferably a description of several sequentially related task steps, used to represent the solution path from the current state to the target state.

[0045] In one implementation, natural language planning includes not only the direct operational steps corresponding to the target task, but also the dependencies between these steps. For example, when the target task is to acquire a certain target resource, the large language model can first generate a subtask to acquire the tool, and then generate a subtask to use the tool to acquire the target resource. When intermediate subtasks can be further decomposed, the large language model generates finer-grained step descriptions. In this way, natural language planning can form hierarchical task representations, providing a semantic foundation for subsequent logical program compilation.

[0046] After step S2 is completed, the system obtains the natural language plan corresponding to the target task, and uses the natural language plan as input for constructing the neural symbolic logic program and predicate classification in step S3.

[0047] Step S3: Construction of the Neural Symbolic Logic Program and Predicate Classification This step is divided into two parts: neural symbolic logic program construction and predicate classification. Step 1: Construction of Neural Symbolic Logic Program In this embodiment, a neural symbolic logic program with hierarchical task decomposition relationships is constructed based on the natural language plan obtained in step S2. The neural symbolic logic program is used to convert the high-level task plan in natural language form into a task structure that can be formalized for reasoning and executed for scheduling, so as to support subsequent atomic predicate recognition, neural predicate generation and task execution.

[0048] Specifically, the neural symbolic logic program is represented by a set of logical rules, each rule describing the dependency relationship between the target task and subtasks. Preferably, the logical rules are constructed based on chained meta-rules, and their form can be expressed as: .in, , and The predicate that represents the task objective or subtask. , and Representing the state space The state variables in the table correspond to the start state, intermediate state, and end state of task execution, respectively. The above rule indicates that if starting from state... to state Subtasks can be completed And from the state to state Subtasks can be completed Then from the state to state The target task can be completed. .

[0049] Based on the aforementioned meta-rules, the task steps and dependencies in natural language planning are instantiated into multiple first-order predicates and corresponding logical rules, thereby obtaining the neural symbolic logic program corresponding to the target task. The predicates in the neural symbolic logic program are preferably represented in a binary form of state input and state output, used to describe whether a task can cause the agent to transition from the input state to the output state that satisfies the target condition.

[0050] Furthermore, in this embodiment, the background knowledge provided by the large language model is assumed to be... The chain-like meta-rule set is The target task is The construction process of a neural symbolic logic program can then be represented as: generating logical hypotheses. ,make and and The combination can deliver the target task In other words, the purpose of this step is to generate a set of structured logical rules that can support the achievement of the target task based on natural language planning.

[0051] In one implementation, when the target task is to acquire a specific resource, the large language model can first generate two upper-level subtasks: "acquiring tools" and "performing collection." Then, "acquiring tools" can be further decomposed into lower-level subtasks such as "collecting materials" and "manufacturing tools." Based on this decomposition, hierarchical logical rules can be constructed, allowing the top-level task to be progressively expanded into multiple intermediate tasks until it is decomposed into lower-level tasks that cannot be further expanded by logical rules. In this way, natural language planning is transformed into a program representation with a clear hierarchical structure.

[0052] Furthermore, since this invention is designed for real-time interactive environments and the execution results of some underlying skills are uncertain, the neural symbolic logic program retains an explicit control flow structure in addition to describing the task decomposition relationship, in order to support sequential execution, conditional branching, loop calls and failure backtracking in subsequent execution stages.

[0053] Step 2: Predicate Classification In this embodiment, after the neural symbolic logic program is constructed, the predicates in the neural symbolic logic program are classified into definition predicates and atomic predicates, which serve as the basis for subsequent atomic predicate implementation and neural predicate training.

[0054] Specifically, let the set of predicates in a neural symbolic logic program be... Then it can be expressed as: .in, This represents the definition of a set of predicates. This represents a set of atomic predicates. A predicate is defined as a predicate that appears as the head of at least one logical rule in a neural symbolic logic program and can be further expanded and decomposed through logical rules. An atomic predicate is a predicate that does not have a further symbolic definition in the current neural symbolic logic program and therefore needs to be directly mapped to an environment-executable action.

[0055] In one implementation, predicates are defined primarily to encode high-level task structures, which can be recursively decomposed into multiple sub-goals through logical reasoning. Atomic predicates can no longer be expanded through symbolic rules, but correspond to specific underlying skills. They need to be implemented in subsequent steps by calling existing skills or generating new neural predicates.

[0056] After step S3 is completed, the system obtains the neural symbolic logic program corresponding to the target task, as well as the divided set of defined predicates and set of atomic predicates. The defined predicates are used for subsequent logical reasoning and the atomic predicates are used for the implementation in step S4.

[0057] Step S4: Atomic predicate landing matching In this embodiment, after obtaining the set of atomic predicates in step S3, the atomic predicates are processed to determine the specific execution implementation of each atomic predicate in the real-time interactive environment. The processing includes two methods: calling existing skills from the predicate library and generating new neural predicates.

[0058] Specifically, for any atomic predicate identified in step S3, the system first checks if there is an existing execution implementation corresponding to that atomic predicate in the predicate library. If there is a corresponding trained skill in the predicate library, the trained skill is directly called as the execution implementation of that atomic predicate. If a corresponding skill does not exist in the predicate library, the atomic predicate is identified as the neural predicate to be generated and trained and constructed in subsequent steps. In this way, the system can both reuse existing skills and automatically supplement a completely new execution interface when a corresponding skill is missing.

[0059] In one implementation, the predicate library stores trained skill modules that can be directly executed in the environment. Each skill module corresponds one-to-one with an atomic predicate, enabling a mapping from the initial perceived state of the environment to action output. When the target task contains atomic predicates consistent with existing skills, the corresponding skills can be directly called from the predicate library, thereby reducing repetitive training and improving task execution efficiency.

[0060] After step S4 is completed, atomic predicates corresponding to existing skills can directly enter the subsequent execution stage, while atomic predicates that are missing execution implementation will enter step S5 to construct the corresponding training objectives and reward signals.

[0061] Step S5: Construction of training objectives driven by back-learning In this embodiment, for the neural predicates identified as to be generated in step S4, their corresponding training objectives, success criteria, and reward signals are constructed based on inverse learning to form the reinforcement learning task corresponding to the neural predicate. The purpose of this step is to transform atomic predicates in the logic program that do not yet have an execution implementation into trainable low-level skill objectives, thereby providing a clear optimization direction for subsequent reinforcement learning training.

[0062] Specifically, atomic predicates lacking execution implementation are treated as terms to be inversely derived. Combined with task objectives, background knowledge, logical program constraints, and state change information before and after execution, a training objective corresponding to the atomic predicate is generated. The training objective describes the state change outcome that the atomic predicate should satisfy when it executes successfully in the environment. The success criterion determines whether the atomic predicate has completed its corresponding sub-objective, and the reward signal guides the reinforcement learning policy to optimize in a direction that satisfies the aforementioned state changes.

[0063] In this embodiment, let a certain neural predicate to be generated be denoted as... Then, a corresponding reinforcement learning task can be constructed for it. Its form can be expressed as: .in, Representing the state space, Represents the action space, This indicates the process of environmental state transition. Indication and neural predicate The corresponding reward function, This represents the discount factor. Reinforcement learning tasks share the state space, action space, and state transition process with the original environment, but construct different reward functions for different neural predicates. .

[0064] Furthermore, in this embodiment, the reward function Instead of a fixed, universal reward, the reward is generated through inverse learning based on the semantic constraints of the atomic predicate in the logical program and the requirements for target state changes. Specifically, the system combines the natural language description of the atomic predicate, background knowledge, and environmentally accessible interfaces to determine the state change features that characterize the success or failure of the atomic predicate, and constructs a reward function accordingly. For example, when an atomic predicate corresponds to a task like "collecting target items," the reward signal can be defined based on the change in the quantity of target items in the agent's internal state. When an atomic predicate corresponds to a "tool manufacturing" task, the reward signal can be defined based on whether the target tool appears in the agent's state. In this way, the reward function can maintain semantic consistency with the atomic predicate.

[0065] Furthermore, in this embodiment, in addition to the reward function, a success determination condition is constructed for the neural predicate. The success determination condition is used to determine whether the current atomic predicate has been completed during the training or execution phase. For example, it can be determined whether the sub-goal corresponding to the atomic predicate has been achieved by comparing changes in the number of target objects, the state of the target tool, key attribute markers, or other task semantics-related state variables before and after execution.

[0066] After step S5 is completed, the reinforcement learning task, reward function and success criteria corresponding to each neural predicate to be generated are output, and then step S6 is entered.

[0067] Step S6: Reinforcement learning training of neural predicates In this embodiment, based on the reinforcement learning task, reward signal and success determination condition constructed in step S5, the low-level policy corresponding to the neural predicate to be generated is trained to establish a mapping relationship from the original perception state of the environment to the output of environmental actions, thereby realizing the executable implementation of atomic predicates in a real-time interactive environment.

[0068] Specifically, the initial perceived state of the environment is used as the input to the reinforcement learning model, and the environmental action interface is used as the model output. The policy parameters are updated according to the reward function constructed in step S5. During training, the agent continuously interacts with the environment, receiving current environmental observations at each time step, outputting action control commands, and obtaining corresponding reward signals based on the state changes after the action is executed. Through continuous iterative optimization, the low-level policy gradually learns behavioral patterns that satisfy the semantic constraints of the corresponding atomic predicates.

[0069] Furthermore, during training, the success criteria defined in step S5 can be used to determine whether the neural predicate has been successfully executed in the current training round. When the success criteria are met, it can be considered that the current neural predicate has completed its corresponding atomic subtask. If the success criteria are not met, the policy parameters are further optimized based on the reward signal. After training, the corresponding low-level policies are encapsulated into neural predicates that correspond one-to-one with the atomic predicates, maintaining the logical predicate structure in form and possessing the ability to execute environmental actions in implementation. These predicates are then added to a predicate library for storage, allowing for direct invocation or combined reuse in subsequent tasks.

[0070] After step S6 is completed, the system obtains the neural predicate corresponding to the atomic predicate to be generated, and uses it as an execution module that can be called in the logical reasoning execution stage in step S7.

[0071] Step S7: Task Execution and Closed-Loop Control This step is divided into three stages: execution of the logic program guided by the large language model, verification of predicate results and adjustment of control flow, and output of task execution results.

[0072] Step 1: Execution of Neural Symbolic Logic Programs Guided by Large Language Models In this embodiment, after obtaining the neural symbolic logic program in step S3 and completing the atomic predicate landing in steps S4 to S6, the top-level predicate corresponding to the target task is used as the query entry point. Under the control of the large language model, the candidate predicates in the neural symbolic logic program are selected, and the corresponding predicates are executed according to the selected calling order, thereby realizing the dynamic scheduling from high-level task target to low-level action control.

[0073] Specifically, under the current environmental state, the system first determines the set of candidate predicates that can be executed based on the neural symbolic logic program and the current state. The candidate predicate set includes defined predicates or atomic predicates that satisfy the preconditions under the current state and can be invoked to advance the target task. Then, combining the current environmental state, the task objective, and the structural constraints in the neural symbolic logic program, the system uses a large language model to evaluate and rank the candidate predicates, selecting the target predicate that should be invoked first. In this way, the large language model is no longer only used for front-end natural language planning but also for dynamically controlling the order of predicate invocation during the execution phase.

[0074] In one implementation, let the current state be... The current set of executable candidate predicates is The system then selects the current predicate from the candidate predicate set. To satisfy:

[0075] in, Represents a neural symbolic logic program. Indicates the current state Below, candidate predicates The comprehensive evaluation value represents the progress of the target task. This comprehensive evaluation value is jointly provided by the large language model and the logical program structure, current state information, and task objective, and is used to characterize the priority of calling this predicate.

[0076] If the selected predicate is a defined predicate, its corresponding logical rules can be expanded to form lower-level sub-predicate call candidates. If it is an atomic predicate, then the existing skill determined in step S4 or the neural predicate trained in step S6 is invoked to perform the corresponding action control in the environment. In this way, the order of predicate invocation in the neural symbolic logic program is not completely fixed, but dynamically scheduled by the large language model in combination with the current environmental context, thereby improving the adaptability of the task execution process to complex environmental changes.

[0077] Furthermore, in this embodiment, the large language model's control over the predicate invocation order remains constrained by the structure of the neural symbolic logic program. That is, the large language model selects only from the candidate predicates allowed by the logic program, without arbitrarily generating execution steps outside the task dependencies of the neural symbolic logic program. In this way, both the contextual decision-making capability of the large language model and the formal constraint capability of the neural symbolic logic program can be preserved simultaneously.

[0078] In one implementation, when multiple candidate predicates can be executed in the current state, the large language model can combine the current environment state, remaining task objectives, historical execution trajectory and logical program structure information to prioritize the predicate that is more conducive to achieving the objective. When a predicate fails to execute or the environment changes, the system retrieves the current set of executable candidate predicates and the large language model determines the predicate to be invoked next. Thus, the system can achieve dynamic predicate scheduling based on the environment context during execution.

[0079] Step 2: Verification of Predicate Results and Adjustment of Control Flow In this embodiment, after each atomic predicate is executed, the execution result of the atomic predicate is verified, and the control flow of the neural symbolic logic program is adjusted according to the verification result to determine whether the current sub-goal has been achieved and whether it is necessary to continue execution, backtrack, retry, or select other branches.

[0080] Specifically, since atomic predicates include both directly invoked existing skills and neural predicates obtained through training, and the execution results of neural predicates have a certain degree of randomness and uncertainty, it is not possible to determine whether their execution was successful solely based on logical reasoning. Therefore, this embodiment includes a verification module to compare the environmental states before and after the atomic predicate execution and determine whether the sub-goal corresponding to the current atomic predicate has been achieved. Let the currently executing atomic predicate be... The environment state before execution is The environment status after execution is The verification process can then be represented as: Its output is or Among them, when the output result is When the current atomic predicate is executed successfully, the corresponding sub-goal has been achieved; When the output result is When this occurs, it indicates that the current atomic predicate has failed to execute and the corresponding sub-goal has not yet been achieved.

[0081] Furthermore, the verification module can compare key state variables before and after execution based on the success criteria corresponding to the atomic predicate. For example, when the atomic predicate corresponds to a resource collection task, it can compare whether the number of target resources increases before and after execution. When an atomic predicate corresponds to a tool manufacturing task, it can be compared whether the target tool appears in the post-execution state. When an atomic predicate reaches its corresponding position or completes an action-based task, the relevant state markers, spatial relationships, or behavioral results before and after execution can be compared to see if preset conditions are met. This method allows for the determination of whether an atomic predicate has completed its corresponding sub-goal.

[0082] When the verification result is successful, the system maintains the execution path of the current logic program unchanged and continues to execute subsequent predicates; When the verification result fails, corresponding adjustments are performed according to the control flow relationships pre-reserved in the neural symbolic logic program. If the current atomic predicate is allowed to be retried, the system can retry that atomic predicate. When an alternative branch exists in the current execution path, the system can return to the predicate defined at the upper level and select another executable branch. When an intermediate task cannot be completed, the system can backtrack to a higher-level task node to rearrange the execution path. By combining the verification module with the control flow adjustment mechanism, the neural symbolic logic program can achieve sequential execution, conditional branching, loop iteration, and failure backtracking during execution, thereby improving the stability and reliability of complex task execution. If the task corresponding to the top-level target predicate has been verified as successfully completed, then proceed to step 3 to output the task execution result; If the top-level objective is not yet achieved, the subsequent predicates will continue to be executed according to the adjusted control flow.

[0083] Step 3: Output the task execution results In this embodiment, when the verification results in step 2 indicate that the task corresponding to the top-level target predicate has been completed, the task execution result is output, and the neural predicates generated during the execution process are saved for subsequent task calls and combination reuse.

[0084] Specifically, when the top-level target predicate is determined to have executed successfully, the system marks the current task as completed and outputs the execution result corresponding to that task. The execution result may include information such as the target task completion status, task execution path, execution status of each subtask, and the final environment state. Further, the neural predicates trained in step S6 during execution are written into a predicate library for storage. The predicate library stores skill modules that already possess execution capabilities, enabling the neural predicates to be directly invoked in subsequent tasks without repeated training. In this way, the present invention can gradually accumulate executable skills for different atomic tasks, improving the system's ability to combine and reuse complex tasks.

[0085] In one implementation, when the neural symbolic logic program corresponding to a subsequent task contains atomic predicates with semantics consistent with the saved neural predicates, the system can directly call the corresponding neural predicates from the predicate library to participate in execution, thereby shortening task preparation time and improving task execution efficiency. Thus, this invention not only completes the current target task but also enhances the system's adaptability and generalization ability to unseen tasks through skill accumulation and reuse.

[0086] Upon completion of this step, the implementation process of real-time environmental action planning based on the large language model of neural symbolic logic programming and reinforcement learning in this embodiment is finished.

Claims

1. A method for planning real-time environmental actions based on a neural symbolic large language model, characterized in that, Includes the following steps: S1. Obtain the task objective to be executed, the status information of the real-time interactive environment, and the background knowledge related to the task; S2. Based on a pre-trained large language model, a hierarchical natural language planning mechanism is generated according to the task objective, environmental state information, and background knowledge. S3. Compile the hierarchical natural language program into a neural symbolic logic program with hierarchical task decomposition and explicit control flow, and divide the predicates in the program into definition predicates that can be recursively expanded and atomic predicates that need to be directly mapped to environmental actions; S4. Perform a predicate library matching verification on the atomic predicate. If there is a trained skill in the predicate library that corresponds to the atomic predicate, then directly bind the trained skill to the atomic predicate as the execution implementation. If there is no corresponding trained skill in the predicate library, then the atomic predicate is identified as the neural predicate to be generated; S5. For the neural predicate to be generated, based on the inverse learning framework, and combining the task objective, background knowledge, the constraint relationship of the neural symbolic logic program and the environmental state change information, construct the training objective, success judgment condition and reward function corresponding to the neural predicate; S6. Based on the training objective, success determination conditions and reward function, construct the corresponding reinforcement learning task, take the original environmental perception state as input and the environmental action interface as output, obtain the low-level action policy corresponding to the neural predicate through reinforcement learning training, establish the mapping relationship from the original environmental perception state to the action output, and after training, encapsulate the low-level action policy into a neural predicate and store it in the predicate library. S7. Using the top-level predicate corresponding to the target task as the query entry point, the task scheduling is executed based on the neural symbolic logic program, the predicate is recursively expanded, and the trained skills or neural predicates corresponding to the atomic predicates are called to execute environmental action control. Simultaneously, the environmental state before and after the execution of the atomic predicate is verified, and subsequent predicate scheduling, execution retry, path backtracking or branch switching are performed based on the verification results until the target task is completed.

2. The method for implementing real-time environmental actions based on a neural symbolic large language model according to claim 1, characterized in that: The neural symbolic logic program consists of multiple logic rules, which are constructed based on chained meta-rules. The expression of the chained meta-rules is as follows: ; in, , , These are the predicates corresponding to the target task or subtask, respectively. , , These are the state variables in the state space, corresponding to the start state, end state, and intermediate state of the task execution, respectively. The predicates in the logical rules all adopt the form of binary predicates with state input and state output, which are used to characterize the completion state of the corresponding subtask when the agent transitions from the input state to the output state.

3. The method for planning real-time environmental actions based on a neural symbolic large language model according to claim 1, characterized in that: The defined predicate is a high-level task predicate that appears as the head of at least one logical rule in a neural symbolic logic program and can be decomposed through recursive expansion of logical rules. The atomic predicates are skill predicates that have no further symbolic definition in the current neural symbolic logic program and need to be directly mapped to executable actions at the underlying environment level.

4. The method for planning real-time environmental actions based on a neural symbolic large language model according to claim 1, characterized in that: In step S5, the process of constructing the training objective, success criteria, and reward function based on inverse learning is as follows: The neural predicate to be generated is taken as the inverse term. Combining the task objective, background knowledge, and the constraints of the neural symbolic logic program, the target state change conditions that the neural predicate must satisfy to be executed successfully are determined. Based on the target state change conditions, corresponding success determination conditions are generated, and then a corresponding reward function is generated based on the success determination conditions and environmental state change information.

5. The method for planning real-time environmental actions based on a neural symbolic large language model according to claim 4, characterized in that: The success criteria are determined by comparing the changes in the number of target objects, the state of target tools, key attribute markers, or task-related state variables in the environment before and after the execution of the atomic predicate.

6. The method for planning real-time environmental actions based on a neural symbolic large language model according to claim 4, characterized in that: The reward function adopts a reward shaping method based on the difference in the amount of target objects held, and the specific process is as follows: Execute the current action and obtain the observation status, base reward value, termination signal and additional information returned by the environment; If the current action does not belong to the preset set of legal actions, then an illegal action penalty is applied to the base reward value; Obtain the current holding amount of the target object from the additional information, compare it with the holding amount of the target object at the previous time step, and if the current holding amount is greater than the holding amount at the previous time step, apply the target achievement reward to the base reward value and set the termination signal to true; The final output includes the observation status, the updated reward value, the termination signal, and additional information.

7. The method for planning real-time environmental actions based on a neural symbolic large language model according to claim 1, characterized in that: In step S6, the specific process of reinforcement learning training is as follows: Construct a reinforcement learning task corresponding to the neural predicate to be generated. The reinforcement learning task includes a state space, an action space, an environmental state transition process, a reward function, and a discount factor. During training, the agent continuously interacts with the environment and iteratively updates the policy parameters according to the reward function until the success criteria are met. After training, the low-level action strategies are encapsulated into neural predicates that correspond one-to-one with the atomic predicates and stored in the predicate library to achieve reusable storage.

8. The method for planning real-time environmental actions based on a neural symbolic large language model according to claim 1, characterized in that: In step S7, the task scheduling process employs a predicate dynamic selection mechanism guided by a large language model, specifically as follows: Based on the neural symbolic logic program and the current environment state, determine the set of candidate predicates that can be executed at present; The current environment state, task objectives, historical execution trajectory, and structural constraints of the neural symbolic logic program are input into the large language model. The large language model evaluates the priority of the predicates in the candidate predicate set and selects the predicate to be called first. The large language model selects predicates only from the set of candidate predicates.

9. The method for planning real-time environmental actions based on a neural symbolic large language model according to claim 1, characterized in that: In step S7, the specific process of result verification and control flow adjustment is as follows: Obtain the first environment state before the execution of the atomic predicate and the second environment state after the execution. Based on the success determination condition corresponding to the atomic predicate, compare the first environment state and the second environment state. If the comparison result meets the success criteria, then continue with the subsequent predicate scheduling. If the comparison result does not meet the success criteria, then according to the explicit control flow structure of the neural symbolic logic program, the execution of the atomic predicate is retried, backtracked to the upper-level defined predicate, or switched to other executable branches.

10. The method for planning real-time environmental actions based on a neural symbolic large language model according to claim 1, characterized in that: The trained skills and encapsulated neural predicates stored in the predicate library can be directly called and combined for reuse in subsequent tasks without repeated training.

Images