A control method of a body-equipped intelligent agent and a computing device

Abstract

This application provides a control method and computing device for an embodied intelligent agent, relating to the field of artificial intelligence technology, to improve the accuracy and success rate of an embodied intelligent agent when performing a task. The method includes: inputting the task to be performed and initial context-aware information of the embodied intelligent agent into a readable cognitive model; determining the causal rules corresponding to the task to be performed through the readable cognitive model; the readable cognitive model is used to determine the causal rules that must be followed when performing the task in the current environment based on causal knowledge learned from multiple historical interaction data; inputting the task to be performed, initial context-aware information, and causal rules into an instruction generation model; determining a target action sequence through the instruction generation model; the instruction generation model is used to determine the action sequence that can complete the task in the current environment in a manner consistent with the causal rules; and controlling the embodied intelligent agent to perform actions according to the action order indicated in the target action sequence.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a control method and computing device for an embodied intelligent agent. Background Technology

[0002] In the wave of artificial intelligence development, embodied intelligent agents, as an intelligent form capable of deep interaction with the physical world and possessing perception, decision-making, and action capabilities, should have demonstrated strong application potential. However, current embodied intelligent agents rely too heavily on pre-programmed skills when performing tasks, which strictly limits their action patterns and functional scope, resulting in a lack of flexibility and adaptability. This means they can only function in specific, familiar environments; once they enter unknown scenarios, they are essentially "blind" and "deaf," exhibiting extremely poor generalization ability and difficulty in completing complex tasks. This greatly limits the widespread application of embodied intelligence in real life. Summary of the Invention

[0003] This application provides a control method and computing device for an embodied intelligent agent, which improves the accuracy and success rate of the embodied intelligent agent when performing a task in the current environment.

[0004] In a first aspect, embodiments of this application provide a control method for an embodied intelligent agent. The method includes: inputting the task to be executed and the initial context-aware information of the embodied intelligent agent into a readable cognitive model, and determining the causal rules corresponding to the task to be executed through the readable cognitive model; wherein, the initial context-aware information includes the environmental perception data and the self-state data of the embodied intelligent agent in the current environment; the readable cognitive model is used to determine the causal rules that need to be followed when executing the task to be executed in the current environment based on causal knowledge learned from multiple historical interaction data; the historical interaction data includes: historical actions performed by the embodied intelligent agent, context change information of the context-aware information when performing historical actions, and task execution results; inputting the task to be executed, the initial context-aware information, and the causal rules into an instruction generation model, and determining a target action sequence through the instruction generation model; the instruction generation model is used to determine the action sequence that can complete the task to be executed in the current environment in a manner that conforms to the causal rules; and controlling the embodied intelligent agent to execute actions in the order of actions indicated in the target action sequence.

[0005] The embodied agent control method provided in this application, upon receiving a task to be executed, inputs the task to be executed and the initial context-aware information of the embodied agent into a readable cognitive model. Since the initial context-aware information represents the embodied agent's environmental perception data and its own state data in the current environment, the readable cognitive model can fully grasp "what the current environment is like and what the agent's own state is." Combined with causal knowledge from historical interaction data, invalid rules that do not match the current context can be eliminated, ensuring that the determined causal rules fit the actual execution scenario and providing a reliable basis for the generation of subsequent action sequences. Furthermore, the task to be executed, the initial context-aware information, and the causal rules are input into an instruction generation model, enabling the instruction generation model to perform reasoning and planning based on the causal rules, thereby generating specific action steps that can realize the task to be executed in the current environment, resulting in a target action sequence. Finally, the embodied agent is controlled to execute actions according to the action sequence indicated in the target action sequence to realize the task to be executed. In other words, the solution provided in this application first combines the causal knowledge learned by the readable cognitive model from multiple sets of historical interaction data to achieve contextualized causal understanding when determining the target action sequence. Then, the causal understanding is transformed into an executable target action sequence through the instruction generation model, enabling the intelligent agent to plan as if it had common sense related to the current scene, which significantly improves the accuracy and success rate of the embodied intelligent agent in completing tasks in complex and dynamic environments.

[0006] In one possible implementation, the instruction generation model is specifically used to: determine the first action required to complete the task in the current environment based on the initial context-aware information of the embodied agent, in accordance with causal rules; determine the nth action required to complete the task based on the estimated context-aware information of the embodied agent after the (n-1)th action is executed, until the obtained action sequence can complete the task; the action sequence includes the first action and the nth action, where n is an integer greater than 1; wherein, the estimated context-aware information is determined by a prediction model; the prediction model is used to predict the context-aware information after the current action is executed based on the context-aware information of the embodied agent before the current action is executed.

[0007] In this implementation, the first action of the target action sequence is determined directly based on real initial context-aware information, without relying on predicted data. This avoids initial planning biases from the outset, laying a reliable foundation for the entire action sequence. Subsequent actions are based on the "predicted context-aware data after the previous action" output by the prediction model. This allows for real-time matching of state changes after action execution, avoiding the accumulation of biases caused by planning all actions using initial context-aware data. This ensures that each action aligns with the dynamically changing actual scenario, ultimately improving the compliance and task completion rate of the overall action sequence.

[0008] In one possible implementation, the context-aware information is estimated through multiple prediction models. The method further includes: determining multiple candidate actions that the embodied agent can execute; for each candidate action, predicting the first context-aware information after the embodied agent executes the candidate action using multiple prediction models; determining the intrinsic motivation corresponding to the candidate action based on the deviation between the first context-aware information predicted by each prediction model in the multiple prediction models; wherein, the larger the deviation, the stronger the intrinsic motivation; controlling the embodied agent to execute a target action, and determining the second context-aware information after the embodied agent executes the target action; the target action is a candidate action among multiple candidate actions whose intrinsic motivation meets preset conditions; the preset conditions are: the strongest intrinsic motivation, the intrinsic motivation is greater than a preset motivation threshold, and the top n with the highest intrinsic motivation; determining the prediction error corresponding to the multiple prediction models based on the first and second context-aware information predicted by each prediction model; and updating the multiple prediction models based on the prediction error when the prediction error is greater than or equal to a preset error threshold, or the intrinsic motivation is greater than or equal to a preset motivation threshold.

[0009] In this implementation, the intrinsic incentive is quantified by the deviation of prediction results from multiple prediction models. The larger the deviation, the stronger the incentive, ensuring that candidate actions that can fill the cognitive gaps in the model (such as actions with high prediction uncertainty) are prioritized as target actions. The agent's exploration process is no longer a random and inefficient blind attempt, but an active exploration with a clear goal. It can autonomously identify and fill cognitive gaps, systematically exploring the boundaries of its own action space. Then, based on the actual situational awareness information after the target action is executed (second situational awareness information), combined with the prediction deviation, the prediction error is calculated, and the prediction model is updated based on the error. This allows the prediction model to continuously absorb new environment exploration data, gradually reducing the prediction deviation for new environments, and providing more reliable prediction support for subsequent action planning.

[0010] In one possible implementation, the method further includes: determining whether the estimated context-aware information of the embodied agent after each action is performed satisfies the task objective of the task to be performed; if it is determined that the task objective of the task to be performed is satisfied, then it is determined that the obtained action sequence can complete the task to be performed.

[0011] In this implementation, the alignment between the estimated context-aware data and the task objective is verified after each action planning step. This allows for timely detection of deviations from the objective and avoids the generation of invalid action sequences due to overall planning lag, significantly improving the adaptability of action sequences to the task objective. Furthermore, planning stops immediately when the objective is met, reducing the design and computational overhead of redundant actions. If the objective is not met, the next action is iteratively planned based on the current estimated context, ensuring the coherence and logical closure of the action sequence. This approach guarantees effective task completion, significantly improves action planning efficiency, and reduces the time and computational costs of agent task execution.

[0012] In one possible implementation, the prediction model is used to predict, based on the third context-aware information of the embodied agent in its current state, the first context-aware information after the embodied agent has performed a candidate action; after determining the second context-aware information after the embodied agent has performed the target action, the control method further includes: In this implementation, the real environment is dynamic and full of uncertainty. Therefore, when using the prediction model for prediction, the real-time context-aware information (third context-aware information) of the embodied agent is used as input to predict the first context-aware information after the embodied agent has executed the candidate action.

[0013] Furthermore, after executing the target action, the data collected during the exploration process (target action, third-context awareness information, and second-context awareness information) is used as historical interaction data. This data is used to update the readable cognitive model while simultaneously training the prediction model. The prediction model updates its parameters to improve prediction accuracy, and the readable cognitive model updates its knowledge base and interaction rules simultaneously. This ensures that the selection of subsequent candidate actions (based on the readable cognitive model) and the prediction of action consequences (based on the prediction model) are based on the same batch of the latest exploration data. The decision-making logic and prediction capabilities are matched in real time, avoiding ineffective exploration due to information asynchrony. This allows the readable cognitive model to learn the core features of the environment, the interaction logic between actions and the environment, and its own action space boundaries during exploration. It then gradually builds an interpretable cognitive framework adapted to the current environment, providing more accurate cognitive support for subsequent candidate action selection and exploration strategy optimization.

[0014] In one possible implementation, after controlling the embodied agent to execute actions in the order of actions indicated in the target action sequence, the method further includes: obtaining the execution result and target context awareness information after the embodied agent executes the target action sequence; and using the initial context awareness information, the target action sequence, the target context awareness information, and the execution result as historical interaction data.

[0015] In this implementation, the data during the execution of the task to be executed is used as historical interaction data. This allows the readable cognitive model to simultaneously absorb multi-dimensional causal knowledge from the complete execution of the task, including task adaptation logic, the relationship between context and result, the causal mapping between action and environment, and the direction of execution feedback optimization. This enables continuous iteration of the cognitive rule base, significantly improving the accuracy of subsequent task planning and environmental adaptability. At the same time, it enables real-time accumulation and reuse of cognitive experience, avoiding repeated exploration and improving the efficiency and robustness of the agent's task execution.

[0016] In one possible implementation, the method further includes: if the task execution result of the task to be executed is execution failure, taking the target action sequence, the task to be executed and the initial context-aware information as negative samples, and updating the instruction generation model based on the negative samples.

[0017] In this implementation, the triplet of the failure scenario (target action sequence + task + initial situation) is used as negative sample input. The instruction generation model can clearly learn that "in the initial situation of XX, generating the action sequence of XX for the task of XX will lead to failure", forming a two-way cognition of "effective and ineffective". When generating instructions in the future, such erroneous action sequences can be directly avoided, which greatly reduces the probability of repeatedly executing failed actions.

[0018] In one possible implementation, controlling the embodied intelligent agent to execute actions in the order of actions indicated in the target action sequence includes: constructing a virtual operating environment based on initial context-aware information; wherein the virtual operating environment is used to simulate the current environment; controlling the virtual intelligent agent to simulate execution in the virtual operating environment according to the target action sequence to obtain a simulation result; the virtual intelligent agent is a digital clone of the embodied intelligent agent; and if the simulation result indicates successful execution, controlling the embodied intelligent agent to execute actions in the order of actions indicated in the target action sequence.

[0019] In this implementation, a virtual operating environment is first constructed based on an initial scenario, and a virtual agent is controlled to simulate the execution of a target action sequence. Real execution is then initiated after successful simulation. This approach not only verifies the logical coherence of the target action sequence's code in advance through virtual simulation, preventing interruptions during real execution due to logical errors, but also predicts and eliminates potentially dangerous actions such as collisions and power overloads, reducing the risk of the embodied agent damaging itself or the environment during execution in the real environment. Furthermore, it eliminates the need for repeated trial and error in the real environment, reducing ineffective execution costs. Ultimately, while ensuring safety and reliability, it improves the efficiency and success rate of task completion.

[0020] In one possible implementation, the method further includes: if the simulation result indicates execution failure, inputting the simulation result into the instruction generation model, and using the instruction generation model to redetermine a target action sequence based on the simulation result.

[0021] In this implementation, the triplet of the failure scenario (target action sequence + task + initial context) is used as negative sample input. The instruction generation model can clearly learn that "in the initial context, generating an action sequence for the task will lead to failure", forming a two-way cognition of "effective and ineffective". When generating instructions in the future, such erroneous action sequences can be directly avoided, which greatly reduces the probability of repeatedly executing failed actions.

[0022] In a second aspect, embodiments of this application provide a computing device, which includes a processor and a memory; the processor is coupled to the memory; the memory is used to store computer instructions, which are loaded and executed by the processor to enable the computing device to implement the methods provided in the first aspect and its possible implementations described above.

[0023] Thirdly, embodiments of this application provide a computer-readable storage medium comprising: computer software instructions; and, when the computer software instructions are executed in a computing device, causing the computing device to implement the method provided by the first aspect and its possible implementations described above.

[0024] Fourthly, embodiments of this application provide a computer program product that, when run on a computing device, causes the computing device to execute the steps of the relevant method described in the first aspect above, so as to implement the method of the first aspect above.

[0025] The beneficial effects of the second to fourth aspects mentioned above can be referred to the corresponding description of the first aspect, and will not be repeated here. Attached Figure Description

[0026] Figure 1 A scenario diagram provided for an embodiment of this application; Figure 2 A flowchart illustrating a control method for an embodied intelligent agent provided in this application embodiment. Figure 1 ; Figure 3 A flowchart illustrating a control method for an embodied intelligent agent provided in this application embodiment. Figure 2 ; Figure 4 A flowchart illustrating a control method for an embodied intelligent agent provided in this application embodiment. Figure 3 ; Figure 5 This is a schematic diagram of the structure of a control device for an embodied intelligent agent provided in an embodiment of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] It should be noted that in the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design scheme described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0029] To facilitate a clear description of the technical solutions of the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish the same or similar items with essentially the same function and effect. Those skilled in the art can understand that the terms "first" and "second" are not intended to limit the quantity or execution order.

[0030] For ease of description, the application scenarios involved in this application will be introduced first.

[0031] Please see Figure 1 , Figure 1 This is a schematic diagram of a scenario provided for an embodiment of this application. For example... Figure 1 As shown, the control system for the embodied intelligent agent provided in this application embodiment may include: an embodied intelligent agent 11 and a control device 12 for the embodied intelligent agent. The embodied intelligent agent 11 and the control device 12 can communicate with each other via wired or wireless means, and this application embodiment does not impose any limitations on this.

[0032] In this embodiment of the application, the embodied intelligent agent 11 can specifically refer to intelligent devices that have the ability to interact with the environment as physical entities, such as drones, industrial robots, warehousing and handling equipment (such as AGV automated guided vehicles), service robots (such as food delivery robots and home cleaning robots), logistics sorting equipment, collaborative robots, unmanned delivery vehicles, inspection robots, etc. Figure 1 The humanoid robot in the example is only an example. The core feature of the embodied intelligent agent 11 is that it can perceive the environment and perform physical actions through its own hardware (such as robotic arms, mobile chassis, sensors, etc.) to complete specific tasks (such as grasping, carrying, sorting, inspection, etc.). The embodiments of this application do not limit the specific implementation of the embodied intelligent agent 11.

[0033] Because the actions of an embodied intelligent agent are highly dependent on environmental adaptability and its own capabilities, a lack of environmental awareness can lead to collisions with obstacles, inaccurate target object positioning, or task failure due to mismatched environmental parameters. Ignoring its own state data can cause equipment malfunctions (such as shutdowns due to high-load tasks performed with low battery) and insufficient motion precision (such as a robotic arm continuing to perform grasping operations according to standard parameters after joint wear). Therefore, when controlling the movement of an embodied intelligent agent, it is necessary to first acquire environmental perception data and its own state data to ensure the safety, accuracy, and efficiency of task execution, allowing subsequent decisions to better align with the needs of the actual scenario.

[0034] In some embodiments, the embodied intelligent agent 11 may be equipped with a variety of sensors to collect contextual awareness information of the environment in which the embodied intelligent agent is located, such as: visual sensors, infrared sensors, radar temperature and humidity sensors, gas sensors, attitude sensors, power sensors, load sensors or fault diagnosis sensors, so as to accurately obtain environmental awareness data of the environment in which the embodied intelligent agent is located and its own state data.

[0035] Environmental perception data refers to the external environmental feature data directly related to the current task being performed, collected by the embodied intelligent agent through various external sensors. Its core purpose is to determine "where the task is being performed" and "what the external constraints of the task execution are." Environmental perception data can include one or more of the following categories: Spatial and location data: such as the three-dimensional dimensions (length / width / height) of the space where the agent is currently located, the precise coordinates of the target object (such as the shelf level and coordinate point of materials in a warehouse scenario), the position and outline of obstacles (such as equipment next to the production line, other agents in the aisle, and ground protrusions), and the agent's own positioning information in the environment (such as pose based on SLAM algorithm and GPS coordinates). Physical environment data: such as ambient temperature and humidity, air pressure, light intensity (affecting the accuracy of visual sensors), noise decibels (affecting voice interaction skills), presence of dust / flammable and explosive gases (special operating scenarios), ground flatness / friction (affecting AGV movement), etc. Target object data: such as the attributes of the object to be operated (shape, size, weight, material, surface roughness), the state of the object (such as whether the material is packaged or whether the equipment is in operation), the relative positional relationship of multiple target objects (such as the alignment of the material with the conveyor belt in a sorting task), etc. Dynamic environmental data includes: the movement trajectory of moving objects in the environment (such as the walking path of other robots in the workshop), real-time changes in ambient lighting (such as direct sunlight / shading in outdoor scenes), and external interference sources (such as electromagnetic interference and personnel movement).

[0036] Self-state data refers to the data on the capabilities and operational status of an embodied intelligent agent collected through built-in sensors or control systems. It is primarily used to determine whether the embodied intelligent agent "can perform a task" and "in what state" it performs the task. Specifically, it can include one or more of the following categories of data: Power and energy consumption data: such as remaining battery power / range, charging status, remaining fuel / gas supply (such as the air pressure value of the pneumatic robotic arm), real-time motor power / energy consumption data, etc. Hardware operating condition data: such as the rotation angle / wear of the robotic arm joints, the gripping force / adsorption force of the actuators (grippers, suction cups), the tire pressure / motor speed of the mobile chassis, the working status of the sensors themselves (such as whether the camera is faulty or whether the lidar is offline), the temperature / vibration value of the transmission components, etc. Operational posture and capability data: the embodied agent's current motion posture, load status, range of executable actions, cumulative duration of executed tasks / failure rate, etc. Software and communication data: such as the operating memory usage of the control system, task execution progress (e.g., completed grabbing and pending handling), communication signal strength / delay with the control device of the embodied intelligent agent, locally cached instruction execution records, fault codes (e.g., sensor malfunction, reason code for action execution failure), etc.

[0037] For example, the embodied intelligent agent 11 can collect features such as the shape, position, and color of target objects in the environment through visual sensors (such as high-definition cameras and 3D depth cameras), detect environmental parameters such as the distribution of surrounding obstacles, spatial dimensions, and distances through infrared sensors and radar, and monitor environmental conditions in special scenarios (such as high-temperature or flammable and explosive environments) through temperature and humidity sensors and gas sensors, thereby using the collected data as environmental perception data. It can also obtain motion posture and position information through posture sensors (such as gyroscopes and accelerometers) built into the embodied intelligent agent, provide real-time feedback on remaining battery power through a power sensor, monitor the current load weight through a load sensor, and collect equipment data such as the working condition of the robotic arm joints, the operating status of the motors, and the effectiveness of sensor operation through a fault diagnosis sensor, thus obtaining the embodied intelligent agent's own state data.

[0038] After collecting this raw data, the embodied agent can upload it to its control device in real time. The control device then integrates this data into complete context-aware information, providing comprehensive and accurate foundational support for subsequent action sequence generation. As one implementation method, the embodied agent's control device can first perform targeted preprocessing on the received raw data (such as noise reduction through filtering algorithms, removal of duplicate and redundant data, and standardization of heterogeneous data from different sensors into a standardized format), and then integrate the processed effective data into complete and standardized context-aware information. This effectively avoids decision-making biases caused by messy and error-prone raw data.

[0039] In this embodiment, the control device 12 of the embodied intelligent agent can be integrated into the embodied intelligent agent 11 or set up independently of the embodied intelligent agent 11; this embodiment does not impose any limitations on this. Furthermore, this embodiment does not limit the specific form of the control device 12 of the embodied intelligent agent. The control device 12 of the embodied intelligent agent can be a controller; it can also be an electronic device, such as a personal computer (PC), laptop computer, mobile device, tablet computer, etc. Alternatively, the control device 12 of the embodied intelligent agent can also be a server, or a server cluster composed of multiple servers. In some implementations, the server cluster can be a distributed cluster server.

[0040] The control device 12 of the embodied intelligent agent can integrate an interaction module or a communication module, and then receive tasks to be executed through the integrated interaction module (such as voice interaction, touch interaction components) or communication module (such as wireless communication, wired communication unit). That is to say, the control device 12 of the embodied intelligent agent can determine that it has received a task to be executed when it receives a voice command (such as a task command issued by the user through the voice interaction module), a task signal sent by a remote terminal, or a task command input on the local touch panel. The embodiments of this application do not limit the triggering conditions of the task to be executed.

[0041] In this embodiment, after receiving a task to be executed, the control device 12 of the embodied intelligent agent can communicate and interact with the embodied intelligent agent to obtain environmental perception data and its own state data in the current environment, thereby obtaining initial situational perception information. Then, the task to be executed and the initial situational perception information of the embodied intelligent agent are input into a readable cognitive model, which determines the causal rules corresponding to the task to be executed. Further, the control device 12 of the embodied intelligent agent inputs the obtained causal rules, the task to be executed, and the initial situational perception information into an instruction generation model. The instruction generation model is used to determine the target action sequence that can complete the task to be executed in the current environment in a manner that conforms to the causal rules. Finally, the control device 12 of the embodied intelligent agent controls the embodied intelligent agent to execute actions according to the action sequence indicated in the target action sequence.

[0042] It can be seen that when determining the target action sequence, the control device 12 of the embodied intelligent agent first combines the causal knowledge learned from multiple sets of historical interaction data by the readable cognitive model to achieve scenario-based causal understanding. Then, through the instruction generation model, the causal understanding is transformed into an executable target action sequence, enabling the intelligent agent to plan as if it had common sense related to the current scenario. This significantly improves the accuracy and success rate of the embodied intelligent agent in completing tasks in complex and dynamic environments.

[0043] It should be understood that the readable cognitive model and instruction generation model can be deployed in the control device 12 of the embodied intelligent agent or in other electronic devices. The control device 12 of the embodied intelligent agent obtains the causal rules and target action sequence corresponding to the task to be executed through communication and interaction with other electronic devices. This application embodiment does not limit this.

[0044] It should be noted that the system architecture and application scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. Those skilled in the art will understand that, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0045] The control method for embodied intelligent agents provided in the embodiments of this application will now be described with reference to the accompanying drawings.

[0046] The control method for embodied intelligent agents provided in this application embodiment can be applied to the control device 12 for embodied intelligent agents described above, specifically to the processor of the control device 12. Please refer to... Figure 2 The control method for an embodied intelligent agent provided in this application includes the following steps: S101. Input the initial context-aware information of the task to be executed and the embodied intelligent agent into the readable cognitive model, and determine the causal rules corresponding to the task to be executed through the readable cognitive model.

[0047] The initial contextual awareness information includes the embodied agent's environmental perception data and its own state data in the current environment. Environmental perception data refers to the embodied agent's perception of its external environment (such as spatial layout, obstacles, lighting, etc.), while its own state data refers to the embodied agent's monitoring information of its own operational status (such as battery level, mechanical component position, sensor operation status, etc.). The specific content of the environmental perception data and the own state data can be found in the above embodiments and will not be repeated here.

[0048] In this embodiment of the application, the readable cognitive model is used to determine the causal rules that need to be followed when performing a task in the current environment based on causal knowledge learned from multiple historical interaction data; the historical interaction data includes: historical actions performed by the embodied intelligent agent, situational change information of situational awareness information when performing historical actions, and task execution results.

[0049] A readable cognitive model is an intelligent decision-making framework built upon causal reasoning and cognitive science principles. Its core lies in mining causal relationships between variables from historical interaction data and transforming them into a set of rules that are understandable to humans and executable by machines. This guides embodied agents to make transparent, explainable, and adaptive decisions in the current environment. By simulating the human process of "causal cognition," a readable cognitive model not only endows intelligent agents with the ability to understand task objectives and causal rules but also explains decision-making logic through explicit rules, significantly improving the efficiency of human-machine collaboration and the reliability of task execution.

[0050] As an implementation approach, a readable cognitive model is not a purely data-driven neural network; it typically comprises two key components: Data storage module: also known as a structured knowledge base, similar to the "memory bank" of an intelligent agent, is specifically used to store interpretable and searchable historical interaction data (such as tables, graphs, and structured text). Cognitive reasoning rule layer: Based on the data stored in the data storage module, it summarizes the interaction rules between actions and the environment, environmental constraints, etc., to form interpretable reasoning logic (such as the rule "execute action A → environment state changes from X to Y"), and finally obtains the causal rules.

[0051] A readable cognitive model can first identify causal relationships between key variables from multiple historical interaction data sets, and then generate the causal rules that need to be followed in the corresponding scenario. For example, if the historical interaction data is: an agent pushes a cup on a table by 10cm, and the cup falls off the table and breaks, the causal knowledge that the readable cognitive model can identify can include direct and indirect causal relationships. A direct causal relationship can be represented as: "The agent applies force" → "The cup moves 10cm on the table"; an indirect causal relationship can be represented as: "The cup moves 10cm" → "The cup's center of gravity goes beyond the edge of the table" → "The cup loses its support and falls and breaks." Based on this, the readable cognitive model can summarize the causal rule that needs to be followed in the scenario of "pushing something on a table": if the cup's center of gravity goes beyond the edge of the table, it will break.

[0052] After inputting the initial context-aware information of the task to be executed and the embodied intelligent agent into a readable cognitive model, the model will first retrieve interaction data related to the current scene from a massive amount of historical interaction data. Here, "related" may be related in terms of task dimension (such as the same task type or consistent core requirements in the task description) or similar in terms of context dimension (such as similar spatial layout of the environment, similar distribution of obstacles, or similar battery level or component status of the intelligent agent itself). The specific dimension of association is not limited in the embodiments of this application.

[0053] Furthermore, the readable cognitive model analyzes the retrieved relevant interaction data to identify the deep-seated causal factors affecting the success or failure of the task. For example, when analyzing the "desktop object moving" task, the model not only learns the commonality that "the object falling causes the task to fail," but more importantly, it can summarize the causal relationship between the core physical properties that cause the object to fall (such as the object's fragility, weight, value, etc.) and the task objective (such as "safely moving the object" or "quickly clearing the desktop") from the data.

[0054] Based on this, readable cognitive models can identify refined and conditional causal rules. As a feasible implementation method, readable cognitive models can summarize causal rules through structured analysis of historical interaction data. This process involves three steps: First, extracting standardized quadruplets (task type, object attribute, environmental state, action result) from the data. For example, in a moving task, recording the object material, the coefficient of friction of the table, the applied pushing force, and whether it ultimately falls and breaks. Second, constructing a probabilistic causal graph based on a large number of quadruplets, using causal discovery algorithms to identify stable causal directions between variables, and distinguishing between correlation and causality through counterfactual verification. For example, verifying "if the object material is changed from glass to sponge, is it still easily broken?" Finally, the model encapsulates the verified causal path into conditional production rules, in the form of "if the task type is T, the object attribute satisfies C, and the environmental state is S, then action A may lead to result O, and result O has an impact of E on the task objective." Therefore, the model does not derive simple prohibitions (such as "no falling"), but rather causal knowledge that can be dynamically reasoned: it can judge the causal consequences and task utility of the same physical event (such as falling) in different situations based on real-time perceived attributes (such as the fragility of the object and the urgency of the task), thereby generating differentiated decision-making basis.

[0055] For example, when performing the "desktop moving" task, the retrieved causal rules are as follows: If the object is fragile (such as a glass) or an important item, "object falling" will result in "item damage" or "task failure". Therefore, it is necessary to dynamically monitor the distance between the object and the edge, and limit the range of movement when the distance is too close, in order to prevent falling due to cause and effect.

[0056] If the object is durable and not a critical item (such as a pillow), and the core objective of the task is to "quickly clear the desktop," then "object falling" might be considered an acceptable, or even more efficient, implementation path. Therefore, the rule will not trigger movement restrictions, or it may generate different action strategies (such as directly pushing it down).

[0057] S102. Input the task to be executed, the initial context-aware information, and the causal rules into the instruction generation model, and determine the target action sequence through the instruction generation model.

[0058] The instruction generation model is used to determine the sequence of actions that can complete the task in the current environment in accordance with causal rules. In other words, the instruction generation model is used to generate a sequence of actionable actions that can effectively achieve the task objective in the current environment based on the understanding and reasoning of causal rules, and to ensure that each action in the sequence and its expected consequences are consistent with the world's operating mechanism described by the causal rules.

[0059] The essence of the instruction generation model is an algorithmic model with scene adaptation and logical execution capabilities. It integrates and analyzes three key types of information: "task to be executed (target)," "initial contextual awareness information (current environment and self-state)," and "causal rules (cognition of the interaction mechanisms between entities in the environment)." Using causal rules as a reasoning engine, it deduces how different action chains lead to state transitions in the state space, thereby proactively generating a logically coherent, logically sound sequence of actions, where the expected effect of each step is causally explained. In other words, it acts like a "strategic planner" for the intelligent agent, not only knowing "what to do" and "what is happening now," but also understanding "why things are developing this way," thus enabling it to formulate a detailed, causally sound, explainable, and goal-oriented action plan.

[0060] In the process of generating the target action sequence, the instruction generation model can generate actions one by one in a "step-by-step prediction" manner. During the generation of each action, compliance verification of causal rules must be carried out simultaneously: on the one hand, threshold verification is performed on the parameters of the current action to be generated (such as the magnitude of force, movement distance, execution speed, etc.) to ensure that the parameter values ​​do not exceed the limits of the causal rules; on the other hand, logical verification is performed on the order of the current action and the generated actions to ensure that the connection between actions conforms to the "causal chain" (such as the sequential logic of "adjusting the position of the execution component first, and then performing the operation action"), thereby eliminating the risk of task failure caused by disordered action order from the source.

[0061] In some embodiments, to help the instruction generation model generate target action sequences more accurately, a prediction model can be introduced. This model predicts the estimated context-aware data that the embodied agent's environment and its own state may present after performing a single action, thereby providing a reference for subsequent action planning that is more in line with the actual execution effect.

[0062] Specifically, as a feasible implementation method, the instruction generation model is used for: S11. Based on the initial context-aware information of the embodied agent, determine the first action required to complete the task to be performed in the current environment in a manner that conforms to the causal rules.

[0063] The instruction generation model can define the "completion criteria" (i.e., what state parameters must be met to be considered a task completed) based on the target parameters of the task to be executed (such as the task endpoint and accuracy requirements) and the initial situational awareness information (such as the initial state of the environment and the initial state of the agent itself). This is achieved by using causal rules to establish a causal state evolution path from the "initial state" to the "target state" of the embodied agent. Furthermore, the model can use causal inference to select atomic actions (i.e., indivisible basic execution units) that can drive the system state towards the "target state" along this causal path (excluding meaningless or deviating actions from the target) from all the atomic actions that the embodied agent can perform. From these selected actions, the model determines the action that conforms to the causal rules, fits the initial situation, and lays the groundwork for subsequent actions, and uses it as the first action required to complete the task.

[0064] As a feasible approach, when determining the first action of the task to be executed, the corresponding quantitative parameters of the action must be clearly defined simultaneously. The instruction generation model will be guided by the target parameters of the task to be executed, and combine specific data from the initial context-aware information (such as the relative distance between the execution component and the target object, the upper limit of the agent's own power, environmental spatial boundary parameters, etc.) to quantify the core parameters of the action (including displacement, power output value, execution time, and accuracy error range). After the calculation is completed, the model will perform compliance verification on the parameters according to the threshold constraints in the causal rules (such as the upper limit of power output and displacement accuracy requirements), and exclude parameter values ​​that exceed the limit range of the rules.

[0065] After successful validation, the model generates the first action of the task to be executed. This action must explicitly specify the execution component (such as a specific robotic arm or mobile chassis), include complete quantified parameters after validation, and simultaneously satisfy causal rule compliance and initial context adaptability.

[0066] S12. Based on the estimated contextual awareness information after the embodied agent completes the (n-1)th action, determine the nth action required to complete the task to be performed, until the obtained action sequence can complete the task to be performed.

[0067] The action sequence includes the first action and the nth action, where n is an integer greater than 1; the estimated context-aware information is determined by a prediction model.

[0068] After determining the first action in the target action sequence, subsequent actions need to be planned to form a complete action sequence. Since the accurate determination of subsequent actions relies on the embodied agent's real-time contextual awareness information, and the first action has not yet been executed in the current planning phase, the instruction generation model cannot directly obtain the real contextual awareness information after the action is executed. Furthermore, after the first action is executed, the embodied agent's own state (such as the position of its executing components and power consumption) and its relationship with the environment (such as the relative distance to the target object and the interaction state with the surrounding space) will dynamically change. If subsequent actions are planned based on the initial contextual awareness information, it is easy to lead to planning deviations.

[0069] Therefore, a predictive model needs to be introduced. Based on the initial context perception information before the execution of the first action and the complete execution parameters of the first action (including execution components, quantization parameters, etc.), the model predicts the context perception information after the execution of the first action and finally generates the predicted context perception information containing the predicted environmental perception parameters and the predicted self-state parameters, so as to provide accurate context basis for determining other actions in the future.

[0070] In other words, when determining actions other than the first action (the nth action), it is necessary to first use a prediction model to determine the estimated context-aware information after the previous action (the (n-1th action) has been executed. Then, the instruction generation model can determine the nth action based on the estimated context-aware information and the task to be executed. It should be understood that the specific method for determining the nth action can be referred to in S11 above, and will not be repeated here.

[0071] As a feasible implementation method, after determining the estimated context-aware data for each action, it is necessary to verify whether the estimated context-aware data meets the task objective of the task to be executed. If it meets the requirements (e.g., the estimated object position is consistent with the final coordinates required by the task, and the error is within the allowable range), then the obtained action sequence is determined to complete the task, the planning stops, and the complete target action sequence is output; if it does not meet the requirements (e.g., the estimated displacement does not meet the task requirements, or there are still unfinished action steps), then it is determined that the next action needs to be planned, and the planning process of subsequent actions is repeated based on the estimated context of the last action in the current sequence.

[0072] In this implementation, the alignment between the estimated context-aware data and the task objective is verified after each action planning step. This allows for timely detection of deviations that fail to meet the objective (such as insufficient displacement or excessive positional error), preventing the generation of invalid action sequences due to overall planning lag and significantly improving the adaptability of action sequences to the task objective. Furthermore, planning stops immediately when the objective is met, reducing the design and computational overhead of redundant actions. If the objective is not met, the next action is iteratively planned based on the current estimated context, ensuring the continuity and logical closure of the action sequence. This approach guarantees effective task completion, significantly improves action planning efficiency, and reduces the time and computational costs of agent task execution.

[0073] In this embodiment of the application, the prediction model is used to predict the context-aware information after the current action is completed based on the context-aware information of the embodied agent before the current action is executed.

[0074] To determine the estimated contextual awareness information after the current action is executed, the predictive model first needs to acquire contextual awareness information prior to the action (such as object attributes and spatial layout in the environment, and the state and dynamic parameters of the agent's own execution components); and complete execution parameters for the current action (such as execution components, displacement, and power output). Next, the predictive model combines physical rules (such as Newton's laws and the friction formula) to calculate the changes in the state of the environment and itself after the action. For example, displacement is calculated based on thrust and object mass, and power consumption is calculated based on power output. Simultaneously, bias compensation (correcting theoretical values ​​based on historical data) is introduced to obtain the estimated change. Finally, the estimated change is superimposed on the pre-execution context to generate estimated contextual awareness information, which can be verified using causal rules (such as ensuring the estimated position does not penetrate obstacles and power consumption does not exceed the initial reserve) to ensure its usability for subsequent action planning.

[0075] In this implementation, the first action of the target action sequence is determined directly based on real initial context-aware information, without relying on predicted data. This avoids initial planning biases from the outset, laying a reliable foundation for the entire action sequence. Subsequent actions are based on the "predicted context-aware data after the previous action" output by the prediction model. This allows for real-time matching of state changes after action execution, avoiding the accumulation of biases caused by planning all actions using initial context-aware data. This ensures that each action aligns with the dynamically changing actual scenario, ultimately improving the compliance and task completion rate of the overall action sequence.

[0076] S103. Control the embodied intelligent agent to perform actions in the order of actions indicated in the target action sequence.

[0077] The control device of the embodied intelligent agent can send complete target action sequence instructions to the control module of the embodied intelligent agent according to the action sequence indicated in the target action sequence, specifying the execution order, execution component, and specific parameters of each action. After receiving the instructions, the control module of the embodied intelligent agent sends action execution signals to the corresponding execution components (such as robotic arms or mobile chassis) one by one in sequence. After the previous action is completed, the execution instruction of the next action is triggered, ensuring that the intelligent agent strictly follows the sequence order to complete all actions and ultimately achieves the task to be performed.

[0078] As one feasible approach, please refer to Figure 3 S103 can be implemented as follows: S1031. Construct a virtual operating environment based on initial context-aware information.

[0079] The virtual runtime environment is used to simulate the current environment.

[0080] First, the collected initial context awareness information is preprocessed to filter out core parameters related to the current environment simulation (such as environmental space size, object position / shape / material properties, obstacle distribution, ground flatness, etc.) and remove redundant or invalid data; then, virtual environment construction tools (such as physics engine, 3D modeling module) are called to convert the preprocessed parameters into corresponding elements in the virtual space.

[0081] As one approach, based on initial context-aware information, the corresponding digital avatars (covering embodied intelligent agents and various objects in the environment) can be retrieved from the virtual object model library of the security sandbox, and a virtual operating environment can be built in the security sandbox using these digital avatars.

[0082] The high-fidelity physics simulation engine within the security sandbox has built-in, unchanging initial physics rules. These rules, as inherent attributes of the engine, must be initialized before each task execution. The dynamic state of the current environment within the security sandbox (such as object locations) can be retrieved using corresponding digital avatars from a virtual object model library, thus completing the construction of the virtual operating environment. This virtual object model library includes embodied intelligent agents and digital avatars of common objects in the environment, providing fundamental digital asset support for program code verification and action simulation.

[0083] S1032. In the virtual operating environment, control the virtual intelligent agent to simulate the execution of the target action sequence and obtain the simulation results.

[0084] The virtual intelligent agent is a digital clone of the embodied intelligent agent. First, the hardware structure, execution component parameters (such as the range of motion of the robotic arm joints and the upper limit of power output), sensor accuracy and other attributes of the embodied intelligent agent are mapped to the virtual space to generate a digital clone (i.e., the virtual intelligent agent) that is completely consistent with the embodied intelligent agent in terms of function and performance.

[0085] The target action sequence is then broken down into instruction formats recognizable by the virtual agent, clearly defining the virtual execution components and virtual action parameters (such as the displacement of the virtual robotic arm and the virtual power output value) corresponding to each action. Subsequently, the simulation execution process is initiated in the virtual operating environment, controlling the virtual agent to execute each virtual action in the order of the action sequence, and simultaneously recording key data during the simulation process (such as the position change of the virtual agent after executing the action, the state feedback of the virtual object, and whether there are any anomalies such as collisions or action failures). Finally, the simulation result is generated based on the recorded data. If all actions are completed as expected, no anomalies occur, and the task objective is achieved, the simulation result is considered successful; otherwise, it is considered a failure.

[0086] S1033. If the simulation result indicates successful execution, control the embodied agent to execute actions in the order of actions indicated in the target action sequence.

[0087] When the simulation in the virtual runtime environment shows successful execution, it indicates that the code logic of the target action sequence is error-free and there are no execution interruption issues caused by instruction conflicts, parameter errors, etc., ensuring that the instructions can be correctly parsed by the agent during actual execution. Furthermore, it indicates that the virtual agent did not engage in potentially dangerous actions such as colliding with obstacles, exceeding safe operating limits, or experiencing power overload during the simulation, demonstrating that the action sequence conforms to the safety constraints of the real environment. Based on these two verifications, it can be determined that the target action sequence meets the conditions for execution in the real environment. This allows the embodied agent to be controlled to perform actual execution in the real environment according to the action sequence instructions, thus avoiding code logic risks and reducing the probability of safety incidents.

[0088] As an alternative implementation, if the simulation results indicate execution failure, the simulation results are input into the instruction generation model. This allows the instruction generation model to learn from the failed simulation results and regenerate a new target action sequence.

[0089] When the simulation results in the virtual runtime environment indicate execution failure (such as a virtual agent experiencing a collision or failing to achieve the task target parameters), the complete simulation failure result (including the failed action step, the virtual environment state at the time of failure, and key parameter deviation data) needs to be fed back to the instruction generation model. Upon receiving the feedback, the instruction generation model will break down and analyze the failure result. It will locate the cause of failure from the failed simulation data (such as an excessively large displacement parameter in a certain action leading to a collision, or insufficient power output causing the target object to not move), and transform the environment-action correlation patterns corresponding to these failure causes into empirical data storage. Furthermore, based on accumulated experience and initial contextual awareness information, the instruction generation model needs to adjust the action planning logic (such as correcting deviations in action parameters and optimizing the action execution order) to regenerate a new target action sequence adapted to the current environment and task requirements. This new sequence can then be imported back into the virtual runtime environment for simulation verification until a target action sequence that can be successfully simulated is generated.

[0090] As can be seen from S1031-S1033, in this implementation, a virtual operating environment is first constructed based on the initial scenario, and the virtual agent is controlled to simulate the execution of the target action sequence. Real execution is then initiated after the simulation is successful. This not only verifies the code logic of the target action sequence in advance through virtual simulation, avoiding interruptions during real execution due to logical errors, but also predicts and eliminates potential dangerous actions such as collisions and power overloads, reducing the risk of the embodied agent damaging itself or the environment during execution in the real environment. Furthermore, it eliminates the need for repeated trial and error in the real environment, reducing ineffective execution costs. Ultimately, while ensuring safety and reliability, it improves the efficiency and success rate of completing the task.

[0091] As can be seen from S101-S103, the control method for the embodied intelligent agent provided in this application, after receiving the task to be executed, inputs the task to be executed and the initial situational awareness information of the embodied intelligent agent into a readable cognitive model. Since the initial situational awareness information can represent the environmental awareness data and self-state data of the embodied intelligent agent in the current environment, the readable cognitive model can fully grasp "what the current environment is like and what the intelligent agent's own state is". Combined with the causal knowledge in the historical interaction data, invalid rules that do not match the current situation can be eliminated, ensuring that the determined causal rules fit the actual execution scenario and providing a reliable basis for the subsequent generation of action sequences. Furthermore, the task to be executed, the initial situational awareness information, and the causal rules are input into the instruction generation model, enabling the instruction generation model to reason and plan based on the causal rules, thereby generating specific action steps that can realize the task to be executed in the current environment, and obtaining the target action sequence. Finally, the embodied intelligent agent is controlled to execute actions according to the action sequence indicated in the target action sequence to realize the task to be executed. In other words, the solution provided in this application first combines the causal knowledge learned by the readable cognitive model from multiple sets of historical interaction data to achieve contextualized causal understanding when determining the target action sequence. Then, the causal understanding is transformed into an executable target action sequence through the instruction generation model, enabling the intelligent agent to plan as if it had common sense related to the current scene, which significantly improves the accuracy and success rate of the embodied intelligent agent in completing tasks in complex and dynamic environments.

[0092] As one feasible implementation method of this application embodiment, please refer to Figure 4 The control method for embodied intelligent agents provided in this application embodiment further includes: S201. Determine multiple candidate actions that the embodied intelligent agent can perform.

[0093] As a feasible implementation method, the control device of the embodied intelligent agent can start executing S201-S205 when it is determined that the similarity between the current environment and the historical environment is lower than a preset threshold. The fact that the similarity between the current environment and the historical environment is lower than the preset threshold indicates that the embodied intelligent agent is in an unknown new environment. In order to improve the prediction model's ability in the current environment, it is necessary to actively explore the environment first and update the prediction model based on the exploration data.

[0094] It should be noted that when exploring an unknown new environment, the prediction model and the readable cognitive model can be updated simultaneously based on real-time environmental data collected during the exploration process (such as spatial layout details, object physical properties, dynamic change patterns, etc.). This allows the prediction model to more accurately predict situational changes after an action is executed by absorbing data such as the physical rules and object interaction characteristics of the new environment (such as accurately calculating the force and displacement of objects with new materials and the collision risk under a new spatial layout). The readable cognitive model, by integrating information such as the characteristics of target objects and environmental constraints in the new environment, can more clearly construct an environmental cognitive framework (such as clarifying the operational logic of new types of objects and sorting out the task execution priorities in a new scenario), thereby providing more suitable and reliable model support for the generation and execution of subsequent action sequences. For ease of description, this embodiment will first illustrate the example of updating only the prediction model.

[0095] When an embodied agent needs to explore an unknown new environment, it must first determine several candidate actions that can be executed under its current hardware capabilities and environmental constraints. These candidate actions are essentially atomic actions that the embodied agent can independently complete and cannot be further divided (such as "rotating a single joint of the robotic arm by 5°", "moving the chassis forward by 10cm", "initiating a single environmental scan by the sensor", etc.). When determining these candidate actions, it is necessary to screen out the basic actions that can be realized at the hardware level based on the hardware configuration of the embodied agent (such as the type of execution component, the range of power output, and the function of the sensors), and exclude actions that exceed the hardware capabilities (such as rotational actions outside the range of motion of the robotic arm joints). In addition, it is also necessary to refer to the preliminary perception information of the current unknown environment (such as spatial height restrictions, whether there are preliminarily identified obstacles), and eliminate actions that conflict with the initial constraints of the environment (such as excluding the atomic action of "fully raising the robotic arm" in a low space), and finally form several candidate actions that are adapted to the current hardware and the initial state of the environment.

[0096] As a feasible approach, candidate actions that an embodied agent can perform in the current environment can be determined using a readable cognitive model. Examples include [move_left, move_right, grasp, push, etc.]. A readable cognitive model can first access stored knowledge of the embodied agent's hardware capabilities to identify the basic action types the agent can independently perform at the hardware level. Furthermore, the readable cognitive model combines preliminary perception data of the current environment (such as spatial dimensions, object distribution, and environmental constraints) with cognitive reasoning to eliminate actions that conflict with the environment (e.g., eliminating actions involving large limb extensions in a narrow passage, or high-force contact actions around fragile objects). Finally, the readable cognitive model filters candidate actions from the atomic action library that both match the agent's execution capabilities and are suitable for the current environmental state, providing a precise set of actions for subsequent action selection.

[0097] S202. For each candidate action, predict the first context-aware information after the embodied agent has executed the candidate action using multiple prediction models.

[0098] For each candidate action, multiple prediction models are invoked to perform parallel predictions. Each prediction model calculates and outputs the first context-aware information that the embodied agent may acquire after executing the candidate action.

[0099] As a feasible implementation method, the prediction model is used to predict the first contextual awareness information of the embodied agent after it has executed a candidate action, based on the third contextual awareness information of the embodied agent in its current state. The prediction model, based on the embodied agent's current real-time state, enables the prediction results to closely match the dynamically changing actual environment, thereby effectively improving the accuracy, safety, and adaptability of subsequent action planning and avoiding planning failures caused by sudden environmental changes or lag in state information.

[0100] S203. Determine the intrinsic motivation corresponding to the candidate action by the deviation between the first context-aware information predicted by each of the multiple prediction models. The larger the deviation, the stronger the intrinsic motivation.

[0101] After obtaining the first context-aware information corresponding to a candidate action through m prediction models, it is necessary to calculate the deviation between these prediction results. Specifically, this can be done by comparing the key context parameters output by each model (such as the estimated position of the object after the action is performed, the estimated change in the environmental state, etc.), and using variance, standard deviation, or similarity algorithms to quantify the degree of difference between the results of different models; then, based on the judgment rule that "the larger the deviation, the stronger the intrinsic motivation", the calculated deviation value is converted into the intrinsic motivation corresponding to the candidate action.

[0102] If the predictions from multiple prediction models deviate significantly, it indicates that the consequences of the candidate action are highly uncertain in the current environment. Exploring such actions can acquire more information about the unknown environment to fill the cognitive gaps in the models, thus giving them a stronger intrinsic incentive. If the deviation is small, it indicates that the models' predictions of the consequences of the action are relatively consistent, the exploration value is relatively low, and the corresponding intrinsic incentive is weak. The intrinsic incentive determined in this way can serve as an important basis for prioritizing high-exploration-value actions from the candidate actions.

[0103] As a feasible approach, the prediction variance among the first context-aware information predicted by multiple prediction models can be calculated, and the prediction variance can be used as the bias.

[0104] By calculating the sum of squared deviations and the average of the deviations of the first context-aware information (such as the estimated position of objects and action feedback parameters) output by each model from the mean of all results, the differences in the prediction results of different models can be intuitively reflected, avoiding the problem of subjective judgment bias and providing an objective and quantifiable basis for the subsequent determination of intrinsic motivation. Secondly, the prediction variance is more sensitive to extreme deviation results. When the prediction result of a certain model differs significantly from that of other models (such as a certain model predicting the displacement of an object much greater than that of other models), the variance will increase significantly. This can quickly capture the "abnormal divergence" in multi-model prediction. Such divergence often means that there is information in the current environment that the model has not fully recognized. The corresponding candidate actions have higher exploration value, which is highly compatible with the judgment logic of "the greater the deviation, the stronger the intrinsic motivation". This can provide a clear direction for subsequent model optimization and further ensure the accuracy and efficiency of action selection in the exploration phase.

[0105] S204. Control the embodied intelligent agent to execute the target action, and determine the second context-aware information after the embodied intelligent agent has completed the target action.

[0106] The target action is a candidate action among multiple candidate actions whose intrinsic motivation meets preset conditions. The preset conditions are: the strongest intrinsic motivation, the intrinsic motivation is greater than the preset motivation threshold, and the top n intrinsic motivations.

[0107] After identifying multiple candidate actions and their corresponding intrinsic stimuli, the target action that meets preset conditions is first selected from the candidate actions. This application embodiment does not restrict the preset conditions; users can select them according to their needs. For example, if high exploration value is prioritized, the candidate action with the strongest intrinsic stimuli can be directly selected; if a balance between safety and exploration is required, candidate actions with intrinsic stimuli greater than a preset stimuli threshold can be selected (if there are multiple actions above the threshold, one is selected using additional rules such as "lowest difficulty in execution" or "least environmental interference"); if some selection flexibility is required, the top n candidate actions with the highest intrinsic stimuli (n is a preset positive integer greater than 1) can be selected first, and then, combined with the real-time state of the current environment (such as suddenly appearing dynamic obstacles or detailed information collected by sensors), one action best suited to the current environment can be further selected from these n highly stimuli actions as the final target action.

[0108] Subsequently, the embodied agent is controlled to perform the selected target actions in the current unknown new environment. After the actions are completed, environmental state change data is collected through the sensors (such as visual sensors, tactile sensors, position sensors, etc.) on the agent. Based on this data, a second contextual awareness information is formed after the embodied agent has completed the target actions. This information can be used to update the prediction model and the readable cognitive model, further improving the model's adaptability to the current environment.

[0109] S205. Based on the first context-aware information and the second context-aware information predicted by each prediction model, determine the prediction error corresponding to multiple prediction models.

[0110] After the embodied intelligent agent executes the target action and acquires the second context-aware information, for each prediction model, the previously output first context-aware information and second context-aware information are compared dimension by dimension. The prediction error corresponding to each prediction model is determined by quantitatively calculating the difference between the two. Specifically, key indicators with corresponding relationships between the two types of information (such as the actual position and predicted position of an object, the actual change value and predicted change value of environmental dynamic parameters, etc.) are selected, and a preset error calculation method (such as mean square error, mean absolute error, etc.) is used to calculate the deviation between the predicted value and the actual value under each indicator dimension. Then, the overall prediction error of the prediction model is obtained by combining the deviations of each dimension.

[0111] For example, a prediction model might estimate that an object will move to coordinates (x1, y1) after performing an action, while the second context-aware information shows the object's actual coordinates as (x2, y2). The prediction error for the "object position" metric can be obtained by calculating the sum of the squares of the distances or coordinate differences between the two points. This error can then be combined with other metric errors to comprehensively assess the model's prediction accuracy. This prediction error data serves as the core basis for subsequent optimization of the parameters of each prediction model, helping the model better adapt to the current unknown new environment.

[0112] It should be noted that when determining the prediction error for multiple prediction models, one of the following two paths can be used: Path 1: First, for each prediction model, compare the first context-aware information output by it with the second context-aware information obtained after the embodied agent performs the target action in a dimension-by-dimensional manner (such as comparing the predicted values ​​and actual values ​​of indicators such as spatial location, object state, and dynamic features). Calculate the individual error corresponding to each prediction model using a preset error algorithm (such as mean square error or mean absolute error). Then, based on the individual errors of all prediction models, determine the overall prediction error through statistical integration (such as calculating the mean, maximum value, or weighted sum of individual errors), or directly retain the individual errors of each model for subsequent targeted model updates.

[0113] Path 2: First, fuse the first context-aware information output by multiple prediction models (e.g., by weighting the average based on model confidence and taking the median of the predicted values ​​of each indicator) to determine a total prediction context information that comprehensively reflects the prediction results of multiple models; then, compare the total prediction context information with the second context-aware information in all dimensions, calculate the difference between the two through a preset error algorithm, and directly obtain the overall prediction error, which can be used for subsequent overall model optimization or exploration effect evaluation.

[0114] Two paths can be selected based on actual needs: Path 1 preserves the error details of a single model and is suitable for scenarios where each model is optimized individually; Path 2 focuses on the overall prediction effect and is suitable for scenarios where the accuracy of joint prediction by multiple models needs to be quickly evaluated. This application does not limit the specific implementation method for determining the prediction error in its embodiments.

[0115] For example, assuming the current time is t, and the observed context-aware information is o_t, the first encoding obtained after feature encoding is s_t=φ(o_t). Each candidate action a and the first encoding are then input into the prediction model, enabling the model to generate features for predicting the next state based on "internal imagination": .

[0116] By using multiple prediction models F1 / F2 / … / Fn, their prediction variances are calculated as intrinsic stimulus values, and the resulting target action is selected as a_t. The second context-aware information o_{t+1} after executing a_t is encoded as s_{t+1}=φ(o_{t+1}). The true error can then be calculated. .

[0117] S206. When the prediction error is greater than or equal to a preset error threshold, or the intrinsic excitation is greater than or equal to a preset excitation threshold, update multiple prediction models based on the prediction error.

[0118] After obtaining the prediction errors e corresponding to multiple prediction models, it is first determined whether the model update conditions are met. If the prediction error of any prediction model is greater than or equal to the preset error threshold, it indicates that the model's prediction accuracy for the current environment is insufficient, and the model parameters need to be adjusted based on its prediction error data (such as correcting the physical rule calculation coefficients and optimizing the environmental feature weights). If, when previously selecting target actions, the intrinsic stimulus of the action is greater than or equal to the preset stimulus threshold, it means that the environmental exploration value corresponding to the action is high, and the prediction error it generates (regardless of the error size) can provide the model with key new environment adaptation information. Similarly, the prediction model needs to be trained based on the prediction errors of all prediction models to specifically optimize the model's logic for capturing and predicting new environmental features.

[0119] As a feasible approach to reduce the computational cost of frequent model updates, a strategy of "centralized update after multiple explorations" can be adopted: the embodied agent executes multiple rounds of target actions (each round of action corresponds to steps such as acquiring candidate actions, multi-model prediction of first-context awareness information, and collecting second-context awareness information after action execution), continuously accumulating multiple exploration data. Each exploration data point contains the first-context awareness information of a single prediction model, the corresponding second-context awareness information, and the model's individual error. Once the number of explorations reaches a preset threshold (i.e., the accumulated exploration data meets the model update requirements), the model update process is then initiated uniformly, and multiple prediction models are centrally optimized.

[0120] When multiple prediction models are used, they can complement each other by leveraging their differences, thus reducing the risk of prediction bias from a single model. However, if the same training data is used during training, the models will become increasingly similar and unable to learn anything.

[0121] Therefore, as one implementation method, m exploration datasets are constructed from multiple exploration datasets, where m is the number of prediction models; then, a prediction model is trained using the exploration data in each exploration dataset.

[0122] Based on the multiple exploration data accumulated by the embodied agent in the unknown new environment, m independent exploration datasets (denoted as S1, S2, ..., Sm) are constructed according to the number of prediction models m. m Each exploration dataset is constructed by selecting samples from the total exploration data through a preset sampling strategy (such as random replacement sampling, stratified sampling, etc.), and the sample size of each dataset meets the training requirements of the corresponding prediction model. Then, the m exploration datasets are matched one-to-one with the m prediction models, and the corresponding prediction models are trained independently using the exploration data in each dataset to complete the initialization or iterative optimization of the model parameters.

[0123] As one approach, mean squared error can be used as the loss function when training the prediction model, and gradient descent can be used to minimize the loss function.

[0124] As can be seen from S201-S206, the solution provided in this embodiment quantifies the intrinsic incentive through the deviation of the prediction results from multiple prediction models. The larger the deviation, the stronger the incentive, ensuring that candidate actions that can fill the cognitive gaps of the model (such as actions with high prediction uncertainty) are preferentially selected as target actions. The exploration process of the intelligent agent is no longer a random and inefficient blind attempt, but an active exploration with a clear goal. It can autonomously identify and fill cognitive gaps, and systematically explore the boundaries of its own action space. Then, based on the actual situational awareness information after the target action is executed (second situational awareness information), the prediction error is calculated in combination with the prediction deviation, and the prediction model is updated based on the error. This allows the prediction model to continuously absorb new environment exploration data, gradually reduce the prediction deviation of the new environment, and provide more reliable prediction support for subsequent action planning.

[0125] In some embodiments, when an embodied agent is in an unknown new environment, it needs to actively explore that environment first. For example, it can collect information such as the layout of the environment, the distribution of objects, and spatial boundaries through its own sensors, or perform small-scale exploratory actions (such as lightly touching objects or moving a short distance) and record the feedback to obtain initial interaction data in that environment. This interaction data specific to the new environment can be used as the basis for subsequently refining causal rules. Only based on this data can the causal rules that must be followed when performing tasks in that environment be clearly defined (such as object weight limits, spatial traversability, etc.), providing a compliance basis for subsequent task execution.

[0126] As a feasible implementation method, after determining the second context-aware information after the embodied intelligent agent has performed the target action, the method provided in this application embodiment further includes: using the target action, the third context-aware information, and the second context-aware information as historical interaction data.

[0127] When updating the readable cognitive model synchronously, in order for the model to accurately learn the contextual change logic before and after the action (such as changes in environmental state, object interaction results, etc.), it is necessary to form a complete contextual link by combining the third contextual perception data before the action and the second contextual perception data after the action. This ensures that the readable cognitive model can summarize interpretable action-environment interaction rules based on the full amount of contextual data, thereby improving the integrity of the cognitive framework and the accuracy of reasoning.

[0128] As one approach, the target action, third-context perception information, and second-context perception information after the action is executed can be organized into historical interaction data in a preset structured format and stored in the data storage module that supports the readable cognitive model. This provides complete data support for the model to learn the action-context change logic and summarize interaction rules.

[0129] As a feasible approach, since environmental perception data (such as contextual images) collected during the operation of intelligent agents often exists in the form of unstructured images, it not only consumes storage resources but is also difficult to retrieve and utilize directly by the model. Therefore, this type of image-based perception data can be first input into a visual language model. Through visual semantic parsing and multimodal data alignment, the model transforms it into structured, interpretable text / symbol data, thereby forming historical interaction data that is easy to store, recognize, and reuse. This data is then stored in the data storage module of a readable storage model.

[0130] This application does not limit the historical interaction data stored in the readable cognitive model. Historical interaction data can be exploration data, task execution data, user interaction data, environmental feedback data, or predefined rule adaptation data; this application does not impose any restrictions on this. Task execution data refers to the interaction data generated by the embodied intelligent agent during the completion of a specific target task (such as object grasping or path planning), including task instructions, execution action sequences, contextual awareness data before and after task execution, and task success / failure feedback. User interaction data refers to the data generated by the embodied intelligent agent interacting with human users through commands, voice, gestures, etc., such as user-issued action commands, feedback on the agent's execution results, and descriptions of scene requirements. Environmental feedback data refers to the interaction data generated by changes in the environment or external interference when the embodied intelligent agent does not actively perform exploration / task actions (such as dynamic obstacle movement, changes in ambient light, changes in the natural state of objects), and contextual data passively perceived by the agent. Predefined rule adaptation data refers to interactive data that is pre-configured based on domain knowledge or historical experience and can be invoked by the agent without actual execution (such as physical rule parameters of the known environment, attribute data of common objects, safety constraint thresholds for action execution, etc.).

[0131] Using data collected during the exploration process as historical interaction data allows for updating the readable cognitive model while simultaneously training the predictive model. If the two models are updated asynchronously (e.g., the predictive model is updated first, followed by the readable cognitive model), the knowledge base / rules of the readable cognitive model will lag behind the optimization progress of the predictive model. For example, the predictive model may have corrected its prediction parameters for the "object friction coefficient" using new exploration data, but the readable cognitive model may still be selecting candidate actions based on old rules, potentially continuing to select actions already deemed "low-value" by the predictive model, resulting in exploration redundancy. During synchronous updates, exploration data is received by both models simultaneously: the predictive model updates its parameters to improve prediction accuracy, while the readable cognitive model synchronously updates its knowledge base and interaction rules. This ensures that subsequent candidate action selection (based on the readable cognitive model) and action consequence prediction (based on the predictive model) are based on the same batch of the latest exploration data, matching decision logic and prediction capabilities in real time, avoiding ineffective exploration due to information asynchrony.

[0132] In other words, the interactive data generated during the exploration process in S201-S206 is stored in a readable cognitive model, which enables the readable cognitive model to learn the core features of the environment, the interaction logic between actions and the environment, and the boundaries of its own action space during exploration. This allows for the gradual construction of an interpretable cognitive framework that adapts to the current environment, providing more accurate cognitive support for subsequent candidate action selection and exploration strategy optimization.

[0133] In some embodiments, because the environment in which the agent exists may be dynamically changing, such as adjustments to the environment layout, changes in the state of objects, or the intervention of external interference, the causal rules learned in the initial stage may have limitations or local applicability. Therefore, as the embodied agent is used continuously, it also needs to have the ability to learn continuously, providing accurate and comprehensive cognitive support for subsequent target action selection and situation prediction, and avoiding inefficient exploration and decision-making errors due to lagging or incomplete rules.

[0134] As a feasible implementation method, after S103, the method provided in this application embodiment further includes: S104. Obtain the execution results and target context awareness information after the embodied intelligent agent executes the target action sequence.

[0135] Obtain relevant feedback data after the embodied intelligent agent executes the target action sequence, which may include execution results and target context awareness information.

[0136] The execution results may include: the completion degree of the target action sequence (such as whether it is executed according to preset parameters, whether the action connection is smooth), the success / failure indicators at the task level (such as whether the exploration goal is achieved, whether the expected environmental interaction effect is achieved), and the real-time feedback status of the environment (such as whether a collision occurs, whether the object state changes as expected, etc.).

[0137] Target context awareness information refers to the overall state data of the current environment (such as spatial layout, object positions and attributes, and dynamic environmental features) collected by sensors (such as cameras, radar, and force feedback sensors) on the intelligent agent after the action sequence is completed. This data is used to fully depict the final state of the environment after the action sequence is executed.

[0138] S105. Use the initial context-aware information, target action sequence, target context-aware information, and execution results as historical interaction data.

[0139] The initial context-aware information collected before executing the target action sequence, the target action sequence itself, the target context-aware information, and the execution results are integrated and organized into unified historical interaction data according to a preset structured format (such as field standardization, temporal labeling, and explicit association). This data is then stored in the data storage module of the readable cognitive model. This allows the readable storage model to learn new causal knowledge from the execution of the current task, such as the selection criteria for target actions in different task scenarios, temporal arrangement rules, the impact of specific environmental preconditions on task achievement rate, the causal mapping relationship between action sequences and target context-aware information, such as key influencing factors when the task fails and core constraints for successful execution. This allows for continuous iteration of the cognitive rule base in the model, improving the accuracy of subsequent task planning and environmental adaptability.

[0140] In this implementation, the data during the execution of the task to be executed is used as historical interaction data. This allows the readable cognitive model to simultaneously absorb multi-dimensional causal knowledge from the complete execution of the task, including task adaptation logic, the relationship between context and result, the causal mapping between action and environment, and the direction of execution feedback optimization. This enables continuous iteration of the cognitive rule base, significantly improving the accuracy of subsequent task planning and environmental adaptability. At the same time, it enables real-time accumulation and reuse of cognitive experience, avoiding repeated exploration and improving the efficiency and robustness of the agent's task execution.

[0141] As a feasible implementation method, the method provided in this application embodiment further includes: when the task execution result of the task to be executed is execution failure, taking the target action sequence, the task to be executed and the initial context-aware information as negative samples, and updating the instruction generation model based on the negative samples.

[0142] When an instruction generation model is trained solely on positive samples (successfully executed task data), it can only learn "which action sequences are effective in a specific context," but cannot identify "which action sequences will cause task failure in that context." By using the triple of the failure scenario (target action sequence + task + initial context) as negative sample input, the instruction generation model can explicitly learn "in the initial context of XX, generating the XX action sequence for the XX task will lead to failure," forming a two-way understanding of "effectiveness and ineffectiveness." Subsequent instruction generation can directly avoid such erroneous action sequences, significantly reducing the probability of repeatedly executing failed actions.

[0143] The root cause of task execution failure is often that "the instruction generation model has a deviation in matching the task requirements, contextual features, and action sequences" (e.g., in the task of "grabbing an object in a narrow space," the generated action sequence of "raising the robotic arm at a large angle" leads to collision failure). Using the triple of this failed scenario as a negative sample, the model can reverse-engineer the deviation and then adjust the generation rules to make the instruction generation logic more consistent with the actual causal rules.

[0144] As one implementation of this application, the control method for embodied intelligent agents provided in this application embodiment may include two stages. The first stage is used to explore the embodied intelligent agent and construct a readable cognitive model, while the second stage is used for skill synthesis and execution.

[0145] The first stage may include the following steps: S21. Determine the current context-aware information.

[0146] By using sensors (such as cameras, radar, force feedback modules, etc.) mounted on the embodied intelligent agent, real-time data on the current environment in all dimensions is collected, including spatial layout, object positions and attributes, dynamic interference factors, etc., forming structured current situation perception information, providing an objective and accurate environmental input basis for the incentive calculation of subsequent candidate actions.

[0147] S22. Calculate the intrinsic motivation for each candidate action using a prediction model.

[0148] Based on current contextual awareness information, multiple prediction models are invoked to predict all candidate actions in parallel. By quantifying the bias of each model's prediction results (such as the uncertainty of context changes after action execution), the bias is transformed into intrinsic motivation to measure the value of action exploration. The greater the bias, the more cognitive gaps the action can fill, and the higher the corresponding intrinsic motivation.

[0149] S23. Select the candidate action with the greatest intrinsic motivation as the target action.

[0150] Based on the intrinsic incentive calculation results of each candidate action, the action with the highest incentive value is selected as the target action. This ensures that the agent prioritizes the exploration behavior with the highest information gain, avoids indiscriminate random exploration, and improves the targeting and efficiency of the exploration process.

[0151] S24. Execute the target action and determine the second context-aware information after the target action is completed.

[0152] The agent executes the selected target action according to preset parameters. After the action is completed, it collects environmental state data again through sensors to form a second context perception information, which fully records the actual impact of the target action on the environment and provides "action-result" related data for subsequent data storage and model updates.

[0153] S25. Generate historical interaction data and store it in a readable cognitive model.

[0154] The "current context-aware information (action preconditions), target action (execution behavior), and second context-aware information (action postconditions)" are integrated into historical interaction data in a structured format and stored in the supporting data storage module of the readable cognitive model, providing materials for the model to continuously learn action-environment interaction rules and build a cognitive framework.

[0155] S26. Update the prediction model.

[0156] The prediction error (i.e., the difference between the predicted value of the target action execution result by the prediction model and the actual value of the second context perception information) is calculated based on "current context perception information, target action, and second context perception information". This error is used to iteratively optimize the parameters of the prediction model, reduce the bias of subsequent action predictions, and improve prediction accuracy.

[0157] Repeat steps S21-S26 until the preset termination condition is met: the intrinsic excitation value of each candidate action tends to stabilize (convergence), indicating that the agent has fully explored the action space boundary and filled the core cognitive gaps. If the information gain of continuing to explore is lower than the threshold, the exploration stops and the cognitive construction of the unknown environment is completed.

[0158] The second stage may include the following steps: S31. Receive tasks to be executed.

[0159] The task to be performed can be a voice command described in natural language (such as "grab the red cube on the left side of the desktop"), or a text command, a graphical command, or a preset task identifier. This application embodiment does not limit the specific input form of the task, as long as it ensures that the task requirements can be accurately parsed by the intelligent agent to provide a clear target orientation for the generation of subsequent action sequences.

[0160] S32. Determine causal rules through readable cognitive models.

[0161] The system invokes a readable cognitive model and, based on historical interaction data stored within the model, retrieves and extracts causal rules that match the current task to be performed and potential environmental constraints. These rules are interpretable cognitive results accumulated by the model through continuous learning, such as "performing a 'precise grasp' action in an unobstructed environment can cause the target object to detach from its original placement surface" and "in a narrow space, the robotic arm's posture must be adjusted before performing the extension and retraction action to avoid collisions," providing a logical basis for generating the target action sequence.

[0162] S33. Generate the target action sequence and control the embodied agent to execute the target action sequence. Combining the core requirements of the task to be executed with the causal rules determined by S32, a target action sequence with temporal logic and clear parameters is generated (such as "move to the front of the target object → adjust the height of the robotic arm to the appropriate position → start the gripping device → raise the robotic arm by 5cm"), ensuring that the action sequence is compatible with the task objective, causal rules and the hardware capabilities of the intelligent agent; then, execution instructions are issued to the embodied intelligent agent to control it to execute step by step according to the preset action sequence, realizing the implementation of the task from instruction to actual operation.

[0163] S34. Store the executed data as historical interaction data in a readable cognitive model. After the target action sequence is completed, collect the full-link data of this task execution, including the initial context awareness information before execution, the target action sequence, the final context awareness information after execution, and the task execution results (success / failure indicators, environmental feedback, etc.), and integrate them into historical interaction data in a structured format; store this data in the supporting data storage module of the readable cognitive model, so that the model can absorb the execution experience of this task, iteratively update the internal causal rule base, and provide more accurate cognitive support for the planning and execution of similar tasks in the future.

[0164] It should be understood that the specific implementation methods of steps S21-S26 and S31-S34 can be referred to the above embodiments, and will not be repeated here.

[0165] As can be seen, the solution provided in this application enables an embodied intelligent agent to understand its own capabilities and the physical laws of the environment through short-term autonomous exploration, without any pre-programmed skills or prior task training, and to dynamically synthesize action sequences to complete complex tasks. This solves the core contradiction of traditional methods, which rely on a fixed skill set and cannot adapt to unknown environments and new tasks, achieving out-of-the-box universal adaptability.

[0166] In an exemplary embodiment, this application also provides a control device for an embodied intelligent agent. This control device can be a computing device that executes the aforementioned control method for an embodied intelligent agent, or it can be a processor within a computing device. The control device for an embodied intelligent agent may include one or more functional modules for implementing the control method for an embodied intelligent agent as described in the above method embodiments.

[0167] Figure 5 This is a schematic diagram of the architecture of a control device for an embodied intelligent agent provided in an embodiment of this application. Figure 5As shown, the control device 12 of the embodied intelligent agent includes: one or more memories 120, one or more processors 110, a communication bus 140, and a communication interface 130. The processors 110 and memories 120 are connected via the communication bus 140; the one or more memories 120 are used to store computer program code, which includes computer instructions; when the one or more processors 110 execute the computer instructions, the control device 12 of the embodied intelligent agent performs the control method of the embodied intelligent agent provided in this embodiment.

[0168] Optionally, the memory 120 may be a non-transitory computer-readable storage medium, such as a read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc. The embodiments of this application do not impose any limitations on this.

[0169] The processor 110 may be a central processing unit (CPU), a network processor (NP), a digital signal processor (DSP), a microprocessor, a microcontroller, a programmable logic device (PLD), or any combination thereof, and the embodiments of this application do not impose any limitations on this.

[0170] The communication bus 140 can be an industry standard architecture (ISA) bus, a peripheral component interconnect (PCI) bus, or an extended industry standard architecture (EISA) bus, etc. This communication bus 140 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 It is represented by a single thick line, but this does not mean that there is only one bus or one type of communication bus.

[0171] Communication interface 130 uses any transceiver-like device for communicating with other devices or communication networks, such as control systems, radio access networks (RAN), wireless local area networks (WLAN), etc.

[0172] In an exemplary embodiment, this application also provides a computing device, which may include a processor and a memory. The processor may be a computing cluster composed of multiple computing nodes, and the memory may adopt a distributed memory architecture. The processor integrated into the computing device is configured to execute the control method of the embodied intelligent agent according to any of the above embodiments.

[0173] It should be noted that the system architecture and application scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0174] This application also provides a computer-readable storage medium. All or part of the processes in the above method embodiments can be executed by computer instructions instructing related hardware; for example, the related hardware can be a processor of a computing device. The program instructions can be stored in the above-mentioned computer-readable storage medium, and when executed, the processes of the above method embodiments can be implemented. The computer-readable storage medium can be memory. The above-mentioned computer-readable storage medium can also be an external storage device, such as a hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. Further, the above-mentioned computer-readable storage medium can include both memory and external storage devices. The above-mentioned computer-readable storage medium is used to store the above-mentioned computer program instructions and other programs and data required by the above-mentioned control method of the embodied intelligent agent.

[0175] This application also provides a computer program product, which includes a computer program that, when run on a computing device, causes the computing device to execute any of the embodied intelligent agent control methods provided in the above embodiments.

[0176] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, disclosure, and appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0177] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

[0178] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A control method for an embodied intelligent agent, characterized in that, include: The initial context-aware information of the task to be performed and the embodied intelligent agent is input into a readable cognitive model, and the causal rules corresponding to the task to be performed are determined through the readable cognitive model. The initial context-aware information includes the embodied agent's environmental perception data and its own state data in the current environment; the readable cognitive model is used to determine the causal rules that need to be followed when performing the task to be performed in the current environment based on causal knowledge learned from multiple historical interaction data; the historical interaction data includes: the historical actions performed by the embodied agent, the context change information of the context-aware information when performing the historical actions, and the task execution results; The task to be executed, the initial context-aware information, and the causal rules are input into the instruction generation model, and the target action sequence is determined by the instruction generation model. The instruction generation model is used to determine the action sequence that can complete the task to be executed in the current environment in a manner that conforms to the causal rules. Control the embodied intelligent agent to perform actions in the order of actions indicated in the target action sequence.

2. The control method according to claim 1, characterized in that, The instruction generation model is specifically used for: In accordance with the causal rules, the first action required to complete the task to be performed in the current environment is determined based on the initial context-aware information of the embodied intelligent agent. Based on the estimated contextual awareness information after the embodied agent completes the (n-1)th action, the nth action required to complete the task to be performed is determined, until the obtained action sequence can complete the task to be performed; the action sequence includes the first action and the nth action, where n is an integer greater than 1; The estimated context-aware information is determined by a prediction model; the prediction model is used to estimate the context-aware information after the current action is completed based on the context-aware information of the embodied agent before the current action is executed.

3. The control method according to claim 2, characterized in that, The estimated context-aware information is determined through multiple prediction models; The method further includes: Identify multiple candidate actions that the embodied intelligent agent can perform; For each candidate action, the first context-aware information after the embodied agent completes the candidate action is predicted by the multiple prediction models; The intrinsic motivation corresponding to the candidate action is determined by the deviation between the first context-aware information predicted by each of the plurality of prediction models; wherein, the larger the deviation, the stronger the intrinsic motivation. The system controls the embodied intelligent agent to perform a target action and determines the second contextual awareness information after the embodied intelligent agent completes the target action; the target action is a candidate action among the plurality of candidate actions whose intrinsic motivation meets preset conditions; the preset conditions are: the intrinsic motivation is the strongest, the intrinsic motivation is greater than a preset motivation threshold, and the intrinsic motivation is among the top n highest. Based on the first context-aware information and the second context-aware information predicted by each prediction model, the prediction error corresponding to the plurality of prediction models is determined. If the prediction error is greater than or equal to a preset error threshold, or if the intrinsic excitation is greater than or equal to a preset excitation threshold, the plurality of prediction models are updated based on the prediction error.

4. The control method according to claim 3, characterized in that, The prediction model is used to predict the first context perception information after the embodied agent has completed the candidate action, based on the third context perception information of the embodied agent in the current state. After determining the second context-aware information after the embodied intelligent agent has performed the target action, the control method further includes: The target action, the third context-aware information, and the second context-aware information are used as the historical interaction data.

5. The control method according to any one of claims 2-4, characterized in that, The control method further includes: Determine whether the estimated context-aware information of the embodied intelligent agent after each action is performed satisfies the task objective of the task to be performed. If the task objective of the task to be executed is determined to be met, then the obtained sequence of actions is determined to be able to complete the task to be executed.

6. The control method according to any one of claims 1-5, characterized in that, After controlling the embodied intelligent agent to execute actions in the order of actions indicated in the target action sequence, the control method further includes: Obtain the execution result and target context awareness information after the embodied intelligent agent executes the target action sequence; The initial context-aware information, the target action sequence, the target context-aware information, and the execution result are used as the historical interaction data.

7. The control method according to claim 6, characterized in that, The control method further includes: If the task to be executed fails, the target action sequence, the task to be executed, and the initial context-aware information are used as negative samples, and the instruction generation model is updated based on the negative samples.

8. The control method according to any one of claims 1-7, characterized in that, The control of the embodied intelligent agent to execute actions according to the action sequence indicated in the target action sequence includes: A virtual operating environment is constructed based on the initial context-aware information; wherein, the virtual operating environment is used to simulate the current environment; In the virtual operating environment, the virtual intelligent agent is controlled to simulate and execute according to the target action sequence to obtain the simulation result; the virtual intelligent agent is a digital clone of the embodied intelligent agent. If the simulation result indicates successful execution, control the embodied agent to execute actions in the order of actions indicated in the target action sequence.

9. The control method according to claim 8, characterized in that, The control method further includes: If the simulation result indicates execution failure, the simulation result is input into the instruction generation model, and the instruction generation model redetermines a target action sequence based on the simulation result.

10. A computing device, characterized in that, The computing device includes a processor and a memory; the processor is coupled to the memory. The memory is used to store computer instructions; The computer instructions are loaded and executed by the processor to enable the computing device to implement the control method as described in any one of claims 1-9.

Images