Control methods, devices, equipment and storage media for intelligent agents

By constructing a two-layer architecture that separates the basic capability layer and the user instruction control layer, and combining environmental features and user intentions to generate control signals, the problem of uncontrollable agent behavior in the end-to-end model is solved, and the stability and controllability of agent actions are improved.

CN120669576BActive Publication Date: 2026-11-17PAXINI TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202510656786.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-05-21
Publication Date
2026-11-17
Estimated Expiration
2045-05-21

AI Technical Summary

Technical Problem

Existing end-to-end deep learning models lack explicit constraints in the process of agent action generation, resulting in poor output stability and uncontrollable agent behavior.

Method used

A two-layer architecture is constructed, separating the basic capability layer and the user instruction control layer. The first action sequence is generated through environmental feature data, and control signals are generated by combining user instruction intent information to control the first action sequence in order to improve the stability of the agent's actions.

Benefits of technology

In the process of generating actions for the intelligent agent, both environmental constraints and user intent are taken into account, and action generation is explicitly constrained, which enhances the controllability and stability of the agent's behavior.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120669576B_ABST
    Figure CN120669576B_ABST
Patent Text Reader

Abstract

This application provides a control method, apparatus, device, and storage medium for an intelligent agent. The method includes: generating a first action sequence for the intelligent agent based on first environmental feature data of the environment in which the intelligent agent is located; responding to a received user instruction, parsing the user instruction to generate instruction intent information; generating a control signal based on the instruction intent information, the first environmental feature data, and the first action sequence; controlling the first action sequence based on the control signal to generate a second action sequence for the intelligent agent; and controlling the intelligent agent to perform actions based on the second action sequence. Thus, in the process of generating the intelligent agent's actions, not only are environmental constraints on the intelligent agent's actions considered, but also the explicit constraints of user intent on the intelligent agent's actions are integrated, improving the stability of the intelligent agent's actions and thereby enhancing the controllability of the intelligent agent's behavior.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of intelligent agent technology, and in particular to a control method, device, equipment and storage medium for an intelligent agent. Background Technology

[0002] With the rapid development of artificial intelligence (AI) technology in fields such as robot control, autonomous driving, and human-computer interaction, intelligent agents such as robots, robotic arms, end effectors, intelligent vehicle terminals, and human-computer interaction systems are becoming increasingly widespread. In related technologies, to achieve the core goal of AI technology—intelligent decision-making and behavior generation of agents in dynamic environments—end-to-end deep learning models are typically relied upon. These models are trained on large-scale datasets to directly map environmental inputs to action outputs. However, because end-to-end models rely on black-box mapping and lack explicit constraints on the action generation process, the model output stability is poor, making the agent prone to uncontrollable behavior. Summary of the Invention

[0003] This application proposes a control method, device, equipment, and storage medium for an intelligent agent. This method is beneficial because, during the generation of the intelligent agent's actions, it not only considers the constraints imposed by the environment on the intelligent agent's actions but also integrates the explicit constraints imposed by the user's intentions on the intelligent agent's actions, thereby improving the stability of the intelligent agent's actions and enhancing the controllability of the intelligent agent's behavior.

[0004] In a first aspect, embodiments of this application provide a control method for an intelligent agent, the method comprising:

[0005] Based on the first environmental feature data of the environment in which the agent is located, a first action sequence of the agent is generated.

[0006] In response to a received user instruction, the user instruction is parsed to generate instruction intent information;

[0007] Based on the instruction intent information, the first environmental feature data, and the first action sequence, a control signal is generated;

[0008] The first action sequence is modulated based on the modulation signal to generate the second action sequence of the agent;

[0009] Based on the second action sequence, the agent is controlled to perform actions.

[0010] Secondly, embodiments of this application provide a control device for an intelligent agent, comprising:

[0011] An action sequence generation module is used to generate a first action sequence of the agent based on first environmental feature data of the environment in which the agent is located;

[0012] The intent parsing module is used to respond to a received user instruction, parse the user instruction into intent, and generate instruction intent information.

[0013] The control signal generation module is used to generate a control signal based on the instruction intent information, the first environmental feature data, and the first action sequence;

[0014] The control module is used to control the first action sequence based on the control signal to generate the second action sequence of the intelligent agent;

[0015] An action execution module is used to control the agent to perform actions based on the second action sequence.

[0016] Thirdly, embodiments of this application provide an intelligent agent, including: one or more processors; and a memory storing one or more programs thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the control method of the intelligent agent as described in the first aspect.

[0017] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the control method for an intelligent agent as described in the first aspect.

[0018] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the control method for an intelligent agent as described in the first aspect.

[0019] In this embodiment, a first action sequence of the agent is generated based on first environmental feature data of the agent's environment. In response to a received user instruction, the user instruction is parsed to generate instruction intent information. Based on the instruction intent information, the first environmental feature data, and the first action sequence, a control signal is generated. The first action sequence is then controlled using the control signal to generate a second action sequence of the agent. Finally, the agent is controlled to execute actions based on the second action sequence. Thus, by constructing a two-layer architecture separating the basic capability layer and the user instruction control layer—the basic capability layer generates the first action sequence based on the first environmental feature data, while the user instruction control layer parses user intent information from user instructions and generates control signals based on the user intent information, the first environmental feature data, and the first action sequence, and controls the first action sequence using these control signals—the generation of the agent's actions not only considers environmental constraints on the agent's actions but also integrates explicit constraints imposed by user intent, thereby improving the stability of the agent's actions and enhancing the controllability of the agent's behavior. Attached Figure Description

[0020] Figure 1A flowchart illustrating an embodiment of the control method for an intelligent agent provided in this application;

[0021] Figure 2 Another schematic flowchart of an embodiment of the control method for the intelligent agent provided in this application;

[0022] Figure 3 Another schematic flowchart of an embodiment of the control method for the intelligent agent provided in this application;

[0023] Figure 4 Another schematic flowchart of an embodiment of the control method for the intelligent agent provided in this application;

[0024] Figure 5 Another schematic flowchart of an embodiment of the control method for the intelligent agent provided in this application;

[0025] Figure 6 Another schematic flowchart of an embodiment of the control method for the intelligent agent provided in this application;

[0026] Figure 7 Another schematic flowchart of an embodiment of the control method for the intelligent agent provided in this application;

[0027] Figure 8 Another schematic flowchart of an embodiment of the control method for the intelligent agent provided in this application;

[0028] Figure 9 Another schematic flowchart of an embodiment of the control method for the intelligent agent provided in this application;

[0029] Figure 10 A flowchart illustrating an embodiment of the control device for an intelligent agent provided in this application;

[0030] Figure 11 A schematic diagram of the structure of an embodiment of the intelligent agent provided in this application. Detailed Implementation

[0031] To enable those skilled in the art to better understand the technical solutions of this application, the technical solutions provided in this application will be described in detail below with reference to the accompanying drawings.

[0032] Exemplary embodiments will be described more fully below with reference to the accompanying drawings; however, the described exemplary embodiments may be embodied in different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this application will be thorough and complete, and will enable those skilled in the art to fully understand the scope of this application.

[0033] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0034] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of a feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded.

[0035] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0036] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this application, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined in the embodiments of this application.

[0037] In related technologies, to achieve the core goal of artificial intelligence technology, namely to realize intelligent decision-making and behavior generation of intelligent agents (such as robots, robotic arms, end effectors, intelligent vehicle terminals, and human-computer interaction systems) in dynamic environments, it is usually necessary to rely on end-to-end deep learning models. The models are trained through large-scale datasets to directly map environmental inputs to action outputs. For example, some autonomous driving systems use multimodal sensor fusion and deep neural networks to generate driving decisions.

[0038] However, because end-to-end models rely on black-box mapping and lack explicit constraints on the action generation process, the model output stability is poor, making the agent prone to uncontrollable behavior. For example, reinforcement learning-based robots, robotic arms, and end effector grasping strategies may suddenly generate dangerous actions (such as high-speed collisions with objects) under environmental disturbances, leading to safety hazards.

[0039] Based on this, embodiments of this application provide a control method, apparatus, device, and storage medium for an intelligent agent. The method generates a first action sequence for the intelligent agent based on first environmental feature data of the environment in which the agent exists; responds to a received user instruction by parsing the user instruction to generate instruction intent information; generates a control signal based on the instruction intent information, the first environmental feature data, and the first action sequence; controls the first action sequence based on the control signal to generate a second action sequence for the intelligent agent; and controls the intelligent agent to perform actions based on the second action sequence. Thus, by constructing a two-layer architecture that separates a basic capability layer and a user instruction control layer—that is, the basic capability layer generates the first action sequence based on the first environmental feature data, while the user instruction control layer parses user intent information from user instructions and generates a control signal based on the user intent information, the first environmental feature data, and the first action sequence, and controls the first action sequence using the control signal—the generation of the intelligent agent's actions not only considers the constraints of the environment on the intelligent agent's actions but also integrates the explicit constraints of the user intent on the intelligent agent's actions, improving the stability of the intelligent agent's actions and thus enhancing the controllability of the intelligent agent's behavior.

[0040] The embodiments of this application will be further described below with reference to the accompanying drawings.

[0041] Please see Figure 1 This is a flowchart illustrating a control method for an intelligent agent provided in an embodiment of this application. The intelligent agent can be any electronic device based on artificial intelligence technology, including robots, robotic arms, end effectors, intelligent vehicle terminals, or human-computer interaction systems, etc. Figure 1 As shown, the control method of the intelligent agent includes the following steps S101 to S105.

[0042] Step S101: Generate the first action sequence of the agent based on the first environmental feature data of the environment in which the agent is located.

[0043] Before step S101 above, the intelligent agent can perceive the environment in which it is located through an environmental sensing device, perceive the sensing data that can characterize the environmental state of the environment in which the intelligent agent is located, and form the first environmental feature data based on the sensing data, so as to realize the construction of the environmental cognition foundation of the intelligent agent.

[0044] The aforementioned environmental sensing device can be any device or equipment capable of sensing the environmental state used to characterize the environment in which the intelligent agent is located. Specifically, the environmental sensing device can be a multi-source sensing device, and the multi-source sensing device can display the environmental state of the environment in which the intelligent agent is located through multiple aspects such as visual perception, tactile perception, and proprioception. That is, the multi-source sensing device can form multimodal sensing data by sensing the environment in which the intelligent agent is located through stereo vision units, tactile sensor arrays, and proprioception units.

[0045] It should be noted that the aforementioned environmental sensing device can be installed within the intelligent body, meaning that the environmental sensing device is a component of the intelligent body, in which case the intelligent body directly collects sensing data through its environmental sensing device; or, the environmental sensing device can also be a device independently installed outside the intelligent body, in which case the environmental sensing device senses the sensing data and transmits it to the intelligent body in real time, without any limitation.

[0046] The aforementioned sensing data can be any data that characterizes the environmental state of the environment in which the intelligent agent is located. It can include the intelligent agent's own data, data of other objects or entities in the environment, and data of the environmental space, etc. Specifically, when the aforementioned environmental sensing device is a multi-source sensing device, the aforementioned multimodal sensing data can include multi-dimensional information such as the intelligent agent itself, the three-dimensional spatial pose (position coordinates and attitude quaternions or other mathematical expressions) of the target object, surface physical properties (friction coefficient, material stiffness), and environmental constraints (obstacle distance, operating space boundary).

[0047] The formation of the first environmental feature data based on the aforementioned sensing data can be achieved by directly using the sensing data as the first environmental feature data; or, the agent can preprocess the sensing data to obtain the first environmental feature data. For example, the agent can perform at least one of the following processing on the received environmental data: timestamp alignment, noise filtering, and dimensionality reduction (more specifically, timestamp alignment, noise filtering, and dimensionality reduction on the received environmental data) to form the first environmental feature data.

[0048] In some embodiments, before step S101 above, the following may also be included:

[0049] Acquire multimodal sensor data of the environment in which the intelligent agent is located;

[0050] Multimodal sensing data is input into a second encoder. The second encoder extracts redundant information from the multimodal sensing data based on multi-layer nonlinear transformation to generate first environmental feature data. The data dimension of the first environmental feature data is lower than that of the multimodal sensing data.

[0051] In this embodiment, the second encoder can extract redundant information from multimodal sensing data based on multi-layer nonlinear transformation, thereby converting high-dimensional multimodal sensing data into low-dimensional first environmental feature data, achieving data dimensionality reduction, thereby reducing the complexity of data processing in intelligent agent behavior control, and thus improving the response speed of intelligent agent behavior control.

[0052] The second encoder mentioned above can be any encoder capable of extracting redundant information from high-dimensional data based on multi-layer nonlinearity, such as a deep neural network.

[0053] To ensure the effectiveness of the second encoder in data dimensionality reduction, before inputting the aforementioned multimodal sensor data into the second encoder, the initial encoder can be trained based on preset training data to obtain the second encoder. This training data can consist solely of sample data generated under a simulation environment.

[0054] Alternatively, in some implementations, before inputting the multimodal sensing data into the second encoder, the method further includes:

[0055] Obtain the first training sample set generated in the simulation environment;

[0056] The initial encoder is trained based on the first training sample set to obtain the encoder to be transferred.

[0057] Obtain the second training sample set generated in a real-world environment;

[0058] The second encoder is obtained by training the encoder to be transferred based on the second training sample set.

[0059] In this embodiment, after training the encoder to be transferred using sample data in a simulated environment, the agent can further train the encoder to be transferred using real sample data, thereby achieving fine-tuning of the encoder to be transferred and enabling the trained second encoder to have the ability to adapt to the actual physical environment.

[0060] The sample data in the aforementioned simulation environment refers to data generated within a simulated environment. This data is based on the simulation and modeling of real-world environments. For example, in autonomous driving systems, the simulation environment can simulate different traffic conditions, weather conditions, and road types, generating a large amount of driving scenario data. This data includes vehicle position, speed, and information about surrounding obstacles, which is used to train autonomous driving models so that they can make correct driving decisions in the simulated environment.

[0061] The aforementioned sample data in real-world environments refers to data collected in actual, unsimulated environments. This data originates directly from the real world and possesses higher realism and complexity. For example, in smart home systems, sample data in real-world environments might include user operation records of devices and data from environmental sensors (such as temperature and humidity). This data reflects user behavior patterns and environmental changes in real life, and is used to train models to better adapt to real-world usage scenarios.

[0062] It should be noted that the above-mentioned preprocessing of the sensing data by the second encoder can be performed on all the data in the sensing data; or, it can be performed on only a portion of the data in the sensing data.

[0063] In some embodiments, the second encoder described above is configured with constraint information of at least one environmental element; the multimodal sensing data includes: first feature data of at least one environmental element, and second feature data of other environmental elements besides the at least one environmental element;

[0064] The second encoder extracts redundant information from the multimodal sensing data based on multi-layer nonlinear transformation to generate first environmental feature data, including:

[0065] By using the second encoder and constraint information, redundant information in the second feature data is extracted through multi-layer nonlinear transformation to obtain the third feature data;

[0066] Generate first environmental feature data, which includes first feature data and third feature data.

[0067] In this embodiment, by pre-configuring constraint information in the second encoder, the second encoder can extract redundant information only for the second feature data other than the constraint information during the dimensionality reduction of multimodal sensing data. This can achieve both dimensionality reduction of the data and preservation of the integrity of the required information, thereby improving the data processing efficiency in intelligent agent behavior control while ensuring the execution accuracy of intelligent agent behavior control.

[0068] The aforementioned constraint information may include at least one pre-specified environmental element that is associated with the environmental state of the agent's environment. Specifically, the aforementioned constraint information may include environmental elements that have a decisive influence on behavioral planning in the agent's execution of actions.

[0069] The intelligent agent's execution behavior can include any behavior that the intelligent agent can perform. For example, in the grasping action control of intelligent agents such as robots, robotic arms, or end effectors, the constraint information can include key environmental elements such as the friction characteristics of the contact surface and the spatial distribution of obstacles. This makes the first environmental feature data include key parameters such as refined environmental spatial information, target object attributes, and operational constraints. This not only reduces the amount of data in the sensing data and significantly reduces the computational load of subsequent modules, but also improves the adaptability of intelligent agents such as robots, robotic arms, or end effectors to complex working conditions such as changes in lighting and partial occlusion during grasping actions.

[0070] In step S101 above, after the agent obtains the first environmental feature data, the agent can input the first environmental feature data into the preset action engine, and the preset action engine outputs the first action sequence.

[0071] The aforementioned first action sequence may include a series of actions within a continuous time period during the execution of a behavior by an intelligent agent. Specifically, in the control of grasping behavior by intelligent agents such as robots, robotic arms, or end effectors, the first action sequence may include motion parameters of multiple parts (such as fingers, wrists, and palms) of the intelligent agent performing the grasping behavior within a continuous time period, in at least one degree of freedom.

[0072] For example, in intelligent agents such as robots, robotic arms, or end effectors, the first action sequence may include motion parameters of 3-4 finger degrees of freedom, 2-3 wrist degrees of freedom, and 1-2 palm degrees of freedom within a continuous time period, and so on.

[0073] The aforementioned preset motion engine can be a pre-trained neural network model. The input of this model is environmental feature data, and the output is a sequence of actions. Its core function is to convert the compressed environmental features into a basic sequence of actions that conforms to physical laws. For example, for intelligent agents such as robots, robotic arms, or end effectors, the preset motion engine can map abstract environmental representations into executable motion parameters and output composite motion instructions (i.e., action sequences) that include end-effector trajectory, joint angles, and contact force thresholds.

[0074] It should be noted that the aforementioned preset motion engine can adopt an encoder-decoder architecture. The encoder in the preset motion engine can perform multi-level abstraction processing on the input environmental features (i.e., the first environmental feature data) to extract the core elements that affect motion generation (such as the distribution of obstacles, the pose of the target grasping point, etc.) and form a feature vector that represents the potential laws of motion patterns. The decoder in the preset motion engine reconstructs the robot's joint space motion trajectory based on this feature vector and ensures that the output motion meets the dynamic feasibility through the physical constraint layer.

[0075] The aforementioned first action sequence can be an action sequence directly output by the preset action engine. Alternatively, in some embodiments, generating the agent's first action sequence based on the first environmental feature data of the agent's environment includes:

[0076] The first environmental feature data of the environment in which the agent is located is input into the preset action engine, and the preset action engine outputs the initial action sequence;

[0077] In the digital twin system, each action in the initial action sequence is simulated to verify whether each action meets the first verification condition.

[0078] Actions that meet the first verification condition in the initial action sequence are removed to generate the first action sequence.

[0079] In this embodiment, after the preset action engine outputs the initial action sequence based on the first environmental feature data, the agent can further simulate and execute each action in the initial action sequence through the digital twin system, and remove actions in the initial action sequence that meet the first verification condition to generate the first action sequence. Thus, the agent's execution of actions can be controlled through action verification.

[0080] The aforementioned digital twin system can be configured within the aforementioned intelligent agent, enabling the intelligent agent to simulate and execute corresponding actions (which can be referred to as "virtual actions") in a virtual environment based on the action parameters in the initial action sequence. Furthermore, the intelligent agent can verify each simulated action according to a preset first verification condition.

[0081] The aforementioned first verification condition can be any preset condition used to eliminate some actions in the initial action sequence (also referred to as "eliminating some action parameters in the initial action sequence"). Specifically, for intelligent agents such as robots, robotic arms, or end effectors, the aforementioned first verification condition can include at least one of the following: a dangerous action that could cause a collision or exceed the torque limit; a smooth action with continuous acceleration; or an action in which the change in contact force during the action exceeds a preset range.

[0082] In some implementations, the first verification condition may include: a dangerous action that causes a collision or exceeds the torque limit; a smooth action with continuous acceleration; or an action in which the change in contact force exceeds a preset range. This allows for three-level verification of the action sequence output by the action engine, further improving the controllability of the agent's actions.

[0083] To ensure the effectiveness of the preset action engine in the action sequence generation process, before inputting the first environmental feature data of the agent's environment into the preset action engine, the initial action engine can be trained based on preset training data to obtain the aforementioned preset action engine. This training data can consist solely of sample data generated under a simulation environment.

[0084] In some implementations, before inputting the first environmental feature data of the agent's environment into the preset action engine, the method further includes:

[0085] Obtain the third training sample set generated in the simulation environment;

[0086] The initial action engine is trained based on the third training sample set to obtain the action engine to be transferred.

[0087] Obtain the fourth training sample set generated in a real-world environment;

[0088] The action engine to be transferred is trained based on the fourth training sample set to obtain the preset action engine.

[0089] In this embodiment, after the action engine to be transferred is trained using sample data in a simulated environment, the agent can further train the action engine to be transferred using real sample data, thereby achieving fine-tuning of the action engine to be transferred and enabling the trained preset action engine to adapt to the actual physical environment.

[0090] It should be noted that, in addition to fine-tuning the trained motion engine using sample data generated in a real environment, the sample data in the third training sample set can also be perturbed. For example, noise parameters (such as sensor errors, material parameter perturbations, etc.) can be injected into the simulation environment through domain randomization techniques to generate negative samples, thereby enhancing the ability of the preset motion engine to adapt to real-world uncertainties.

[0091] In step S101 above, the generation of the first action sequence may depend solely on the first environmental feature data mentioned above.

[0092] Alternatively, in some embodiments, when the first environmental feature data is obtained by processing multimodal sensing data based on the second encoder, the second encoder is configured with a memory network component.

[0093] After generating the first environmental feature data by extracting redundant information from the multimodal sensing data through the second encoder based on multi-layer nonlinear transformation, the above method may further include:

[0094] The second encoder obtains the second environmental feature data recorded by the memory network component. The second environmental feature data is the environmental feature data generated at a historical moment, and the historical moment is the moment before the moment when the first environmental feature data was generated.

[0095] Based on the first environmental feature data of the agent's environment, the agent's first action sequence is generated, including:

[0096] Based on the first environmental feature data and the second environmental feature data, the first action sequence of the agent is generated.

[0097] In this embodiment, the agent can generate a first action sequence based on the first environmental feature data at the current moment and the second environmental feature data at historical moments, thereby making the generated first action sequence more accurate.

[0098] The aforementioned historical moment can be any point in time or period other than the current moment when the first environmental feature data is collected. For example, the agent can obtain environmental feature data acquired when it performs an action within 2 minutes before the current moment as the aforementioned second environmental feature data, and so on.

[0099] The above-mentioned generation of the first action sequence of the intelligent agent based on the first environmental feature data and the second environmental feature data can be achieved by weighted fusion of the first environmental feature data and the second environmental feature data to generate fused environmental feature data, and inputting the fused environmental feature data into a preset action engine to obtain the first action sequence.

[0100] It should be noted that when performing weighted fusion of the first and second environmental feature data, the weights of the first and second environmental feature data can be preset according to actual needs. For example, the weights of the first and second environmental feature data can be preset to 0.8 and 0.2, respectively, and so on.

[0101] Alternatively, in some implementations, when the agent obtains the second environmental feature data generated at a historical moment through the memory network component, generating the agent's first action sequence based on the first environmental feature data of the agent's environment may include: determining the second environmental feature data that matches the first environmental feature data; determining the historical action sequence output in the preset action engine history based on the second environmental feature data that matches the first environmental feature data as the first action sequence, thereby realizing the retrospection of historical actions.

[0102] Of course, in some embodiments, where the agent obtains the second environmental feature data generated at a historical moment through the memory network component, the method may further include: generating environmental change trend prediction information based on the first and second environmental feature data, which is used to predict the change trend of the environment in which the agent is located. This provides forward-looking decision support for the agent's behavioral control.

[0103] In this application, after step S101, the method may further include: displaying third information, wherein the third information is used to indicate the first action sequence and / or the generation process information of the first action sequence. Thus, by displaying the third information, the visualization of the first action sequence and its generation process can be achieved.

[0104] In some implementations, the above-mentioned display of third information may include:

[0105] Display third information and second adjustment controls, the third information being used to indicate the first action sequence and / or the generation process information of the first action sequence;

[0106] The method may further include: adjusting a first action sequence in response to a second operation on a second adjustment control, wherein the second operation is based on third information input.

[0107] In this way, a second operation can be input into the second control space based on the third information, and then the first action sequence can be adjusted through the second operation, thereby realizing the online control of the generation process of the first action sequence.

[0108] The second operation described above can be any operation performed on the input of the second control space based on the third information. For example, the second operation can be at least one of voice input, touch input, and air gesture input.

[0109] Step S102: In response to the received user instruction, perform intent parsing on the user instruction and generate instruction intent information.

[0110] In this step, when the agent receives a user instruction, it can parse the user instruction according to a preset intent parsing model and generate instruction intent information.

[0111] The aforementioned instruction intent information can be any information that characterizes the behavior the user expects the agent to perform. Specifically, the instruction intent information can include expected action parameters. For example, assuming the user instruction includes the text "decrease slightly by 0.1mm on the Z-axis while rotating 0.5° around the X-axis," then the expected action parameters could be (decrease by 0.1mm on the Z-axis, rotate 0.5° around the X-axis), and so on.

[0112] It should be noted that the above-mentioned intelligent agent receiving user instructions, and performing intent parsing on user instructions to generate instruction intent information, can be executed simultaneously with step S101, or before step S101, or after step S101, without any limitation here.

[0113] The aforementioned user instruction can be any instruction input by the user to instruct the intelligent agent to perform a behavior, and there is a correlation between the user instruction and the aforementioned first environmental feature data. For example, during the grasping action of an intelligent agent such as a robot, robotic arm, or end effector, the user instruction could be "Please handle the frosted glass on the left side of the table gently, and be careful not to knock over the soy sauce bottle next to it." In this case, the aforementioned first environmental feature data could include data such as the target object being a tall frosted glass (e.g., fragile, with a surface friction coefficient of 0.3), a soy sauce bottle tilted 10cm to the right, and the state of the table surface where water stains cause local slippage.

[0114] The aforementioned user instructions can be a single modality or a combination of multiple modalities. These multiple modalities may include at least two of the following: natural language instructions, action teaching instructions, and structured instructions.

[0115] When the user instructions mentioned above include only one modality, the intent parsing model can parse the user intent of the instructions in that modality based on the instruction parsing mechanism corresponding to that modality, and obtain the aforementioned instruction intent information.

[0116] In cases where the user instructions include multiple modalities, the intent parsing model can parse each modal instruction separately to obtain the user intent corresponding to each modal instruction, and then fuse the user intents corresponding to multiple modal instructions to obtain the aforementioned instruction intent information.

[0117] In some implementations, the aforementioned user instructions include instructions in multiple modalities.

[0118] like Figure 2 As shown, step S102 above may include:

[0119] Step S1021: For each mode in the multiple modes, based on the instruction parsing mechanism corresponding to the mode, perform intent parsing on the instructions of the mode to obtain the intent feature vector of the mode;

[0120] Step S1022: Map the intent feature vectors of multiple modalities to the same semantic space, and dynamically extract the key instruction components of the intent feature vectors in the semantic space through a multi-head attention mechanism to generate instruction intent information.

[0121] In this embodiment, the intent feature vectors corresponding to instructions from multiple modalities can be fused through a multi-head attention mechanism, thereby making the generated instruction intent information more reflective of the user's intent.

[0122] The aforementioned intent parsing model may have a pre-configured instruction parsing mechanism corresponding to the various modalities. Thus, when a user instruction is received, the instruction for each modality can be parsed individually through the instruction parsing mechanism corresponding to each modality, thereby ensuring the accuracy of parsing the intent feature vector of the instruction for each modality.

[0123] Specifically, the above-mentioned instruction parsing mechanism based on the modality performs intent parsing on the modality's instructions to obtain the modality's intent feature vector, including at least one of the following:

[0124] In the case of multi-modal instructions including natural language instructions, the core components of natural language instructions are extracted through part-of-speech tagging and dependency parsing. The corresponding operation object attributes of the core components of the statements are searched in the domain knowledge graph, and the intent feature vector of the corresponding modality of the natural language instruction is generated based on the operation object attributes.

[0125] In the case of multi-modal instructions including action teaching instructions, a spatiotemporal convolutional network is used to extract the trajectory features of the teaching actions in the action teaching instructions, search for natural language instructions that match the trajectory features in the shared embedding space, and generate an intent feature vector corresponding to the matched natural language instructions.

[0126] In the case of multi-modal instructions, including structured instructions, the structured instructions are converted into standard control parameter templates through syntax tree parsing, and the intent feature vector of the corresponding modality of the structured instructions is generated based on the standard control parameter templates.

[0127] In this way, the feature vectors of natural language instructions, action teaching instructions, and structured instructions can be accurately parsed respectively.

[0128] Furthermore, the aforementioned user instructions include natural language instructions, action teaching instructions, and structured instructions. The modality-based instruction parsing mechanism performs intent parsing on the modal instructions to obtain modal intent feature vectors. This includes: extracting the core components of the natural language instructions through part-of-speech tagging and dependency parsing; searching for the corresponding operation object attributes in the domain knowledge graph; and generating the modal intent feature vector corresponding to the natural language instructions based on the operation object attributes. It also involves using a spatiotemporal convolutional network to extract the trajectory features of the teaching actions in the action teaching instructions; searching for natural language instructions matching the trajectory features in the shared embedding space; and generating the intent feature vector corresponding to the matched natural language instructions. Finally, it transforms structured instructions into standard control parameter templates through syntax tree parsing and generates the modal intent feature vector corresponding to the structured instructions based on the standard control parameter templates. This ensures accurate parsing of user intent even when multiple modalities of instructions are input.

[0129] In step S102 above, the instruction intent information can be directly output by the intent parsing model, that is, the instruction intent information depends entirely on the user instruction received at the current moment; or, the intent parsing model can also output instruction intent information by referring to the user's historical instructions.

[0130] In some implementations, the above-described intent parsing of user commands to generate command intent information may include:

[0131] Retrieve historical commands associated with user commands;

[0132] Based on the gating weights corresponding to user commands and historical commands, as well as the user commands and historical commands, command intent information is generated.

[0133] In this way, the intelligent agent can generate the above-mentioned instruction intent information based on the user instruction received at the current moment and the associated historical instructions, thereby enabling the retrospection of the user's historical instructions and making the parsed user intent information more accurately reflect the user's true intention.

[0134] The aforementioned historical instructions associated with user history can be any instruction entered by the user before the current moment that has a relationship with the aforementioned user instructions. For example, the historical instructions can be instructions entered a preset number of times before the current moment.

[0135] The above-mentioned generation of instruction intent information based on the gating weights corresponding to user instructions and historical instructions can be achieved by obtaining the intent feature vectors corresponding to user instructions and historical instructions respectively through an intent parsing model, and then weighting and fusing the intent feature vectors corresponding to user instructions and historical instructions through the gating weights corresponding to user instructions and historical instructions respectively, and determining the fused intent feature vector as the above-mentioned intent instruction information.

[0136] It should be noted that the gating weights corresponding to the aforementioned user commands and historical commands can be set according to actual needs. For example, the gating weights corresponding to the user commands and historical commands can be preset to 0.7, 0.3, etc.

[0137] Alternatively, in other implementations, parsing the user command to generate command intent information may include: obtaining historical commands associated with the user command; and determining the historical intent information associated with the historical commands as the command intent information. This can improve the efficiency of parsing and obtaining command intent information.

[0138] It should be noted that the acquisition of the aforementioned historical instructions and historical intent information can also be achieved by pre-setting a memory network component related to user instructions in the intelligent agent, which will not be elaborated here.

[0139] Step S103: Generate a control signal based on the instruction intent information, the first environmental feature data, and the first action sequence.

[0140] In this step, after the agent obtains the instruction intent information, the first environmental feature data, and the first action sequence, it inputs the instruction intent information, the first environmental feature data, and the first action sequence into the preset control signal generation model to generate a control signal.

[0141] The aforementioned regulatory signal generation model can be a pre-trained neural network-based model. The input of the model is instruction intent information, first environmental feature data, and first action sequence. The corresponding regulatory signal is obtained by predicting the output of the model.

[0142] In some implementations, such as Figure 3 As shown, step S103 above includes:

[0143] Step S1031: Input the instruction intent information, the first environmental feature data and the first action sequence into the first controller, the first controller including the first encoder and the first decoder;

[0144] Step S1032: Construct a conditional latent space based on instruction intent information and first environmental feature data using the first encoder;

[0145] Step S1033: The first action sequence is reparameterized in the conditional latent space using the first decoder to generate latent variables associated with the first action sequence.

[0146] Step S1034: Generate a control signal based on the latent variables and conditional information of the conditional latent space through the first decoder.

[0147] In this embodiment, the first action sequence can be used as the object to be regulated in the first regulator, and the instruction intent information and the first environmental feature data can be used as the basis for adjustment for joint modeling. That is, the first action sequence is reparameterized in the conditional latent space constructed based on the instruction intent information and the first environmental feature data, and a regulation signal is generated based on the latent variables and the conditional information of the conditional latent space, so that the generated regulation signal is more appropriate, thereby further improving the accuracy of the agent's behavior control.

[0148] The first regulator mentioned above can be a pre-trained neural network model. That is, the first regulator can include an encoding network (i.e., a first encoder) and a decoding network (i.e., a first decoder). During the training process, the network parameters of the encoding network and the decoding network can be adjusted separately based on the loss value between the regulation signal output by the first regulator and the expected output signal of the first regulator (i.e., the labeled signal in the training samples) to obtain the first regulator.

[0149] In some implementations, the first regulator is trained based on a target loss, which is calculated in the following way:

[0150] Calculate the reconstruction sub-loss, which reflects the consistency between the control signal output by the first controller and the desired output signal of the first controller;

[0151] Calculate the potential space loss, which reflects the consistency between the first distribution of the control signal output by the first controller and the second distribution of the expected output signal of the first controller;

[0152] Calculate the smoothing loss, which is used to suppress sudden jumps in the control signal output by the first controller.

[0153] In this embodiment, during the training process of the first regulator, the reconstruction sub-loss, latent space loss, and smoothing loss can be used as the training optimization objectives of the first regulator, thereby realizing the adoption of a multi-objective optimization strategy in the model training process, making the regulatory signals output by the model more accurate, and further improving the accuracy of agent behavior control.

[0154] Of course, the first regulator mentioned above can also be trained based on any of the two losses: reconstruction sub-loss, latent space loss, and smoothing loss. No limitation is imposed here.

[0155] As can be seen from the above, the instruction intent information may include expected action parameters. When the instruction intent information includes expected action parameters, the intelligent agent may directly generate the control signal based on the expected action parameters.

[0156] Or, in some implementations, such as Figure 4 As shown, step S103 above may include:

[0157] Step S1035: Abstract the expected action parameters and generate abstract functional features corresponding to the action parameters;

[0158] Step S1036: Generate a control signal based on abstract functional features, first environmental feature data, and first action sequence.

[0159] In this embodiment, abstract functional features can be generated by abstracting the desired action parameters, and a control signal can be generated based on the abstract functional features, the first environmental feature data, and the first action sequence, thereby realizing the abstraction of the control correction strategy (i.e., the control signal).

[0160] The aforementioned abstract functional features can be any feature used to describe the abstract function of the desired action parameters. Specifically, the abstract functional features can include task-oriented spatial features and high-level semantic features, etc. That is, the above-mentioned abstract expression of the desired action parameters to generate abstract functional features corresponding to the action parameters can include at least one of the following:

[0161] Map the desired motion parameters to a task-oriented feature space. For example, map joint space motion parameters (such as angles and angular velocities) to a task-oriented feature space (such as end-effector accuracy, energy efficiency, and safety margin).

[0162] The expected action parameters are transformed into high-level semantic feature dimensions to obtain the high-level semantic features corresponding to the expected action parameters. For example, the control signal is allowed to act on high-level semantic feature dimensions such as "grasping stability coefficient" and "path smoothness factor".

[0163] It should be noted that the above-mentioned abstract expression of the expected action parameters and generation of abstract functional features corresponding to the action parameters can be determined by the mapping relationship between the preset action parameters and the abstract functional features.

[0164] In some implementations, after the intelligent agent generates a control signal based on instruction intent information, first environmental feature data, and a first action sequence, the method further includes: displaying first information, which indicates the control signal and / or information about the generation process of the control signal. This allows for visualization of the control signal and / or the generation process of the control signal.

[0165] The aforementioned display of the first information may refer to displaying the first information only on the display interface for the user to view.

[0166] In some implementations, after generating the control signal based on the instruction intent information, the first environmental feature data, and the first action sequence, the method further includes:

[0167] Display first information and first adjustment control, the first information being used to indicate the control signal and / or information on the generation process of the control signal;

[0168] In response to a first trigger operation on a first adjustment control, an adjustment control signal is adjusted, wherein the first trigger operation is based on a first information input.

[0169] In this embodiment, by displaying first information and a first adjustment space, the user can input a first trigger operation based on the first adjustment control while viewing the first information, thereby achieving timely adjustment of the control signal.

[0170] The aforementioned first adjustment control can be an adjustable control displayed simultaneously with the first information on the display interface, and the first information can change accordingly when the user adjusts the adjustable control. Furthermore, the adjustable control can be displayed in any form, such as a floating ball or a progress bar.

[0171] The aforementioned first information can be any indication of the control signal and / or information regarding the generation process of the control signal. Specifically, the aforementioned first information includes at least one of the following:

[0172] Dynamic heatmaps are used to show the influence weights of each control dimension in the control signal on the end effector pose of the agent;

[0173] A three-dimensional force diagram is used to display the direction and intensity of the mechanical action of the action correction strategy generated by the control signal;

[0174] The decision recording module is used to record relevant information about the action correction strategy generated by the control signal. For example, the decision recording module may include a historical control decision tree for recording the logical relationship of key correction processes in the action correction strategy.

[0175] For example, during the grasping action control of intelligent agents such as robots, robotic arms, or end effectors, when a control signal with a connector offset angle θ > 1° is detected, the historical control decision tree can be used to prioritize activating the "contact surface self-alignment" control dimension (e.g., with a weight of 68%), followed by triggering the "flexible impedance adjustment" dimension (e.g., with a weight of 29%), and so on. This transparent display can shorten the debugging cycle and improve the efficiency of abnormal condition diagnosis.

[0176] Specifically, the aforementioned first information may include the aforementioned dynamic heat map, the aforementioned three-dimensional force line diagram, and the aforementioned decision record module.

[0177] In some implementations, after generating the control signal based on the instruction intent information, the first environmental feature data, and the first action sequence, the method further includes: outputting second information, wherein the second information is prompt information generated based on the control signal strength of the control signal, and the second information includes at least one of the following:

[0178] Information used to indicate the strength of the control signal;

[0179] Predictive information used to indicate the trend of changes in the intensity of control signals;

[0180] Information used to indicate the regulatory risk of adjusting the first action sequence based on the strength of the regulatory signal.

[0181] In this embodiment, it is possible to construct a prompt message that quantitatively assesses the strength of the control signal, so that the user can intervene in the control signal in a timely manner based on the prompt message.

[0182] It should be noted that the above prompts can be any information that can prompt the user, and the output method can be customized according to actual needs.

[0183] For example, an indicator light could be set up so that it displays green when the control signal strength meets |Ct|≤0.3, yellow when the control signal strength meets 0.3<|Ct|≤0.7, and red when the control signal strength meets |Ct|>0.7; or, an intensity trend prediction curve could be displayed to show the trend of the control signal strength in the future (e.g., within the next 3 seconds); or, a voice prompt could be output, issuing a "Pay attention to the correction amplitude" warning when the control signal strength meets 0.3<|Ct|≤0.7 for 3 consecutive seconds, and so on.

[0184] More specifically, the aforementioned second information includes: information for indicating the strength of the control signal; predictive information for indicating the changing trend of the control signal strength; and information for indicating the control risk of controlling the first action sequence based on the control signal strength.

[0185] Step S104: Adjust the first action sequence based on the control signal to generate the second action sequence of the agent.

[0186] In this step, after acquiring the control signal, the agent can input the control signal into a preset action sequence control model, which will then directly output a second action sequence. This action sequence model can be a prediction model trained using a neural network.

[0187] In some implementations, such as Figure 5 As shown, step S104 above includes:

[0188] Step S1041: Input the control signal, the first action sequence, and the first environmental feature data into the action integrator. The action integrator includes a feature fusion network and an action optimization network.

[0189] Step S1042: Through a feature fusion network, based on a temporal attention mechanism, the spatiotemporal features of the control signal, the first action sequence, and the first environmental feature data are dynamically associated to generate a joint feature vector;

[0190] Step S1043: Generate action correction values ​​based on joint feature vectors through the action optimization network;

[0191] Step S1044: Modify the first action sequence based on the action correction amount using the action optimization network.

[0192] In this embodiment, the action integrator can dynamically associate the spatiotemporal features between the control signal, the first action sequence, and the first environmental feature data based on the temporal attention mechanism, generate a joint feature vector, generate an action correction amount based on the joint feature vector, and correct the first action sequence based on the action correction amount, thereby ensuring the accuracy of the correction of the first action sequence.

[0193] The aforementioned feature fusion network can be a network layer that dynamically associates the spatiotemporal features of the control signal, the first action sequence, and the first environmental feature data based on a temporal attention mechanism to generate a joint feature vector. For example, this network layer may include multiple feature extraction networks and a fusion network. The multiple feature extraction networks extract the spatiotemporal features of the control signal, the first action sequence, and the first environmental feature data, respectively. The feature fusion network fuses the spatiotemporal features of the control signal, the first action sequence, and the first environmental feature data based on its attention mechanism to obtain a joint feature vector.

[0194] It should be noted that in the process of the above-mentioned action integrator dynamically associating the control signal, the first action sequence, and the first environmental feature data based on the temporal attention mechanism to generate a joint feature vector, the above-mentioned action integrator can encode the following core information: the original motion pattern of the basic action, the correction requirements proposed by the control signal, and the boundary conditions of the environmental state constraints, thereby providing a reliable input benchmark for subsequent decision-making.

[0195] The aforementioned motion optimization network can output motion corrections based on joint feature vectors. For example, for intelligent agents such as robots, robotic arms, or end effectors, the motion optimization network can output motion corrections for each degree of freedom at the corresponding motion part based on joint feature vectors.

[0196] The above-mentioned modification of the first action sequence by using an action optimization network and based on action correction amount can be achieved by directly increasing or decreasing the action parameters based on the corresponding action parameter amount in the action correction amount, based on the action parameters in each action sequence.

[0197] In some implementations, the action integrator further includes a dynamic weight decision network, and the first action sequence includes first action parameters of the target action part of the agent in the target degree of freedom;

[0198] like Figure 6 As shown, step S104 above further includes:

[0199] Step S1045: Generate the first corrected weights of the target action part in the target degree of freedom through a dynamic weight decision network based on the joint feature vector;

[0200] Step S1044 above includes:

[0201] The first action parameters are corrected by the action optimization network based on the action correction amount and the first correction weight to obtain the second action parameters, wherein the second action sequence includes the second action parameters.

[0202] In this embodiment, the first motion parameter of the target motion part in the target degree of freedom can be corrected based on the motion correction amount and the first correction weight of the target motion part in the target degree of freedom. This allows for different corrections of different degrees of freedom for different motion parts, making the correction of each motion independent.

[0203] The above-mentioned generation of the first corrected weight of the target action part in the target degree of freedom based on the joint feature vector can be achieved by pre-setting a correspondence between the feature vector and the corrected weights of different degrees of freedom of different actions, and determining the corrected weight of the target action part in the target degree of freedom corresponding to the joint feature vector as the first corrected weight according to the correspondence; or, the dynamic weight decision network can be a trained prediction network, and the first corrected weight of the target action part in the target degree of freedom can be predicted by inputting the joint feature vector into the prediction network.

[0204] For example, for different joint feature vectors, different action parts, and different degrees of freedom, the dynamic weight decision network can dynamically determine the corresponding weights in the [0,1] interval as the first correction weights mentioned above, such as determining the translational degree of freedom weights of the end effector and the rotational degree of freedom of the wrist.

[0205] In some implementations, the above-mentioned generation of the first correction weight of the target action part in the target degree of freedom based on the joint feature vector includes at least one of the following:

[0206] If the initial correction weight is less than or equal to the preset weight threshold, the initial correction weight is determined as the first correction weight, wherein the initial correction weight is the weight of the target action part in the target degree of freedom, and is generated by the dynamic weight decision network based on the joint feature vector;

[0207] If the initial correction weight is less than or equal to the preset weight threshold, the preset weight threshold is determined as the first correction weight of the target degree of freedom, and the second action parameter is obtained.

[0208] In this embodiment, the agent can generate initial correction weights for the target action part in the target degree of freedom based on the joint feature vector. When the initial correction weight is less than or equal to a preset weight threshold, the initial correction weight is determined as the first correction weight. When the initial correction weight is greater than the preset weight threshold, the preset weight threshold is determined as the first correction weight, thereby avoiding excessive correction weights and constraining the first correction weight.

[0209] It should be noted that the aforementioned preset weight threshold can be a weight value pre-set according to actual needs, and it can be fixed; or, the aforementioned preset weight threshold can also be dynamically adjusted based on the first environmental feature data. That is, before generating the first correction weight of the target action part in the target degree of freedom based on the joint feature vector, the method further includes: determining the aforementioned preset weight threshold corresponding to the first environmental feature data. For example, when a slippery surface is detected, the weight threshold is reduced compared to a dry surface, and so on.

[0210] Of course, the agent can also determine the first correction weight based on other information. For example, it can be dynamically adjusted based on the information of the stage in which each action is performed (basic actions are emphasized and have a larger weight during the start-up phase, while environmental constraints are emphasized and have a smaller weight during the emergency control phase), or it can be dynamically adjusted based on key correction needs (such as avoiding sudden obstacles).

[0211] The first action parameter is modified based on the action correction amount and the first correction weight to obtain the second action parameter. It can be a fixed weight set in the action optimization network corresponding to the first action sequence. The second action parameter can be the sum of the product of the first action parameter and the fixed weight, and the product of the action correction amount and the first correction weight.

[0212] In some implementations, such as Figure 7 As shown, step S1044 above may include:

[0213] Step S10441: Calculate the first product between the first action parameter and the first correction weight, and calculate the second product between the action correction amount and the second correction weight. The second correction weight is obtained by subtracting the first correction weight from 1.

[0214] Step S10442: Determine the sum of the first product and the second product as the second action parameter after the target degrees of freedom are corrected.

[0215] In this embodiment, the weights corresponding to the first action parameter and the action correction amount can be adjusted simultaneously through the first correction weight, so that the determined second action parameter is more reasonable.

[0216] It should be noted that the above-mentioned regulation of the first action sequence based on the control signal to generate the agent's second action sequence can be based solely on the action correction amount corresponding to each degree of freedom under each action and the first correction weight to modify the first action sequence.

[0217] In some implementations, such as Figure 8 As shown, before step S104 above, the procedure also includes:

[0218] Step S106: Obtain the allowable action correction amount associated with the first action sequence;

[0219] The above step S104 includes:

[0220] Based on the control signal and the minimum of the allowed action correction amount, the first action sequence is controlled to generate the agent's second action sequence.

[0221] In this embodiment, the maximum motion correction amount of the first motion sequence can be limited by adjusting the control signal and the allowable motion correction amount, thereby ensuring the stability of the motion adjustment.

[0222] The aforementioned acquisition of the permissible action correction amount associated with the first action sequence can be achieved by pre-setting a fixed permissible action correction amount in the agent, which is used for any action sequence.

[0223] Or, in some implementations, such as Figure 9 As shown, before step S106 above, the procedure also includes:

[0224] Step S107: Identify the operation scenario type of the first action sequence;

[0225] Step S108: Determine the safety factor corresponding to the second action sequence based on the type of operation scenario;

[0226] Step S106 above may include:

[0227] Based on the safety factor and the intensity of the control signal, the permissible action correction amount associated with the first action sequence is determined.

[0228] In this implementation, different safety factors can be determined according to the operation scenario type of different action sequences, and the allowable action correction amount associated with the first action sequence can be determined according to the safety factor and the control signal strength of the control signal. This allows for the determination of the corresponding allowable action correction amount for different operation scenarios, thereby further improving the safety of action correction.

[0229] The aforementioned determination of the safety factor corresponding to the second action sequence based on the operation scenario type can be achieved by pre-setting a correlation between different scenario types and correction amounts in the agent, and determining the correction amount that is associated with the operation scenario type of the first action sequence as the allowed action correction amount based on this correlation.

[0230] For example, for special scenarios (such as sudden obstacle avoidance), a larger allowable action correction amount can be set to automatically relax the correction restrictions to meet emergency needs; while for regular operation scenarios, a relatively small allowable action correction amount is set to maintain strict constraints to ensure action stability.

[0231] The above-mentioned determination of the allowable action correction amount associated with the first action sequence based on the safety factor and the control signal strength of the control signal can be achieved by setting the allowable action correction amount to be less than or equal to the product between the safety factor and the control signal strength, i.e., allowable action correction amount ≤ safety factor × control signal strength.

[0232] Step S105: Based on the second action sequence, control the agent to perform actions.

[0233] In this step, after obtaining the second action sequence, the agent can directly control the agent to perform actions based on the second action sequence.

[0234] Alternatively, in some implementations, before controlling the agent to perform actions based on the second action sequence, the process may further include: performing simulated actions based on the second action sequence through a digital twin system to verify whether each action meets the second verification condition; removing actions in the initial action sequence that meet the second verification condition, and adjusting the second action sequence.

[0235] The aforementioned adjustment to the second action sequence may involve adjusting the first correction weight and then readjusting the action parameters in the first action sequence based on the adjusted first correction weight and action correction amount.

[0236] The second verification condition mentioned above can be the same as the first verification condition mentioned above. That is, the second verification condition mentioned above can include at least one of the following: a dangerous action that causes a collision or excessive torque; a smooth action with continuous acceleration; or an action in which the change of contact force during the action exceeds a preset range.

[0237] In some implementations, after controlling the agent to perform actions based on the second action sequence, the method further includes:

[0238] Based on the first environmental feature data and the second action sequence, supplementary training samples are generated.

[0239] The preset action engine is trained based on supplementary training samples to update the preset action engine.

[0240] In this embodiment, supplementary training samples can be generated using the first environmental feature data and the second action sequence, and the preset action engine can be retrained based on the supplementary training samples, thereby realizing a closed loop of the data link, updating the preset action engine in the form of a data flywheel, reducing the required data and improving the efficiency of sample utilization.

[0241] In this embodiment, a first action sequence of the agent is generated based on first environmental feature data of the agent's environment. In response to a received user instruction, the user instruction is parsed to generate instruction intent information. Based on the instruction intent information, the first environmental feature data, and the first action sequence, a control signal is generated. The first action sequence is then controlled using the control signal to generate a second action sequence of the agent. Finally, the agent is controlled to execute actions based on the second action sequence. Thus, by constructing a two-layer architecture separating the basic capability layer and the user instruction control layer—the basic capability layer generates the first action sequence based on the first environmental feature data, while the user instruction control layer parses user intent information from user instructions and generates control signals based on the user intent information, the first environmental feature data, and the first action sequence, and controls the first action sequence using these control signals—the generation of the agent's actions not only considers environmental constraints on the agent's actions but also integrates explicit constraints imposed by user intent, thereby improving the stability of the agent's actions and enhancing the controllability of the agent's behavior.

[0242] The control method for an intelligent agent provided in this application can be executed by a control device for the intelligent agent. This application uses the example of a control device for the intelligent agent executing the control method to illustrate the control device for the intelligent agent provided in this application.

[0243] Please see Figure 10 This is a schematic diagram of the structure of the control device for the intelligent agent provided in an embodiment of this application. Figure 10 As shown, the control device 1000 for the intelligent agent of this application includes:

[0244] Action sequence generation module 1001 is used to generate a first action sequence of the intelligent agent based on the first environmental feature data of the environment in which the intelligent agent is located;

[0245] The intent parsing module 1002 is used to respond to a received user instruction, parse the user instruction, and generate instruction intent information.

[0246] The control signal generation module 1003 is used to generate a control signal based on the instruction intent information, the first environmental feature data, and the first action sequence;

[0247] The control module 1004 is used to control the first action sequence based on the control signal to generate the second action sequence of the intelligent agent;

[0248] The action execution module 1005 is used to control the intelligent agent to perform actions based on the second action sequence.

[0249] In some embodiments, the control signal generation module 1003 is specifically used for:

[0250] The instruction intent information, the first environmental feature data, and the first action sequence are input into the first controller, which includes a first encoder and a first decoder.

[0251] The first encoder constructs a conditional latent space based on the instruction intent information and the first environmental feature data.

[0252] The first action sequence is reparameterized in the conditional latent space using the first decoder to generate latent variables associated with the first action sequence.

[0253] The first decoder generates a control signal based on the latent variables and the conditional information of the conditional latent space.

[0254] In some implementations, the first regulator is trained based on a target loss, which is calculated in the following manner:

[0255] Calculate the reconstruction sub-loss, which reflects the consistency between the control signal output by the first controller and the desired output signal of the first controller;

[0256] Calculate the potential space loss, which reflects the consistency between the first distribution of the control signal output by the first controller and the second distribution of the desired output signal of the first controller;

[0257] Calculate the smoothing loss, which is used to suppress sudden jumps in the control signal output by the first controller.

[0258] In some embodiments, the device 1000 further includes:

[0259] The allowed action correction amount acquisition module is used to acquire the allowed action correction amount associated with the first action sequence.

[0260] The control module 1004 can be specifically used for:

[0261] Based on the minimum of the control signal and the allowed action correction amount, the first action sequence is controlled to generate the agent's second action sequence.

[0262] In some embodiments, the device 1000 further includes:

[0263] An operation scenario type identification module is used to identify the operation scenario type of the first action sequence;

[0264] The safety factor determination module is used to determine the safety factor corresponding to the second action sequence based on the operation scenario type.

[0265] The allowed action correction amount acquisition module can be specifically used for:

[0266] Based on the safety factor and the intensity of the control signal of the control signal, the allowable action correction amount associated with the first action sequence is determined.

[0267] In some implementations, the instruction intent information includes expected action parameters.

[0268] The control signal generation module 1003 can be specifically used for:

[0269] The desired action parameters are abstracted and expressed to generate abstract functional features corresponding to the action parameters;

[0270] Based on the abstract functional features, the first environmental feature data, and the first action sequence, a control signal is generated.

[0271] In some embodiments, the device 1000 further includes:

[0272] A first display module is used to display first information and a first adjustment control, wherein the first information is used to indicate the control signal and / or the generation process information of the control signal;

[0273] The control signal adjustment module is used to adjust the control signal in response to a first trigger operation on the first adjustment control, wherein the first trigger operation is based on the first information input.

[0274] In some implementations, the first information includes at least one of the following:

[0275] A dynamic heatmap is used to show the influence weight of each control dimension in the control signal on the end effector pose of the agent.

[0276] A three-dimensional force diagram is used to display the direction and intensity of the mechanical action of the motion correction strategy generated by the control signal;

[0277] The decision recording module is used to record relevant information about the action correction strategy generated by the control signal.

[0278] In some embodiments, the device 1000 further includes:

[0279] An information output module is used to output second information, which is a prompt message generated based on the intensity of the control signal of the control signal. The second information includes at least one of the following:

[0280] Information used to indicate the strength of the control signal;

[0281] Predictive information used to indicate the trend of change in the intensity of the control signal;

[0282] Information used to indicate the regulatory risk of adjusting the first action sequence based on the intensity of the regulatory signal.

[0283] In some embodiments, the control module 1004 is specifically used for:

[0284] The control signal, the first action sequence, and the first environmental feature data are input into the action integrator, which includes a feature fusion network and an action optimization network.

[0285] Through the feature fusion network, the spatiotemporal features of the control signal, the first action sequence, and the first environmental feature data are dynamically associated based on the temporal attention mechanism to generate a joint feature vector.

[0286] The action optimization network generates action correction values ​​based on the joint feature vector.

[0287] The first action sequence is modified based on the action correction amount using the action optimization network.

[0288] In some implementations, the action integrator further includes a dynamic weight decision network, wherein the first action sequence includes first action parameters of the target action part of the agent in the target degree of freedom.

[0289] The device 1000 may further include:

[0290] The corrected weight generation module is used to generate the first corrected weight of the target action part in the target degree of freedom based on the joint feature vector through the dynamic weight decision network.

[0291] The control module 1004 can be specifically used for:

[0292] The action optimization network is used to correct the first action parameters based on the action correction amount and the first correction weight to obtain the second action parameters.

[0293] The second action sequence includes the second action parameters.

[0294] In some embodiments, the control module 1004 may be specifically used for at least one of the following:

[0295] If the initial correction weight is less than or equal to a preset weight threshold, the initial correction weight is determined as the first correction weight, wherein the initial correction weight is the weight of the target action part in the target degree of freedom, and is generated by the dynamic weight decision network based on the joint feature vector;

[0296] If the initial correction weight is less than or equal to a preset weight threshold, the preset weight threshold is determined as the first correction weight for the target degree of freedom.

[0297] In some embodiments, the control module 1004 may be specifically used for:

[0298] Calculate the first product between the first action parameter and the first correction weight, and calculate the second product between the action correction amount and the second correction weight, wherein the second correction weight is obtained by subtracting the first correction weight from 1;

[0299] The sum of the first product and the second product is determined as the second action parameter after the target degrees of freedom are corrected.

[0300] In some embodiments, the device 1000 further includes:

[0301] The sensor data acquisition module is used to acquire multimodal sensor data of the environment in which the intelligent agent is located;

[0302] The environmental feature data generation module is used to input the multimodal sensing data into the second encoder, and generate the first environmental feature data by extracting redundant information from the multimodal sensing data through the second encoder based on multi-layer nonlinear transformation.

[0303] In some embodiments, the device 1000 further includes:

[0304] The first training sample set acquisition module is used to acquire the first training sample set generated in the simulation environment.

[0305] The first training module is used to train an initial encoder based on the first training sample set to obtain the encoder to be transferred.

[0306] The second training sample set acquisition module is used to acquire the second training sample set generated in the real environment.

[0307] The second training module is used to train the encoder to be transferred based on the second training sample set to obtain the second encoder.

[0308] In some embodiments, the second encoder is configured with constraint information of at least one environmental element; the multimodal sensing data includes: first feature data of the at least one environmental element, and second feature data of other environmental elements besides the at least one environmental element.

[0309] The environmental feature data generation module can be specifically used for:

[0310] Using the constraint information through the second encoder, redundant information in the second feature data is extracted through multi-layer nonlinear transformation to obtain the third feature data;

[0311] Generate first environmental feature data that includes the first feature data and the third feature data.

[0312] In some implementations, the second encoder is configured with a memory network component.

[0313] The environmental feature data generation module can also be used for:

[0314] The second encoder is used to obtain the second environmental feature data recorded by the memory network component. The second environmental feature data is environmental feature data generated at a historical moment, and the historical moment is the moment before the moment when the first environmental feature data was generated.

[0315] The action sequence generation module 1001 is specifically used for:

[0316] Based on the first environmental feature data and the second environmental feature data, the first action sequence of the intelligent agent is generated.

[0317] In some implementations, the user instructions include instructions in multiple modalities.

[0318] The intent parsing module 1002 can be specifically used for:

[0319] For each of the multiple modalities, based on the instruction parsing mechanism corresponding to the modality, the instruction of the modality is parsed to obtain the intent feature vector of the modality;

[0320] The intent feature vectors of multiple modalities are mapped to the same semantic space, and key instruction components of the intent feature vectors are dynamically extracted in the semantic space through a multi-head attention mechanism to generate instruction intent information.

[0321] In some implementations, the user instructions include natural language instructions, action teaching instructions, and structured instructions.

[0322] The intent parsing module 1002 can be specifically used for:

[0323] The core components of the natural language instruction are extracted by part-of-speech tagging and dependency parsing. The corresponding operation object attributes of the core components of the statement are searched in the domain knowledge graph. Based on the operation object attributes, the intent feature vector of the modality corresponding to the natural language instruction is generated.

[0324] A spatiotemporal convolutional network is used to extract the trajectory features of the teaching actions in the action teaching instructions. Natural language instructions that match the trajectory features are searched in the shared embedding space, and an intent feature vector corresponding to the matched natural language instructions is generated.

[0325] The structured instructions are converted into standard control parameter templates by parsing the syntax tree, and the intent feature vector of the modality corresponding to the structured instructions is generated based on the standard control parameter templates.

[0326] In some implementations, the intent parsing module 1002 may be specifically used for:

[0327] Retrieve historical instructions associated with the user instruction;

[0328] The instruction intent information is generated based on the gating weights corresponding to the user instruction and the historical instruction, as well as the user instruction and the historical instruction.

[0329] In some embodiments, the action sequence generation module 1001 may be specifically used for:

[0330] The first environmental feature data of the environment in which the agent is located is input into the preset action engine, and the preset action engine outputs the initial action sequence;

[0331] The initial action sequence is simulated and executed in the digital twin system to verify whether each action meets the first verification condition.

[0332] Actions that meet the first verification condition in the initial action sequence are removed to generate the first action sequence.

[0333] In some implementations, the first verification condition includes:

[0334] Dangerous actions that could cause a collision or exceed torque limits;

[0335] For smooth motion with continuous acceleration;

[0336] This refers to an action in which the change in contact force during the action exceeds a preset range.

[0337] In some embodiments, the device 1000 further includes:

[0338] The third training sample set acquisition module is used to acquire the third training sample set generated in the simulation environment.

[0339] The third training module is used to train an initial action engine based on the third training sample set to obtain the action engine to be transferred.

[0340] The fourth training sample set acquisition module is used to acquire the fourth training sample set generated in the real environment;

[0341] The fourth training module is used to train the action engine to be transferred based on the fourth training sample set to obtain the preset action engine.

[0342] In some embodiments, the device 1000 further includes:

[0343] The second display module is used to display third information and a second adjustment control, wherein the third information is used to indicate the first action sequence and / or the generation process information of the first action sequence;

[0344] An action sequence adjustment module is used to adjust the first action sequence in response to a second operation on the second adjustment control, wherein the second operation is based on the third information input.

[0345] In some embodiments, the device 1000 further includes:

[0346] A supplementary training sample generation module is used to generate supplementary training samples based on the first environmental feature data and the second action sequence;

[0347] The motion engine update module is used to train the preset motion engine based on the supplementary training samples in order to update the preset motion engine.

[0348] The control device 1000 for the intelligent agent provided in this application embodiment can execute the technical solution shown in the above-described intelligent agent control method embodiment. Its implementation principle and beneficial effects are similar, and will not be described again here.

[0349] The method for controlling an intelligent agent provided in this application can be executed by a device for controlling the intelligent agent. This application uses an example of a control device for the intelligent agent executing the method for controlling the intelligent agent to illustrate the control device for the intelligent agent provided in this application.

[0350] Please see Figure 11 This is a schematic diagram of the structure of an intelligent agent provided in an embodiment of this application. The intelligent agent includes the aforementioned bus routing node. Figure 11 As shown, the intelligent agent 300 includes:

[0351] One or more processors 1110;

[0352] The memory 1120 stores one or more programs that, when executed by one or more processors 1110, cause the one or more processors 1110 to implement the control method of the intelligent agent described in any of the above embodiments.

[0353] Memory 1120, as a non-transitory network system, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory 1120 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory 1120 may optionally include remotely located memories 1120 relative to processor 1110, which can be connected to processor 1110 via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0354] The memory 1120 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1120 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 1120 and is called and executed by the processor 1110.

[0355] The processor 1110 can be implemented using a general-purpose central processing unit (CPU), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0356] In some embodiments, the agent further includes:

[0357] Input / output interfaces are used to implement information input and output;

[0358] The communication interface is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0359] The bus transmits information between various components of the device (e.g., processor 1110, memory 1120, input / output interface, and communication interface);

[0360] The processor 1110, memory 1120, input / output interface, and communication interface can communicate with each other within the device via a bus.

[0361] One embodiment of this application also provides a computer-readable storage medium storing computer-executable instructions for executing a control method for an intelligent agent described in any of the embodiments above.

[0362] An embodiment of this application also provides a computer program product, including a computer program or computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer program or computer instructions from the computer-readable storage medium and executes the computer program or computer instructions, causing the computer device to perform a control method for implementing an intelligent agent as described in any of the above embodiments.

[0363] The system architecture and application scenarios described in this application are intended to more clearly illustrate the technical solutions of this application and do not constitute a limitation on the technical solutions provided in this application. Those skilled in the art will understand that as system architectures evolve and new application scenarios emerge, the technical solutions provided in this application are also applicable to similar technical problems.

[0364] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0365] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0366] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0367] The above description, with reference to the accompanying drawings, illustrates some embodiments of this application, but does not limit the scope of the invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and spirit of this invention should be considered within the scope of this application.

Claims

1. A control method for an intelligent agent, characterized in that, include: Based on the first environmental feature data of the environment in which the agent is located, a first action sequence of the agent is generated. In response to a received user instruction, the user instruction is parsed to generate instruction intent information; The instruction intent information, the first environmental feature data, and the first action sequence are input into the first controller. The first encoder in the first controller constructs a conditional latent space based on the instruction intent information and the first environmental feature data. The first decoder in the first controller reparameterizes the first action sequence in the conditional latent space to generate latent variables associated with the first action sequence. The first decoder generates a control signal based on the latent variables and the conditional information of the conditional latent space. The control signal, the first action sequence, and the first environmental feature data are input into the action integrator. Through the feature fusion network in the integrator, the spatiotemporal features between the control signal, the first action sequence, and the first environmental feature data are dynamically associated based on the temporal attention mechanism to generate a joint feature vector. Then, through the action optimization network in the integrator, an action correction amount is generated based on the joint feature vector, and the first action sequence is corrected based on the action correction amount to generate the agent's second action sequence. Based on the second action sequence, the agent is controlled to perform actions.

2. The method according to claim 1, characterized in that, The first regulator is trained based on a target loss, which is calculated in the following way: Calculate the reconstruction sub-loss, which reflects the consistency between the control signal output by the first controller and the desired output signal of the first controller; Calculate the potential space loss, which reflects the consistency between the first distribution of the control signal output by the first controller and the second distribution of the desired output signal of the first controller; Calculate the smoothing loss, which is used to suppress sudden changes in the control signal output by the first controller.

3. The method according to claim 1, characterized in that, The method further includes: Obtain the permissible action correction amount associated with the first action sequence; The step of regulating the first action sequence based on the regulation signal to generate the second action sequence of the agent includes: Based on the minimum of the control signal and the allowed action correction amount, the first action sequence is controlled to generate the agent's second action sequence.

4. The method according to claim 1, characterized in that, The instruction intent information includes expected action parameters, and the method further includes: The desired action parameters are abstracted and expressed to generate abstract functional features corresponding to the action parameters; Based on the abstract functional features, the first environmental feature data, and the first action sequence, a control signal is generated.

5. The method according to claim 1, characterized in that, The action integrator also includes a dynamic weight decision network, and the first action sequence includes the first action parameters of the target action part of the agent in the target degree of freedom. Before correcting the first action sequence based on the action correction amount, the method further includes: The dynamic weight decision network generates a first corrected weight for the target action part in the target degree of freedom based on the joint feature vector. The step of modifying the first action sequence based on the action modification amount through the action optimization network includes: The action optimization network is used to correct the first action parameters based on the action correction amount and the first correction weights to obtain the second action parameters. The second action sequence includes the second action parameters.

6. The method according to claim 5, characterized in that, The step of modifying the first action parameters using the action optimization network based on the action correction amount and the first correction weight to obtain the second action parameters includes: Calculate the first product between the first action parameter and the first correction weight, and calculate the second product between the action correction amount and the second correction weight, wherein the second correction weight is obtained by subtracting the first correction weight from 1; The sum of the first product and the second product is determined as the second action parameter after the target degrees of freedom are corrected.

7. The method according to claim 1, characterized in that, The user instructions include instructions in multiple modalities. The step of responding to a received user instruction by parsing the user instruction to generate instruction intent information includes: For each of the multiple modalities, based on the instruction parsing mechanism corresponding to the modality, the instruction of the modality is parsed to obtain the intent feature vector of the modality; The intent feature vectors of multiple modalities are mapped to the same semantic space, and key instruction components of the intent feature vectors are dynamically extracted in the semantic space through a multi-head attention mechanism to generate instruction intent information.

8. A control device for an intelligent agent, characterized in that, include: An action sequence generation module is used to generate a first action sequence of the agent based on first environmental feature data of the environment in which the agent is located; The intent parsing module is used to respond to a received user instruction, parse the user instruction into intent, and generate instruction intent information. The control signal generation module is used to input the instruction intent information, the first environmental feature data, and the first action sequence into the first controller. Through the first encoder in the first controller, a conditional latent space is constructed based on the instruction intent information and the first environmental feature data. Through the first decoder in the first controller, the first action sequence is reparameterized in the conditional latent space to generate latent variables associated with the first action sequence. Through the first decoder, a control signal is generated based on the latent variables and the conditional information of the conditional latent space. The control module is used to input the control signal, the first action sequence, and the first environmental feature data into the action integrator. Through the feature fusion network in the integrator, the spatiotemporal features between the control signal, the first action sequence, and the first environmental feature data are dynamically associated based on the temporal attention mechanism to generate a joint feature vector. Through the action optimization network in the integrator, an action correction amount is generated based on the joint feature vector, and the first action sequence is corrected based on the action correction amount to generate the second action sequence of the agent. An action execution module is used to control the agent to perform actions based on the second action sequence.