Data processing systems, methods, apparatus, and storage media for embodied agents

Abstract

According to embodiments of the present disclosure, a data processing system, method, device and storage medium for embodied agent are provided. The system comprises: a storage engine configured to store data related to an embodied agent, the data comprising at least multi-modal data, the multi-modal data comprising visual information, linguistic information and action information; an embodiment engine configured to train a machine learning model for the embodied agent based on the multi-modal data to obtain a trained machine learning model; a simulation engine configured to provide a simulation environment and a virtual object corresponding to the embodied agent in the simulation environment, and generate a simulation result based on an interaction between the virtual object and the machine learning model; and an evaluation engine configured to evaluate the machine learning model based at least on the simulation result to obtain a first evaluation result for the machine learning model.

Description

Technical Field

[0001] The exemplary embodiments disclosed herein generally relate to the field of information technology, and particularly to data processing systems, methods, apparatuses, computer-readable storage media, and computer program products for embodied intelligent agents. Background Technology

[0002] Driven by both artificial intelligence and robotics technologies, Embodied Artificial Intelligence (EAI) has experienced rapid development. Embodied AI integrates artificial intelligence into physical entities (such as humanoid robots, quadruped robots, or robotic arms) to create intelligent agents with perception, decision-making, and execution capabilities. Summary of the Invention

[0003] In a first aspect of this disclosure, a data processing system for an embodied agent is provided. The system includes: a storage engine configured to store data related to the embodied agent, the data including at least multimodal data, the multimodal data including visual information, language information, and action information; an embodiment engine configured to train a machine learning model for the embodied agent based on the multimodal data to obtain a trained machine learning model; a simulation engine configured to provide a simulation environment and virtual objects located in the simulation environment corresponding to the embodied agent, and to generate simulation results based on the interaction between the machine learning model and the virtual objects; and an evaluation engine configured to evaluate the machine learning model at least based on the simulation results to obtain a first evaluation result for the machine learning model.

[0004] In a second aspect of this disclosure, a data processing method is provided. The method includes: acquiring data related to an embodied agent, the data including at least multimodal data, the multimodal data including visual information, language information, and motion information; training a machine learning model for the embodied agent based on the multimodal data to obtain a trained machine learning model; providing a simulation environment and a virtual object located in the simulation environment corresponding to the embodied agent; generating simulation results based on the interaction between the machine learning model and the virtual object; and evaluating the machine learning model at least based on the simulation results to obtain a first evaluation result for the machine learning model.

[0005] In a third aspect of this disclosure, an electronic device is provided. The electronic device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method of the second aspect of this disclosure when executed by the at least one processing unit.

[0006] In a fourth aspect of this disclosure, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program that can be executed by a processor to perform the method according to a second aspect of this disclosure.

[0007] In a fifth aspect of this disclosure, a computer program product is provided, which is tangibly stored in a computer storage medium and includes computer-executable instructions that, when executed by a device, cause the device to perform the method according to a second aspect of this disclosure.

[0008] It should be understood that the content described in this section is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0009] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein: Figure 1A A block diagram illustrating the application environment of embodied intelligent agents is shown; Figure 1B A schematic diagram of an example architecture for a machine learning model used in embodied agents is shown. Figure 2 A schematic diagram of an example architecture of a data processing system according to some embodiments of the present disclosure is shown; Figure 3 A schematic diagram of an example architecture of a data processing system according to other embodiments of the present disclosure is shown; Figure 4 A flowchart illustrating a data processing procedure according to some embodiments of the present disclosure is shown; and Figure 5 A block diagram of an electronic device in which one or more embodiments of the present disclosure may be implemented is shown. Detailed Implementation

[0010] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0011] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0012] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0013] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0014] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0015] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0016] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may also be included below.

[0017] As used in this paper, the term "model" refers to a model that learns the relationship between inputs and outputs from training data, enabling it to generate corresponding outputs for a given input after training. Model generation can be based on machine learning techniques. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs using multiple layers of processing units. A neural network model is an example of a deep learning-based model. In this paper, "model" may also be referred to as a "machine learning model," "learning model," "machine learning network," or "learning network," and these terms are used interchangeably.

[0018] A neural network is a machine learning network based on deep learning. A neural network processes input and provides a corresponding output, typically consisting of an input layer, an output layer, and one or more hidden layers between the input and output layers. Neural networks used in deep learning applications often include many hidden layers, thus increasing the network's depth. The layers of a neural network are connected sequentially, so that the output of the previous layer is provided as the input to the next layer. The input layer receives the input to the neural network, while the output layer's output serves as the final output. Each layer of a neural network includes one or more nodes (also called processing nodes or neurons), each node processing the input from the layer above.

[0019] Machine learning typically comprises three phases: training, testing, and application (also known as inference). In the training phase, a given model is trained using a large amount of training data, iteratively updating its parameter values ​​until the model can consistently generate inferences that meet the expected goals from the training data. Through training, the model can be considered to have learned the relationship between inputs and outputs (also known as the input-output mapping) from the training data. The parameter values ​​of the trained model are determined. In the testing phase, test inputs are applied to the trained model to test whether it can provide the correct output, thus determining the model's performance. In the application phase, the model can be used to process actual inputs based on the trained parameter values ​​to determine the corresponding output.

[0020] As mentioned above, driven by both artificial intelligence (AI) and robotics, embodied artificial intelligence (EAI) has developed rapidly. Embodied AI integrates artificial intelligence into physical entities to form intelligent agents with perception, decision-making, and execution capabilities. These physical entities can include, but are not limited to, humanoid robots, wheeled / tracked robots, quadruped robots, or robotic arms. For ease of description, these physical entities will be collectively referred to as robots or robotic devices in the following text.

[0021] In some traditional technologies, the embodied intelligence development process mainly includes stages such as simulation rendering, model training, performance evaluation, and actual deployment, typically proceeding sequentially. The connections between these stages often rely on manual intervention or loosely coupled scripts, lacking a unified runtime abstraction mechanism and system-level scheduling capabilities. Especially in large-scale cloud training and multi-robot collaborative operation scenarios, the independent and loosely coupled operating environments of each stage prevent efficient integration and scheduling of cloud computing resources, resulting in low computing power utilization and relatively high difficulty in subsequent maintenance and upgrades.

[0022] In some traditional technologies, the toolchains and platforms upon which embodied intelligence development relies are often siloed, with inconsistent interface standards for data generation, model training, and actual deployment, resulting in severe heterogeneous interface matching challenges. This leads to data flow bottlenecks and cumbersome task orchestration during the R&D process, significantly hindering development efficiency. Furthermore, traditional architectures generally treat model training, online inference, and 3D rendering as independent subsystems or discrete stages, lacking a unified runtime abstraction layer for overall coordination. This fragmentation prevents the formation of a closed-loop system from model training to simulation environment rendering to performance evaluation, reducing the coherence and completeness of the development process. While a model may perform excellently in the training environment, its performance may significantly degrade when migrated to a real-world environment (Simulation-to-Reality Transfer) or a simulation environment due to environmental differences and inconsistencies in the transitions between stages.

[0023] In some traditional technologies, simulation platforms primarily focus on visual presentation and dynamic simulation, lacking the ability to jointly model multi-dimensional and multi-modal structures. In other words, traditional simulation platforms lack the ability to jointly model multi-modal intermediate representations (such as hidden states, policy distributions, and world models), resulting in simulation environments that cannot comprehensively and accurately simulate real-world environments. Traditional simulation platforms also lack universal motion redirection mechanisms for different objects (such as humans, robotic arms, and mobile robots), limiting motion transfer between different objects and restricting the applicability of the simulation environment. Furthermore, traditional simulation platforms lack calibration mechanisms between the simulation environment and the real environment (i.e., the physical environment), leading to relatively poor consistency between the simulation and physical environments and affecting the reliable transfer of models from the simulation environment to the physical environment.

[0024] In some traditional technologies, data processing systems for embodied intelligent agents are typically deployed on local devices or partially in the cloud. This deployment model makes it impossible to improve data generation and simulation efficiency through parallel computing, and it is difficult to flexibly allocate and dynamically adjust computing resources for tasks of different scales and complexities. Scheduling strategies for various computing resources need optimization, resulting in relatively low utilization efficiency of computing resources.

[0025] In some traditional technologies, performance evaluation is often deployed offline, lacking an efficient feedback mechanism with the training system. Evaluation results cannot guide subsequent model training and optimization in a timely and accurate manner, making it difficult to form a closed-loop optimization mechanism of performance evaluation—result feedback—retraining. This results in low efficiency of iterative model optimization and limits its performance improvement.

[0026] In view of this, an improved solution for a data processing system for embodied agents is provided in the embodiments of this disclosure. In this improved solution, the data processing system includes a storage engine, an embodiment engine, a simulation engine, and an evaluation engine. The storage engine is configured to store data related to the embodied agent, the data including at least multimodal data, which includes visual information, language information, and action information. The embodiment engine is configured to train a machine learning model for the embodied agent based on the multimodal data to obtain a trained machine learning model. The simulation engine is configured to provide a simulation environment and virtual objects corresponding to the embodied agent located within the simulation environment, and to generate simulation results based on the interaction between the machine learning model and the virtual objects. The evaluation engine is configured to evaluate the machine learning model at least based on the simulation results to obtain a first evaluation result for the machine learning model.

[0027] According to embodiments of this disclosure, the data processing system for embodied intelligence includes a storage engine, an embodiment engine, a simulation stealth engine, and an evaluation engine, supporting a complete process of data acquisition, model training, simulation environment construction, and performance evaluation. This improves the system's scheduling capabilities for efficient task collaboration, enhances the coupling between different stages, ensures smooth data flow between stages, and improves the utilization rate of computing resources.

[0028] To facilitate understanding, the following text will first combine... Figure 1A and Figure 1B This paper provides an illustrative introduction to embodied agents, their application environments, and machine learning models used for embodied agents.

[0029] Figure 1A A block diagram 100A illustrates the application environment of an embodied intelligent agent. For example... Figure 1A As shown, the embodied intelligent agent can be implemented as a robotic device 110. The robotic device 110 and the user 120 can be located in the environment 160, and the user 120 can control the robotic device 110 to perform various tasks.

[0030] Environment 160 may include, but is not limited to, one or more objects. For example, in a kitchen / dining environment, environment 160 may include, but is not limited to, various items, such as edible food, kitchen utensils, etc., or a combination of one or more of the above. In a wardrobe environment, environment 160 may include, but is not limited to, various types of clothing. Environment 160 may include, but is not limited to, one or more rooms. For example, in a home environment, environment 160 may include, but is not limited to, a living room, bedroom, study, kitchen, toilet, etc., or a combination of one or more of the above.

[0031] like Figure 1A As shown, user 120 can instruct robot device 110 to manipulate various objects in environment 160 and complete corresponding tasks. Here, objects can be various items in environment 160. For example, user 120 can instruct robot device 110 to find a specific object in environment 160. Or, user 120 can instruct robot device 110 to place the found object in a designated location, and so on. In some examples, user 120 can instruct robot device 110 to perform a task in environment 160. For example, in a life service scenario, robot device 110 can perform tasks such as indoor cleaning, item organization, home control, and voice interaction in environment 160. In an industrial manufacturing scenario, robot device 110 can perform tasks such as industrial inspection and production operations in environment 160. However, these are merely examples, and this disclosure does not limit the scope of the invention.

[0032] like Figure 1A As shown, the robot device 110 may include multiple parts. For example, the control unit 111 can serve as the control center of the robot device 110, and an application can be loaded into the control unit 111 to control the various parts of the robot device. The user 120 can use the interaction unit 112 to interact with the robot device 110, for example, by inputting control commands to the robot device 110 to perform desired tasks. The robot device 110 may include an arm 113 for performing actions such as grasping and releasing. For example, the arm 113 can grasp an object and move it to a desired position, and so on.

[0033] Alternatively and / or additionally, the robot device 110 may also include a data acquisition unit 114. Here, the data acquisition unit 114 may include various types, such as an image acquisition unit, a sound acquisition unit, etc. Alternatively and / or additionally, the robot device 110 may further include a sensing unit for detecting surrounding objects, for example, detecting the distance between the robot and surrounding objects based on laser light, etc. The robot device 110 may also include a drive unit 115, for example, the drive unit 115 can drive the robot device 110 to move along a desired path.

[0034] Environment 160 may include one or more acquisition units 130, ..., and 132. For example, one or more image acquisition devices may be deployed in a room to acquire images of the room from various angles. Environment 160 may include a control device 140, which can control one or more acquisition units 130, ..., and 132, etc., via a network (not shown). Alternatively and / or additionally, in a smart home environment, control device 140 can control various electrical devices in environment 160.

[0035] Alternatively and / or additionally, a machine learning model (e.g., machine learning model 150) may be provided to manage environment 160. It should be understood that, although... Figure 1A The machine learning model 150 is shown to be located within environment 160, but it can also be deployed in robotic device 110. Alternatively and / or additionally, the machine learning model 150 can also be located at a remote device outside environment 160, and the control device 140, robotic device 110, or other device can access the remote machine learning model 150 via a network.

[0036] Machine learning model 150 may include one or more models. If machine learning model 150 includes multiple models, these multiple models may include multiple types of models. In some examples, machine learning model 150 may include a Vision-Language-Action (VLA) model. A VLA model is a multimodal model that integrates vision, language, and action. Machine learning model 150 may, for example, include at least a Vision-Language Model (VLM) and an action model. The Vision-Language Model has the ability to process images and natural language. The Action Model can control the robot device 110 to perform various actions. Machine learning model 150 may also include, for example, an image processing model, a text processing model, etc. Alternatively, machine learning model 150 may also include a World Model. A World Model is a generative model that, through learning and analyzing environmental data, can predict the impact of the robot device 110's actions on the environment. Planning for environmental changes and action consequences in the potential space supports the robot device 110's action decisions.

[0037] In some embodiments, the machine learning model 150 may include a content-generating model capable of generating corresponding outputs based on model inputs. In some implementations, the machine learning model may receive text-modal model inputs (e.g., natural language and / or machine language) and / or non-text-modal model inputs (e.g., images, speech, video, etc.), and may obtain corresponding model outputs based on the model inputs and prompts, thereby completing the task execution.

[0038] Machine learning model 150 may include an encoder module 151, a backbone network 152, and a decoder module 156. The encoder module 151 can perform feature encoding on the model input (e.g., encoding an image or video into a feature space). The encoder module 151 may include, but is not limited to, a video encoder, a speech encoder, and an action encoder. The backbone network 152 can perform feature extraction on the encoded model input to obtain a feature representation 155 for performing a task. In some embodiments, the backbone network 152 may include one or more attention modules (e.g., transformer blocks) capable of performing attention-based processing on the encoded model input to extract the feature representation 155. The decoder module 156 then decodes the output feature representation 155 to obtain the model output. The decoder module may include, for example, an action decoder. For example, for a VLA model, the model output may be, for example, the next action the robot will perform. It should be understood that... Figure 1A The illustration shows an example application of machine learning model 150, which is not limited in this disclosure.

[0039] Figure 1B A schematic diagram of an example architecture for a machine learning model 150 for embodied agents is shown. Figure 1B As shown, the machine learning model 150 of an embodiment of this disclosure may include a VLA model. A VLA model is a multimodal model that integrates vision, language, and action. Given visual input 153a (e.g., an image or video, etc.) and language input 153b (e.g., text or voice commands) in the environment space (e.g., environment 160) where the robot device 110 is located, the VLA model can output action commands 157. The action commands 157 can be used to control the robot device 110 to perform the task requested by the visual input 153a or the language input 153b.

[0040] In some embodiments, such as Figure 1B As shown, the VLA model may include a visual encoder 151a, a language encoder 151b, a backbone network 152, and an action decoder 156c. The visual encoder 151a performs feature encoding on the visual input 153a (e.g., encoding an image or video into a feature space) to obtain a visual representation 154a. The language encoder 151b performs feature encoding on the language input 153b, such as text commands or voice commands, to obtain a language representation 154b. The backbone network 152 performs feature extraction based on the visual representation 154a and the language representation 154b to obtain a feature representation 155 for action decision-making. Then, the action decoder 156c performs feature decoding on the feature representation 155 to obtain an action command 157 (or a sequence of action commands).

[0041] The motion command 157 here can be implemented in several ways. As an example, motion command 157 can indicate the relative change in the end-effector posture of the robotic arm (i.e., robot device 110). Based on this relative change using the inverse dynamics model of the robotic arm, motion parameters such as angles, velocities, and accelerations for multiple degrees of freedom of the robotic arm (e.g., joints or end-effector grippers) can be determined. Alternatively, motion command 157 can also indicate the motion parameters for multiple degrees of freedom of the robot device 110. In other words, motion decoder 156c can also directly output the motion parameters for specific degrees of freedom of the robot device 110, allowing the robot device 110 controller to directly drive its movement based on these motion parameters.

[0042] In some embodiments, the VLA model may further include a motion encoder 151c. The model input of the VLA model may also include motion input 153c, which may include, but is not limited to, motion commands 157 provided to the robot device 110 in the previous control cycle or motion sensing results of the robot device 110 in the previous control cycle. The motion encoder 151c may perform feature encoding on the motion input 153c to obtain a motion representation 154c. The backbone network 152 may perform feature extraction based on the visual representation 154a, language representation 154b, and motion representation 154c to obtain a feature representation 155. Of course, the model input of the VLA model described above is merely exemplary. In practical applications, the model input of the VLA model may also include, for example, depth maps, point clouds, force feedback information, or haptic feedback information. In this case, the VLA model may also include, for example, a depth map encoder, a point cloud encoder, a force encoder, or a haptic encoder.

[0043] In some embodiments, the backbone network 152 can perform multimodal fusion and feature extraction on the visual representation 154a, language representation 154b, and action representation 154c based on a cross-attention mechanism to obtain feature representation 155. In some embodiments, the visual encoder 151a, language encoder 151b, and backbone network 152 can be implemented based on a pre-trained VLM model. In other words, a VLA model can be constructed by combining the action encoder 151c and action decoder 156c on the basis of a pre-trained VLM model. Of course, the backbone network 152 described above is only exemplary. In practical applications, any other suitable multimodal model can be used to construct the backbone network 152. The embodiments of this disclosure do not limit this.

[0044] It should be understood that Figure 1A and Figure 1B The embodied agent, application environment, and machine learning model shown are merely examples. In real-world applications, any other suitable embodied agent or machine learning model can be selected based on actual needs. Implementations of this disclosure are not limited in this respect.

[0045] The following further combines Figure 2 and Figure 3 Various example implementations of the data processing system of the embodiments of this disclosure will be described in detail below.

[0046] Figure 2 A schematic diagram of an example architecture 200 for a data processing system for embodied intelligence according to some embodiments of the present disclosure is shown. Figure 2As shown, the data processing system of the embodiments of this disclosure includes a storage engine 210, a simulation engine 220, an embodied engine 230, and an evaluation engine 240.

[0047] In some examples, at least one of the storage engine 210, simulation engine 220, embodiment engine 230, and evaluation engine 240 can be deployed on a cloud device. As an example, the cloud device may include, for instance, a Model DevCloud Workstation, where the storage engine 210, simulation engine 220, embodiment engine 230, and evaluation engine 240 can all be deployed. In this way, on the one hand, a cloud workstation covering data acquisition, simulation environment generation, model training, and performance evaluation can be formed, facilitating collaborative scheduling and smooth data flow between the various engines, avoiding problems such as toolchain fragmentation, data silos, and poor collaboration, and improving the development efficiency of embodied intelligence. On the other hand, this cloud workstation can provide a unified runtime environment for the storage engine 210, simulation engine 220, embodiment engine 230, and evaluation engine 240. Different types of task dependencies, drivers, etc., can be integrated into the runtime environment to ensure that different types of tasks such as training, inference, and rendering can run stably in the same runtime environment, eliminating compatibility issues caused by environmental differences. It should be understood that the deployment methods of the various engines described above are merely exemplary. In practical applications, at least a portion of the data processing system can be deployed on, for example, edge devices or other devices. For instance, components with high computing power requirements (such as storage engine 210, simulation engine 220, and embody engine 230) but low latency requirements can be deployed on cloud devices to fully utilize their computing resources. Components with high latency requirements (such as the evaluation engine) can be deployed on edge devices to reduce network latency and bandwidth consumption during data transmission, thereby lowering response latency.

[0048] The following will provide a detailed description of each component of the data processing system, with reference to the accompanying drawings. For example... Figure 2 As shown, the storage engine 210 can be configured to store data related to the embodied agent. The data includes at least multimodal data, which may include, but is not limited to, visual information, tactile information, language information, and motion information. For example, the storage engine 210 may include a database 215, in which the multimodal data can be stored.

[0049] In some examples, storage engine 210 can acquire at least one of first sensed data of the physical environment (also referred to as, for example, "real environment data") and second sensed data of the simulated environment (also referred to as, for example, "simulated environment data"). Then, storage engine 210 can obtain multimodal data based on at least one of the first sensed data or the second sensed data. As an example, storage engine 210 can receive the first sensed data and perform preprocessing on it, such as noise reduction, deduplication, and format normalization, to obtain multimodal data. In this way, data from different sources and / or in different formats can be converted into a system-recognizable data format, ensuring data consistency and availability.

[0050] In some examples, storage engine 210 may also store model parameters of machine learning model 150. That is, data related to the embodied agent may also include model parameters of machine learning model 150. For example, after machine learning model 150 has been trained, storage engine 210 may receive model parameters of machine learning model 150 from embodied engine 230, and storage engine 210 may save the model parameters of machine learning model 150 to database 215. Of course, during the training process of machine learning model 150, embodied engine 230 may also store model parameters of machine learning model 150 in an intermediate state in storage engine 210. Alternatively and / or additionally, storage engine 210 may store simulation results and simulation data provided by simulation model 220, where the simulation data indicates at least one of the simulation environment or virtual object. Alternatively and / or additionally, storage engine 210 may also store, for example, evaluation results provided by evaluation model 240. It is understandable that the storage engine 210 is not limited to storing data, but can also support other engines (such as simulation engine 220, embodied engine 230, evaluation engine 240) to retrieve or read the required data from the storage engine 210, and at the same time write the processed data back to the storage engine to realize the full life cycle management of data.

[0051] As an example, such as Figure 2As shown, cloud devices may include, for example, a cloud infrastructure layer 260, a data environment layer 270, and a model layer 280. The cloud infrastructure layer 260 may include a distributed storage system and a computing resource pool. The distributed storage system may adopt a hybrid storage architecture combining distributed file storage and object storage. The distributed storage system can possess high capacity, high reliability, and high read / write performance to support secure storage and efficient access to petabyte (PB) level data. The distributed storage system can support data backup and recovery mechanisms, ensuring data security and availability through multi-replica storage, off-site backup, etc., while supporting fast read / write and parallel access to meet the needs of large-scale data processing. The computing resource pool can integrate high-performance heterogeneous computing resources to build a scalable computing resource pool. The computing resource pool can support on-demand allocation and dynamic adjustment, scheduling computing resources according to the computing power requirements of tasks to achieve centralized management and efficient utilization of computing resources. The computing resource pool can possess good compatibility and scalability to support rapid access and integration of computing resources. Heterogeneous computing power here refers to a computing power cluster composed of computing resources of different types and architectures. These computing resources have different characteristics in terms of computing performance and applicable scenarios, and can work together to complete a variety of computing tasks.

[0052] Storage engine 210 can be deployed in data environment layer 270. Data environment layer 270 may also include a data acquisition module, which can acquire first-sensor data related to the physical environment (also referred to as, for example, the "real environment"). The first-sensor data may include, but is not limited to, visual information, tactile information, voice information, or motion information. The data acquisition module may, for example, include a multimodal data interface. The multimodal data interface may include, for example, a visual information interface, a tactile information interface, a voice information interface, a motion information interface, etc. The visual information interface may be coupled to a visual data acquisition device, such as a camera or a depth camera, to receive visual data. The tactile information interface may be coupled to, for example, a pressure sensor or a tactile sensor array, to receive tactile data. The voice information interface may be coupled to an audio acquisition device, such as a microphone array, to receive voice data. The motion information interface may be coupled to a device, such as an inertial measurement unit or a motion capture system, to receive motion data.

[0053] In some examples, the data environment layer 270 may include a data management module. The data management module may be coupled to the storage engine 210. It can add indexes to multimodal data and perform retrieval and updates of the multimodal data. The data management module can employ a data version control mechanism to record data modification and usage records, supporting data rollback and recovery. It can also support data access control to ensure secure access and use of data.

[0054] like Figure 2 As shown, the simulation engine 220 can be configured to provide a simulation environment and virtual objects within that environment corresponding to an embodied agent (e.g., robot device 110), and generate simulation results based on the interaction between the machine learning model 150 and the virtual objects. A virtual object refers to a digital entity constructed in the simulation environment that corresponds to an embodied agent. For example, if the embodied agent is robot device 110, the virtual object can be a simulated robot device. It is understood that the virtual object may differ depending on the specific agent. The simulation results are used to record feedback information generated by the interaction between the machine learning model 150 and the virtual objects in the simulation environment. For example, the machine learning model 150 can be used to control the virtual robot device to perform operations (e.g., grasping or carrying) on ​​simulated objects in the simulation environment. The simulation results may include, for example, action commands provided by the machine learning model 150, the motion trajectory of the virtual robot device, changes in the simulation environment, etc. In some examples, the simulation engine 220 can be deployed at the data environment layer 270.

[0055] In some examples, simulation engine 220 can receive simulation requests. The simulation request may include configuration parameters for at least one of the simulation environment or virtual objects. Simulation engine 220 can then perform rendering operations based on the configuration parameters to generate at least one of the simulation environment or virtual objects, providing an environment for the simulation training and evaluation of machine learning model 150. As an example, configuration parameters may include physical parameters related to the simulation environment. Simulation engine 220 can provide a physical parameter adjustment interface. Users can set physical parameters such as gravity, friction, and elastic coefficient in the simulation environment. Simulation engine 220 can obtain physical parameters through the adjustment interface and perform rendering operations on the simulation environment based on the physical parameters. Of course, configuration parameters are not limited to physical parameters; they may also include any appropriate parameters related to the simulation environment. Embodiments of this disclosure do not limit this. Furthermore, the adjustment interface can also support the rapid addition, deletion, and modification of environmental elements in the simulation environment to improve the efficiency and flexibility of simulation environment construction.

[0056] In some examples, simulation engine 220 may include multiple simulation nodes. In response to a simulation request for a simulation environment, simulation engine 220 may determine multiple simulation tasks corresponding to the simulation environment. Simulation engine 220 may assign multiple simulation tasks to multiple simulation nodes, utilizing the multiple simulation nodes to execute the multiple simulation tasks in parallel to generate the simulation environment. As an example, simulation engine 220 may employ a task-parallel and data-parallel architecture. Simulation engine 220 may also ensure the coordinated operation of each simulation node based on communication and data synchronization mechanisms to efficiently and concurrently perform simulation rendering.

[0057] Figure 3 A schematic diagram of an example architecture 200 of a data processing system according to other embodiments of the present disclosure is shown. Figure 3 As shown, the simulation engine 220 may include one or more physics simulation tools 302, such as physics simulation tools 302-1, 302-2, ..., 302-N, etc., where N is a positive integer. For ease of description, the physics simulation tools are collectively referred to as physics simulation tools 302 or individually in this document. Simulation tools 302 can be used to model one or more physical fields in a simulation environment. These physical fields may include, but are not limited to, mechanical fields, electromagnetic fields, thermal fields, and acoustic fields. The simulation engine 220 uses at least one physics simulation tool 302 to perform rendering operations on the simulation environment. This reduces computational interference between different physical fields and improves the accuracy and efficiency of simulation calculations. It is understood that the simulation engine 220 can also use at least one physics simulation tool 302 to render virtual objects corresponding to embodied intelligent agents in the simulation environment.

[0058] As an example, such as Figure 3 As shown, the data environment layer 270 may also include a physics library 310. The physics library 310 may include multiple physics engines. Based on these multiple physics engines, the physics library 310 can support the simulation engine in generating simulation environments and can also construct diverse task scenarios according to task requirements (such as the grasping, moving, and assembling of the robot device 110). The simulation engine 220 can utilize multiple physics engines (for example, by solving physical equations) to perform simulation and control of virtual objects (such as virtual robot devices). This allows the behavior of virtual objects to conform to the physical laws of the real world. Alternatively, the physics library 310 may also be built based on a single physics engine. The simulation engine 220 can then perform simulations of virtual objects or simulation environments based on this single physics engine. This allows for lightweight simulations to be performed from a single physical dimension, which helps reduce system development and maintenance costs.

[0059] In some examples, simulation engine 220 can store simulation data indicating the simulation environment in storage engine 210. For instance, for recurring simulation environments, simulation engine 220 can save the simulation data to storage engine 210. If the same or similar simulation environment needs to be used again, simulation engine 220 can load the simulation data from storage engine 210 to reuse the simulation environment, or simulation engine 220 can modify or optimize the simulation data to provide a similar simulation environment. This avoids redundant calculations and improves simulation efficiency.

[0060] Return to combination Figure 2As shown, the Embodied Engine 230 is configured to train the machine learning model 150 based on multimodal data to obtain the trained machine learning model 150. The machine learning model 150 here may include any suitable model that can be used for the embodied agent, such as including but not limited to the VLA model or the VLM model.

[0061] In some examples, such as Figure 3 As shown, the embodied engine 230 may include an action redirection module 312. The embodied engine 230 can utilize the action redirection module 312 to obtain action information, which indicates a reference action to be performed by the source object. The source object can be an object in a real environment (e.g., a person, an animal, etc.) or a virtual object in a simulation environment. The action redirection module 312 can perform action redirection based on the action information to obtain a truth value action instruction. This truth value action instruction is used to drive the embodied agent to perform an action that approximates the reference action. In some cases, such as... Figure 3 As shown, the action redirection module 312 can perform action redirection using at least one physics engine in the physics library 310. In this way, the generated truth action instructions can conform to physical constraints.

[0062] As an example, the motion redirection module 312 can obtain motion information related to, for example, humans and robot devices 110 from the storage engine 210. The motion redirection module 312 can extract key features (such as joint angles, motion trajectories, velocity, and acceleration) from the human and robot device 110 motion information. Based on these key features, the motion redirection module 312 can construct a mapping model from human motion information to the execution space of the robot device 110. This mapping model can support motion redirection for robot devices 110 of different types and structures. Through real-time motion capture and feature matching, human motion information is efficiently and accurately mapped to the execution space of the robot device 110 to achieve motion transfer of heterogeneous entities.

[0063] As another example, such as Figure 3As shown, the data environment layer 270 may also include a data synthesis module 308. The data synthesis module 308 can construct a data synthesis model based on physical laws. The data synthesis module 308 can receive simulation data from the simulation module 220 and use the data synthesis model to transform the simulation data, obtaining transformed simulation data. The action redirection module 312 can extract action information related to virtual objects from the transformed simulation data. In this way, the transformed simulation data can conform to the physical laws of the real world (such as energy conservation, momentum conservation, the law of friction, etc.), improving the quality of training samples. In practical application scenarios, the data synthesis module 308 can analyze real environment data and simulation environment data, and perform iterative training on the data synthesis model to optimize its performance, generating simulation data highly adapted to real-world scenarios, thus improving the reliability and effectiveness of the simulation data.

[0064] In some examples, such as Figure 3 As shown, the embodied engine 230 may include a training system 320. The training system 320 may construct training samples 314 based on visual information, language information, and ground truth action instructions. The training samples 314 may include sample model inputs and ground truth action instructions, with the sample model inputs including at least sample visual inputs and sample language inputs. The training system 320 may provide the sample inputs to the machine learning model 150 to be trained to obtain predicted action instructions. Subsequently, the training system 320 may update the model parameters of the machine learning model 150 based on the difference between the predicted action instructions and the ground truth action instructions to obtain a trained machine learning model 150.

[0065] In some examples, continue to combine Figure 3 As shown, the training system 320 may include a pre-training system 322. The pre-training system 322 can construct pre-training samples based on multimodal data, and use these pre-training samples to train the machine learning model 150 to obtain a pre-trained machine learning model 150. For example, the pre-training system 322 can build a distributed pre-training framework that supports parallel input and processing of large-scale data. By employing a vision-language-action joint pre-training method, the machine learning model 150 learns the relationships between visual information, language instructions (i.e., language information), and action execution (i.e., action information), thus constructing a machine learning model 150 with general perception, understanding, and execution capabilities. During pre-training, a training method combining parallel processing of the machine learning model 150 and data parallelism can be adopted to fully utilize heterogeneous computing resources in the cloud and improve training efficiency.

[0066] In some examples, such as Figure 3As shown, the training system 320 may include a post-training system 322. The post-training system 322 can, at least based on simulation results, construct post-training samples and use these samples to train the pre-trained machine learning model 150 to obtain a trained machine learning model. As an example, the embodied engine 230 can interact with the simulation environment using the pre-trained machine learning model 150 to obtain simulation results generated by the simulation engine 220. The post-training system 322 can then construct post-training samples based on these simulation results. During the post-training phase, the simulation engine 220 can utilize multiple physics simulation tools to construct a high-fidelity simulation environment, building diverse task scenarios (such as object grasping, path planning, complex operations, etc.). Learning strategies are selected according to task requirements, and the pre-trained machine learning model 150 is fine-tuned. In this way, the action strategy of the machine learning model 150 can be optimized, improving the accuracy and efficiency of the machine learning model 150 in specific tasks. During the post-training phase, the embodied engine 230 can interact with the simulation environment using the machine learning model 150 to detect performance changes in the machine learning model 150. The post-training system 322 can adjust training parameters (such as learning rate, batch size, etc.) based on the detection results to ensure the training effect of the machine learning model 150.

[0067] In some examples, the embodied engine 230 may include an execution module that can receive task requests for the embodied agent. The task request includes at least visual input and natural language-based input, with the visual input describing the environment relevant to the embodied agent. The execution module can utilize a trained machine learning model 150 to determine action instructions based at least on the visual and language inputs. The action instructions drive the embodied agent to perform the task corresponding to the task request. In some cases, the execution module may include a first task mode and a second task mode. In the first task mode, the execution module can respond to the task request by invoking the machine learning model 150 in real time to determine the action instructions. This is suitable for applications with low latency requirements, enabling rapid response to task requests and feedback of results (i.e., action instructions). In the second task mode, the execution module can respond to the task request based on the load status of the data processing system. Specifically, if the load metric of the data processing system is below a threshold, the execution module can respond to the task request by using the machine learning model 150 to determine the action instructions. If the load metric exceeds the threshold, the execution module can continue to monitor the load metric of the data processing system until the load metric falls below the threshold. The second task mode is suitable for batch inference tasks and can make full use of idle computing resources to improve inference efficiency.

[0068] In some examples, the training system 320 may also include an incremental training system. The incremental training system can construct incremental training samples based on the task execution results of the execution module. Based on these incremental training samples, the incremental training system can train the machine learning model 150 to obtain an incrementally trained machine learning model 150. As an example, the incremental training system can construct an inference data feedback channel to acquire interaction trajectory data (such as robot motion information, environmental feedback data, and data indicating task completion status) generated by the execution module. Interaction trajectory data that meets the requirements for incremental training is selected, and incremental training samples are constructed based on this selected data. The incremental training system can choose to fine-tune or retrain the machine learning model 150 to perform incremental training, achieving a continuous optimization closed loop of data acquisition-model training-inference-incremental training, ensuring that the machine learning model 150 can adapt to constantly changing application scenarios.

[0069] In some examples, the embodied engine 230 can store data such as model parameters and training logs of the trained or incrementally trained machine learning model 150 in the storage engine 210, and can provide the trained or incrementally trained machine learning model 150 to the execution module, which can then schedule relevant resources to perform inference tasks.

[0070] In some examples, such as Figure 2 As shown, the embodied engine 230 can be deployed on the model layer 280. The model layer 280 can support parallel input and processing of multimodal data. It can provide optimized training strategies to improve the training efficiency of the machine learning model 150, enabling it to efficiently learn the relationships between vision, language, and action, thus building a machine learning model 150 with general capabilities. The model layer can also support various reinforcement learning strategies to fine-tune the machine learning model 150 based on simulation results, improving its performance in specific tasks.

[0071] It should be noted that the training process of the embodied engine 230 and the machine learning model 150 described above is merely exemplary. In practical applications, the machine learning model 150 can also employ a single training process. For example, the pre-training process can be omitted. The embodied engine 230 can be post-trained based on a dedicated dataset. This can shorten the training cycle and reduce the computational cost of training.

[0072] like Figure 2As shown, the evaluation engine 240 can be configured to evaluate the machine learning model 150 based at least on simulation results to obtain a first evaluation result for the machine learning model 150. The first evaluation result can indicate the task performance of the machine learning model 150. The machine learning model 150 here can include a machine learning model 150 in an intermediate state (i.e., a machine learning model 150 that has not yet been fully trained), a pre-trained machine learning model 150, a post-trained machine learning model 150, or a machine learning model 150 that has been incrementally trained. In some examples, such as... Figure 2 As shown, the evaluation engine 240 can be deployed on the model layer 280.

[0073] In some examples, the evaluation engine 240 obtains data related to the training of the machine learning model (such as model parameters, training samples, training logs, etc.), inference results, simulation data, and other evaluation data from the storage engine 210. The evaluation engine 240 can evaluate the task performance of the machine learning model 150 from multiple dimensions such as success rate, stability, generalization ability, task completion, physical consistency, and transfer adaptability to obtain a first evaluation result. The evaluation engine 240 can feed back the first evaluation result to the storage engine 210 for storage. The evaluation engine 240 can also send the first evaluation result and optimization suggestions to, for example, the scheduling and runtime layer, where the scheduling and runtime layer schedules the relevant embodied engine 230 to perform iterative training on the machine learning model 150.

[0074] As an example, such as Figure 3 As shown, the evaluation engine 240 can determine the evaluation framework 322 (e.g., software framework or hardware framework, etc.) and evaluation protocol 336 based on the application scenario and task characteristics of the embodied intelligent agent (e.g., robot device 110). The evaluation protocol 336 can specify the evaluation metric system (e.g., task completion, physical consistency, migration adaptability, execution efficiency, energy consumption, etc.), evaluation data collection methods, evaluation process, and evaluation result judgment criteria. Based on the evaluation protocol, the evaluation engine 240 can obtain the evaluation dataset 334 from the storage engine 210. The evaluation dataset 334 may include, for example, data related to the training of the machine learning model 150, inference results, simulation data, and other related data. The evaluation engine 240 can perform multi-dimensional automated evaluation of the performance of the machine learning model 150 based on the evaluation protocol 336 and the evaluation dataset 334. The evaluation process requires no manual intervention and can quickly generate detailed evaluation results, clearly identifying the model's strengths and weaknesses.

[0075] In some cases, the evaluation engine 240 can provide initial evaluation results to the embodied engine 230. Based on the initial evaluation results, the embodied engine 230 can determine the training parameters and training strategy for the machine learning model 150 and perform incremental training of the machine learning model 150. Through this closed-loop feedback mechanism, the machine learning model 150 can be continuously optimized, thereby continuously improving the overall performance of the system.

[0076] In some examples, the evaluation engine 240 is also configured to acquire simulation data, which indicates at least one of the simulation environment or virtual objects. The evaluation engine 240 can then evaluate at least one of the simulation environment or virtual objects based on the simulation data to obtain a second evaluation result. In some examples, the evaluation engine 240 can feed back the second evaluation result to the simulation engine 220. The simulation engine 220 can update at least one of the simulation environment or virtual objects based on the second evaluation result to obtain updated simulation data. In this way, optimization and adjustment of the simulation scene can be achieved.

[0077] In summary, the data processing system of this disclosure includes a storage engine 210, a simulation engine 220, an embodied engine 230, and an evaluation engine 240. It supports operations such as data acquisition, simulation generation, model training, inference execution, evaluation feedback, and deployment of the robot device 110, and can meet the task requirements throughout the entire lifecycle of embodied intelligent agent development. Therefore, it is beneficial to improve the system's scheduling capabilities to achieve efficient task collaboration, to increase the coupling between various stages, to ensure smooth data flow between stages, and to improve the utilization rate of computing resources.

[0078] The embodiments disclosed herein also enable collaboration among multiple physics engines, breaking down barriers between different engines and improving system compatibility and scalability. Based on heterogeneous computing power from cloud devices, efficient scheduling and elastic scaling of computing resources can be achieved, which is beneficial for improving the utilization efficiency of computing resources. Furthermore, this data processing system provides continuous learning capabilities for the machine learning model 150, ensuring that the machine learning model 150 can be continuously optimized and upgraded based on feedback from actual application scenarios.

[0079] Figure 4 A flowchart of a data processing procedure 400 according to some embodiments of the present disclosure is shown. Procedure 400 can be implemented in a data processing system as described above.

[0080] In box 410, the data processing system can acquire data related to the embodied intelligent agent, including at least multimodal data, which includes visual information, language information, and action information.

[0081] In box 420, the data processing system trains a machine learning model for an embodied agent based on multimodal data to obtain a trained machine learning model.

[0082] In box 420, the data processing system provides a simulation environment and virtual objects located within the simulation environment that correspond to the embodied intelligent agent.

[0083] In box 420, the data processing system generates simulation results based on the interaction between the machine learning model and the virtual object.

[0084] In box 420, the data processing system evaluates the machine learning model based at least on the simulation results to obtain a first evaluation result for the machine learning model.

[0085] In some examples, training a machine learning model includes: training a machine learning model to be trained based on pre-training samples to obtain a pre-trained machine learning model, wherein the pre-training samples are constructed based at least on multimodal data; and training the pre-trained machine learning model based on post-training samples to obtain a trained machine learning model, wherein the post-training samples are constructed based at least on simulation results.

[0086] In some examples, the action information indicates a reference action performed by the source object, and the method further includes: performing action redirection based on the action information to obtain a truth action instruction, which is used to drive the embodied agent to perform an action that approximates the reference action.

[0087] In some examples, training a machine learning model includes: constructing training samples based on visual information, linguistic information, and ground truth action instructions; training sample model inputs and ground truth action instructions; the sample model inputs including at least sample visual inputs and sample linguistic inputs; providing the sample inputs to the machine learning model to be trained to obtain predicted action instructions; and updating the model parameters of the machine learning model based on the difference between the predicted action instructions and the ground truth action instructions to obtain a trained machine learning model.

[0088] In some examples, process 400 further includes: receiving a task request for an embodied agent, the task request including at least visual input and natural language-based language input, the visual input describing the environment in relation to the embodied agent; and using a trained machine learning model, based at least on the visual input and language input, to determine action instructions to drive the embodied agent to perform a task corresponding to the task request.

[0089] In some examples, providing a simulation environment and a virtual object includes: receiving a simulation request, the simulation request including configuration parameters for at least one of the simulation environment or the virtual object; and performing a rendering operation based on the configuration parameters to generate at least one of the simulation environment or the virtual object.

[0090] In some examples, the simulation engine includes multiple simulation nodes, providing a simulation environment and virtual objects including: in response to a simulation request for the simulation environment, determining multiple simulation tasks corresponding to the simulation environment; assigning the multiple simulation tasks to the multiple simulation nodes; and using the multiple simulation nodes to execute the multiple simulation tasks in parallel to generate the simulation environment.

[0091] In some examples, process 400 further includes: acquiring simulation data, which indicates at least one of a simulation environment or a virtual object; and evaluating at least one of the simulation environment or virtual object based on the simulation data to obtain a second evaluation result.

[0092] In some examples, process 400 further includes updating at least one of the simulation environment or virtual objects based on the second evaluation result to obtain updated simulation data.

[0093] In some examples, acquiring data related to an embodied agent includes: acquiring at least one of a first sense data of a physical environment or a second sense data of a simulated environment; and obtaining multimodal data based on at least one of the first sense data or the second sense data.

[0094] In some examples, the data also includes at least one of the following: model parameters of a machine learning model, simulation results, evaluation results, or simulation data indicating at least one of the simulation environment or virtual objects.

[0095] Figure 5 A block diagram of an electronic device 500 in which one or more embodiments of the present disclosure may be implemented is shown. It should be understood that... Figure 5 The electronic device 500 shown is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein.

[0096] like Figure 5 As shown, electronic device 500 is in the form of a general-purpose electronic device. Components of electronic device 500 may include, but are not limited to, one or more processors or processing units 510, memory 520, storage device 530, one or more communication units 540, one or more input devices 550, and one or more output devices 560. Processing unit 510 may be a physical or virtual processor and is capable of performing various processes according to programs stored in memory 520. In a multiprocessor system, multiple processing units execute computer-executable instructions in parallel to improve the parallel processing capability of electronic device 500.

[0097] Electronic device 500 typically includes multiple computer storage media. Such media can be any available media accessible to electronic device 500, including but not limited to volatile and non-volatile media, removable and non-removable media. Memory 520 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. Storage device 530 can be removable or non-removable media and can include machine-readable media, such as flash drives, disks, or any other media capable of storing information and / or data and accessible within electronic device 500.

[0098] Electronic device 500 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not explicitly stated... Figure 5 As shown, disk drives for reading from or writing to removable, non-volatile disks (e.g., "floppy disks") and optical disk drives for reading from or writing to removable, non-volatile optical disks can be provided. In these cases, each drive can be connected to a bus (not shown) via one or more data media interfaces. Memory 520 may include computer program product 525 having one or more program modules configured to perform various methods or actions of various embodiments of this disclosure.

[0099] Communication unit 540 enables communication with other electronic devices via a communication medium. Additionally, the functionality of components of electronic device 500 can be implemented using a single computing cluster or multiple computing machines capable of communicating via communication connections. Therefore, electronic device 500 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

[0100] Input device 550 can be one or more input devices, such as a mouse, keyboard, trackball, etc. Output device 560 can be one or more output devices, such as a monitor, speaker, printer, etc. Electronic device 500 can also communicate with one or more external devices (not shown) via communication unit 540 as needed. These external devices include storage devices, display devices, etc., and can communicate with one or more devices that enable user interaction with electronic device 500, or with any device that enables electronic device 500 to communicate with one or more other electronic devices (e.g., network card, modem, etc.). Such communication can be performed via input / output (I / O) interface (not shown).

[0101] According to an exemplary implementation of this disclosure, a computer-readable storage medium is provided that stores one or more computer instructions, wherein the one or more computer instructions are executed by a processor to implement the methods described above. According to an exemplary implementation of this disclosure, a computer program product is also provided, which is tangibly stored on a non-transient computer-readable medium and includes computer-executable instructions that are executed by a processor to implement the methods described above.

[0102] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products implemented according to this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0103] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0104] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions that execute on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0105] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0106] Various implementations of this disclosure have been described above. The foregoing description is exemplary and not exhaustive, nor is it limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to technology in the market, or to enable others skilled in the art to understand the implementations disclosed herein.

Claims

1. A data processing system for embodied intelligent agents, comprising: A storage engine is configured to store data related to the embodied intelligent agent, the data including at least multimodal data, the multimodal data including visual information, language information and action information; An embodied engine is configured to train a machine learning model for the embodied agent based on the multimodal data to obtain a trained machine learning model; The simulation engine is configured to provide a simulation environment and virtual objects located in the simulation environment corresponding to the embodied agent, and to generate simulation results based on the interaction between the machine learning model and the virtual objects; as well as An evaluation engine is configured to evaluate the machine learning model based at least on the simulation results to obtain a first evaluation result for the machine learning model.

2. The system of claim 1, wherein the embodied engine comprises: A pre-training system is configured to construct pre-training samples based on the multimodal data, and use the pre-training samples to train a machine learning model to be trained, so as to obtain a pre-trained machine learning model. as well as The post-training system is configured to construct post-training samples based on the simulation results, and use the post-training samples to train the pre-trained machine learning model to obtain the trained machine learning model.

3. The system of claim 1, wherein the action information indicates a reference action performed by the source object, and wherein the embodiment engine comprises: The action redirection module is configured to perform action redirection based on the action information to obtain a truth action instruction, which is used to drive the embodied agent to perform an action that approximates the reference action.

4. The system of claim 3, wherein the embodiment engine further comprises a training system configured to: Based on the visual information, the language information, and the truth action instructions, training samples are constructed. The training samples include sample model inputs and the truth action instructions. The sample model inputs include at least sample visual inputs and sample language inputs. The sample input is provided to the machine learning model to be trained in order to obtain predicted action instructions; as well as Based on the difference between the predicted action instruction and the true action instruction, the model parameters of the machine learning model are updated to obtain the trained machine learning model.

5. The system of claim 1, wherein the embodiment engine includes an execution module configured to: Receive a task request for the embodied agent, the task request including at least visual input and natural language-based language input, the visual input being used to describe the environment associated with the embodied agent; and Using the trained machine learning model, at least based on the visual input and the language input, action instructions are determined, which drive the embodied agent to perform a task corresponding to the task request.

6. The system of claim 1, wherein the simulation engine is further configured to: Receive a simulation request, the simulation request including configuration parameters for at least one of the simulation environment or the virtual object; and Rendering operations are performed based on the configuration parameters to generate at least one of the simulation environment or the virtual object.

7. The system according to claim 1, wherein the simulation engine comprises a plurality of simulation nodes, and the simulation engine is configured to: In response to a simulation request for the simulation environment, a plurality of simulation tasks corresponding to the simulation environment are determined; Assign the plurality of simulation tasks to the plurality of simulation nodes; as well as The simulation environment is generated by executing the multiple simulation tasks in parallel using the multiple simulation nodes.

8. The system of claim 1, wherein the evaluation engine is further configured to: Acquire simulation data, wherein the simulation data indicates at least one of the simulation environment or the virtual object; and The simulation environment or the virtual object is evaluated based on the simulation data to obtain a second evaluation result.

9. The system according to claim 8, wherein the simulation engine is further configured to: Based on the second evaluation result, at least one of the simulation environment or the virtual object is updated to obtain updated simulation data.

10. The system of claim 1, wherein the storage engine is further configured to: Acquire at least one of the following: first sensing data of the physical environment or second sensing data of the simulated environment; and The multimodal data is obtained based on at least one of the first sensing data or the second sensing data.

11. The system of claim 1, wherein the data further comprises at least one of the following: The model parameters of the machine learning model, The simulation results, The evaluation results, or Simulation data, wherein the simulation data indicates at least one of the simulation environment or the virtual object.

12. The system of claim 1, wherein at least one of the storage engine, the embodied engine, the simulation engine, and the evaluation engine is deployed on a cloud device.

13. A data processing method, comprising: Acquire data related to the embodied intelligent agent, the data including at least multimodal data, the multimodal data including visual information, language information and action information; A machine learning model for the embodied agent is trained based on the multimodal data to obtain a trained machine learning model; Provide a simulation environment and a virtual object located in the simulation environment corresponding to the embodied intelligent agent; Simulation results are generated based on the interaction between the machine learning model and the virtual object; as well as The machine learning model is evaluated at least based on the simulation results to obtain a first evaluation result for the machine learning model.

14. The method of claim 13, wherein training the machine learning model comprises: The machine learning model to be trained is trained based on the pre-training samples to obtain the pre-trained machine learning model, wherein the pre-training samples are constructed based at least on the multimodal data; as well as The pre-trained machine learning model is trained based on the post-training samples to obtain the trained machine learning model, wherein the post-training samples are constructed based at least on the simulation results.

15. The method of claim 13, wherein the action information indicates a reference action performed by the source object, and wherein the method further comprises: Based on the action information, action redirection is performed to obtain a truth action instruction, which is used to drive the embodied agent to perform an action that approximates the reference action.

16. The method of claim 15, wherein training the machine learning model comprises: Based on the visual information, the language information, and the truth action instructions, training samples are constructed. The training samples include sample model inputs and the truth action instructions. The sample model inputs include at least sample visual inputs and sample language inputs. The sample input is provided to the machine learning model to be trained in order to obtain predicted action instructions; as well as Based on the difference between the predicted action instruction and the true action instruction, the model parameters of the machine learning model are updated to obtain the trained machine learning model.

17. The method of claim 13, further comprising: Receive a task request for the embodied intelligent agent, the task request including at least visual input and natural language-based language input, the visual input being used to describe the environment associated with the embodied intelligent agent; as well as Using the trained machine learning model, at least based on the visual input and the language input, action instructions are determined, which drive the embodied agent to perform a task corresponding to the task request.

18. An electronic device comprising: At least one processing unit; as well as At least one memory is coupled to at least one processing unit and stores instructions for execution by the at least one processing unit, which, when executed by the at least one processing unit, cause the electronic device to perform the method according to any one of claims 13-17.

19. A computer-readable storage medium having a computer program stored thereon, the computer program being executable by a processor to implement the method according to any one of claims 13-17.

20. A computer program product tangibly stored in a computer storage medium and comprising computer-executable instructions that, when executed by a device, cause the device to perform the method according to any one of claims 13-17.

Images