Large-model-based robot brain body-equipped intelligent system and task execution method thereof

Abstract

The invention provides a robot brain body-equipped intelligent system based on a large model and a task execution method of the robot brain body-equipped intelligent system. According to the system, a modular distributed architecture is realized, the system is endowed with high expandability and flexibility, and multi-platform deployment is supported. The task planning based on the large model significantly improves the ability of the system to understand and execute complex instructions. Unified sensor data management realizes integration and efficient processing of multi-source data, and enhances the environment perception capability of the system. Meanwhile, a friendly real-time monitoring management interface simplifies monitoring and configuration management of the system. In the aspect of performance, completeness and consistency of task execution are ensured through close cooperation among the modules; the optimized algorithm and model improve the efficiency of task planning and action execution and reduce the system response delay.

Description

Technical Field

[0001] The present invention relates to the fields of artificial intelligence and robotics, and particularly to an embodied intelligent system of "machine brain" based on large models and its task execution method. Background Art

[0002] Existing robot systems mainly rely on preset programs and limited interaction capabilities, and it is difficult to adapt to changing environments and complex task requirements. When dealing with unstructured environments and complex tasks, traditional robot systems usually require a large amount of manual intervention and programming work, lacking the ability to understand natural language instructions and autonomous planning. In addition, existing systems have problems of low efficiency and insufficient flexibility in sensor data processing, task planning, and execution, and it is difficult to meet the requirements of modern industry and service for intelligence and flexibility.

[0003] With the development of artificial intelligence technology, especially the rise of large models and deep learning technology, new possibilities have been provided for the realization of robot embodied intelligence. However, applying large models to robot systems requires solving a series of technical challenges such as the adaptation of the model to physical hardware, real-time requirements, efficient data processing and communication. Summary of the Invention

[0004] The purpose of the present invention is to solve the problems in the prior art, and an embodied intelligent system of "machine brain" based on large models and its task execution method are proposed. This system can understand and execute complex human instructions, and is applied to fields such as automated production, intelligent manufacturing, and service robots, realizing autonomous decision-making and task execution of robots in complex environments.

[0005] The present invention is realized through the following technical solutions. The present invention proposes an embodied intelligent system of "machine brain" based on large models, and the embodied intelligent system includes an embodied intelligent execution system, a sensor information integration system, and a monitoring and management interface;

[0006] The embodied intelligent execution system, as the core part of the embodied intelligent system, integrates a voice interaction module, a task planning module, and an action generation module, realizing accurate recognition of human instructions, task decomposition, and generation and control of corresponding actions; the sensor information integration system is responsible for integrating and managing information from multiple sensors, providing comprehensive environmental perception support for task planning and execution; the monitoring and management interface provides an interaction interface for users, supporting real-time monitoring and configuration management of the state of the embodied intelligent system.

[0007] Furthermore, in the design and implementation of the voice interaction module, the voice interaction module can detect and activate the voice interaction function, convert the user's voice commands into commands that the embodied intelligent system can understand, and generate corresponding voice feedback to improve the naturalness and fluency of the interaction; in the design and implementation of the task planning module, the embodied intelligent system realizes a multi-level and deep-semantic instruction parsing and task decomposition mechanism through deep learning and large language models, achieving accurate understanding and execution of complex instructions; the task planning module parses multi-level task instructions into specific execution steps and formulates the optimal task execution plan according to the current environment and resource status; in the design and implementation of the motion generation module, the embodied intelligent system realizes the accurate execution of robot motions through an overall motion planning and control mechanism; according to the task planning results, the motion generation module generates executable motion instructions, plans the motion trajectory, and controls the robot's posture to ensure the smoothness and safety of the motion.

[0008] Furthermore, in the design and implementation of the sensor information integration system, the embodied intelligent system realizes the efficient integration and processing of multi-source sensor data through the establishment of a unified data interface layer, flexible data management strategies, and modular sensor management; among them, in order to effectively manage the information of various sensors, a unified data interface layer is designed to ensure that sensor data collected by different physical devices can be managed and accessed in a unified format; the data interface layer abstracts different underlying data sources, provides a consistent programming interface, and defines a standardized data structure for easy data parsing and use.

[0009] Furthermore, according to the characteristics and application requirements of the sensors, flexible data management strategies are adopted, including continuous publishing mechanisms, request-response mechanisms, and event-driven mechanism data processing mechanisms; for sensors that require high frequency and real-time performance, the embodied intelligent system supports the management method of continuous publishing mechanisms to meet the requirements of real-time monitoring and quick response; for data with low acquisition frequency or only required under specific conditions, the embodied intelligent system adopts request-response mechanisms and event-driven mechanisms to effectively save system resources and reduce system load.

[0010] Furthermore, the embodied intelligent system realizes a flexible data request layer on the server side; when the large model needs sensor data, it interacts with the sensor information integration system through a standardized communication interface to obtain the required data; the data request layer is responsible for processing these requests, receiving and standardizing the sensor data, enabling the large model to directly use this data for reasoning and planning.

[0011] Furthermore, the embodied intelligent system adopts a modular design concept and designs independent data processing modules for each sensor type, following a unified interface specification.

[0012] Furthermore, in the design and implementation of the monitoring and management interface, the embodied intelligent system provides two major functions: real-time monitoring and system management. The real-time monitoring function enables the interface to dynamically display the operating status, task progress, and key environmental data of each module. The system management function allows users to adjust system configuration parameters online and provides control options for task execution.

[0013] The present invention also proposes a task execution method for an embodied intelligent system with a "machine brain" based on a large model. The method is specifically as follows: The user issues an instruction through a natural interaction method, and the voice interaction module is responsible for the recognition and parsing of the instruction. The task planning module generates a task execution plan based on the parsing result. The action generation module controls the robot to execute specific tasks according to the planning plan. At the same time, the sensor information integration system provides perception data to support the dynamic adjustment and optimization of the task. The operating status and execution result of the embodied intelligent system are real-time fed back to the user through the monitoring and management interface.

[0014] The present invention also proposes an electronic device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the task execution method for an embodied intelligent system with a "machine brain" based on a large model.

[0015] The present invention also proposes a computer-readable storage medium for storing computer instructions. When the computer instructions are executed by a processor, they implement the steps of the task execution method for an embodied intelligent system with a "machine brain" based on a large model.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0017] The present invention provides an embodied intelligent system with a "machine brain" based on a large model and its task execution method. The system realizes a modular distributed architecture, endows the system with high scalability and flexibility, and supports multi-platform deployment. The task planning based on a large model significantly improves the system's ability to understand and execute complex instructions. The unified sensor data management realizes the integration and efficient processing of multi-source data, enhancing the system's environmental perception ability. At the same time, the friendly real-time monitoring and management interface simplifies the system's monitoring and configuration management. In terms of performance, through the close cooperation between modules, the integrity and consistency of task execution are ensured; the optimized algorithms and models improve the efficiency of task planning and action execution, reducing the system response delay. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on the provided drawings.

[0019] Figure 1 This is the workflow diagram for each module of the embodied intelligent system described in the present invention to cooperate to complete tasks. Detailed implementation manners

[0020] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0021] Combined with Figure 1 , the present invention proposes an embodied intelligent system of "machine brain" based on large models. The system improves the robot's ability to understand and autonomously execute complex instructions, and realizes efficient task planning and execution. The embodied intelligent system includes an embodied intelligent execution system, a sensor information integration system, and a monitoring and management interface;

[0022] As the core part of the embodied intelligent system, the embodied intelligent execution system integrates a voice interaction module, a task planning module, and an action generation module to achieve accurate recognition of human instructions, task decomposition, and generation and control of corresponding actions; the sensor information integration system is responsible for integrating and managing information from multiple sensors, providing comprehensive environmental perception support for task planning and execution; the monitoring and management interface provides an interaction interface for users, supporting real-time monitoring and configuration management of the state of the embodied intelligent system.

[0023] In terms of the design and implementation of the embodied intelligent execution system, the system adopts a modular architecture. Each functional module can be distributed on different hardware platforms and cooperate through a unified communication mechanism. This modular design improves the scalability and fault tolerance of the system, facilitating dynamic scheduling and management of resources. Each module exchanges data through standardized interfaces to ensure the independence and decoupling of modules. The system supports cross-platform and cross-network module cooperation, ensuring the security and reliability of data transmission, while improving the maintainability and flexibility of the system.

[0024] In the design and implementation of the voice interaction module, the system adopts a multi-level voice processing flow to achieve human-computer communication. The "multi-level" here means that this process not only covers the basic voice signal capture and preprocessing, but also includes multiple processing stages such as speech recognition, intention recognition, and natural language understanding. This voice interaction module can detect and activate the voice interaction function, convert the user's voice commands into commands that the embodied intelligent system can understand, and generate corresponding voice feedback to improve the naturalness and fluency of the interaction; to optimize performance, the module adopts advanced voice processing technologies to improve the recognition accuracy and environmental adaptability, and at the same time reduces the response delay through intelligent optimization strategies to enhance the user experience.

[0025] In the design and implementation of the task planning module, the embodied intelligent system realizes a multi-level and deep semantic instruction parsing and task decomposition mechanism through deep learning and large language models, achieving precise understanding and execution of complex instructions; this task planning module parses the multi-level task instructions into specific execution steps and formulates the optimal task execution plan according to the current environment and resource status; specifically, the large language model has mastered a vast amount of language expression and task execution data through pre-training and fine-tuning, and has powerful semantic understanding and intention recognition capabilities. After receiving the user's high-level task instructions, the model can automatically extract the key elements and implicit intentions in the instructions, parse the multi-dimensional semantic information of the instructions, and combine domain knowledge with the current environment and resource status to decompose the abstract task into specific executable steps, thus formulating the optimal task execution plan.

[0026] In the design and implementation of the action generation module, the embodied intelligent system realizes the precise execution of robot actions through an overall action planning and control mechanism; according to the task planning results, the action generation module generates executable action instructions, plans the motion trajectory, and controls the robot's posture to ensure the smoothness and safety of the actions. The action planning and control mechanism includes the adaptation of the robot motion model and multi-platform support to ensure the consistency of action execution in different hardware environments, and at the same time adjusts the actions in real time through the feedback mechanism to ensure the accuracy and reliability of the execution.

[0027] In the design and implementation of the sensor information integration system, the embodied intelligent system realizes the efficient integration and processing of multi-source sensor data through the establishment of a unified data interface layer, flexible data management strategies, and modular sensor management. Among them, the robot system often requires a rich variety of sensors to provide sufficient information, such as visual sensors, lidar, temperature sensors, infrared sensors, etc. To effectively manage the information of various sensors, a unified data interface layer is designed to ensure that sensor data collected by different physical devices can be managed and accessed in a unified format. This data interface layer abstracts different underlying data sources, provides a consistent programming interface, and defines a standardized data structure to facilitate data parsing and use.

[0028] According to the characteristics and application requirements of the sensors, flexible data management strategies are adopted, including continuous publishing mechanisms, request-response mechanisms, and event-driven mechanism data processing mechanisms. For sensors that require high frequency and real-time performance, the embodied intelligent system supports the management method of the continuous publishing mechanism to meet the requirements of real-time monitoring and rapid response. For data with low acquisition frequency or only required under specific conditions, the embodied intelligent system adopts the request-response mechanism and event-driven mechanism to effectively save system resources and reduce system load.

[0029] To ensure that the large model on the server can efficiently obtain sensor information, the embodied intelligent system implements a flexible data request layer on the server side. When the large model needs sensor data, it interacts with the sensor information integration system through a standardized communication interface to obtain the required data. The data request layer is responsible for processing these requests, receiving and standardizing the sensor data, enabling the large model to directly use this data for reasoning and planning. To simplify the development process of the large model, the system provides predefined data request interfaces to facilitate the large model to quickly integrate and call the required sensor data, improving the overall efficiency of the system.

[0030] In addition, to enhance the scalability of the sensor system, the embodied intelligent system adopts a modular design concept, designing independent data processing modules for each sensor type, following a unified interface specification. Through this modular design, the system can quickly integrate new sensor types, reduce the complexity of system expansion, and ensure that newly added sensors can cooperate seamlessly with the existing system.

[0031] Overall, this sensor information integration system realizes the efficient integration and sharing of various sensor data through a unified data interface layer, flexible data management strategies, and modular design, meets the needs of the large model for multi-source data, and supports the smooth execution of complex tasks.

[0032] In the design and implementation of the monitoring and management interface, the embodied intelligent system provides two major functions: real-time monitoring and system management. The real-time monitoring function enables the interface to dynamically display the operating status of each module, task progress, and key environmental data information. The system management function allows users to adjust system configuration parameters online and provides control options for task execution, such as start, pause, and stop operations.

[0033] The present invention also proposes a task execution method for an embodied intelligent system with a "machine brain" based on a large model. The method is specifically as follows: The user issues an instruction through a natural interaction method, and the voice interaction module is responsible for the recognition and parsing of the instruction. The task planning module generates a task execution plan based on the parsing result. The motion generation module then controls the robot to execute specific tasks according to the planning plan. At the same time, the sensor information integration system provides perception data to support the dynamic adjustment and optimization of the task. The operating status and execution results of the embodied intelligent system are fed back to the user in real time through the monitoring and management interface. Data exchange between each module is carried out through a standardized communication interface to ensure the timely transmission of information, and the efficient operation of the system is ensured through a collaborative working mechanism.

[0034] The generalization ability of an embodied intelligent system with a "machine brain" based on a large model proposed by the present invention has been improved, enabling it to adapt to diverse tasks and environments, demonstrating strong versatility. It reduces the dependence on manual programming and intervention and enhances the autonomy of the system. The modular design makes it easier to migrate the system to different hardware platforms, significantly reducing the cost of system expansion and upgrade.

[0035] Embodiment 1: Application in an automated production line

[0036] In an automated production line, the robot needs to complete complex assembly and handling tasks according to voice instructions. The operator issues an instruction through voice, such as "Please install part A at position B", and the voice interaction module recognizes and parses the instruction. The task planning module formulates an assembly process based on the parsing result, considering the status and resources of the production line. Subsequently, the motion generation module generates specific assembly motion instructions, and the robot starts to execute the assembly task. Throughout the process, the monitoring and management interface displays the progress of the task in real time, and the operator can view and adjust it at any time. In this way, the automation level of the production line is improved, manual operations are reduced, and production efficiency is increased.

[0037] Embodiment 2: Application in a service robot

[0038] In a hotel environment, the service robot provides services such as food delivery and guided tours according to the guests' instructions. When a guest says to the robot, "Please send a cup of coffee to the room," the voice interaction module recognizes and parses the instruction. The task planning module parses out the food delivery task, plans the optimal route, and avoids obstacles and crowds. The motion generation module controls the robot to move forward along the planned route and adjusts the motion in real time to adapt to environmental changes. The sensor information integration system uses sensors such as cameras and lidar to sense the environment and provides necessary data support. When the robot reaches the destination, it notifies the guest that the task is completed and then returns to the initial position. This application improves the service quality, enhances the user experience, and reduces the labor cost.

[0039] The present invention also provides an electronic device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the steps of the task execution method of the "machine brain" embodied intelligence system based on the large model are implemented.

[0040] The present invention also provides a computer-readable storage medium for storing computer instructions, and when the computer instructions are executed by a processor, the steps of the task execution method of the "machine brain" embodied intelligence system based on the large model are implemented.

[0041] The memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM), and direct rambus RAM (DRRAM). It should be noted that the memory of the method described in the present invention is intended to include but not limited to these and any other suitable types of memories.

[0042] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that integrates one or more available media. The available media can be magnetic media (such as floppy disks, hard disks, magnetic tapes), optical media (such as high-density digital video discs (DVDs)), or semiconductor media (such as solid state discs (SSDs)), etc.

[0043] In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor or the instructions in the form of software. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed by the hardware processor, or executed by a combination of the hardware and software modules in the processor. The software module can be located in a mature storage medium in the art such as random access memory, flash memory, read-only memory, programmable read-only memory, or electrically erasable programmable memory, registers, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0044] It should be noted that the processor in the embodiments of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, the steps of the above method embodiments can be completed by the integrated logic circuit in the hardware of the processor or instructions in the form of software. The above-mentioned processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the art such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, registers, etc. This storage medium is located in the memory, and the processor reads the information in the memory and combines its hardware to complete the steps of the above method.

[0045] The above has introduced in detail the "machine brain" embodied intelligent system based on the large model and its task execution method proposed by the present invention. Specific examples are used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. An embodied intelligent system of "machine brain" based on large models, characterized in that The embodied intelligence system includes an embodied intelligence execution system, a sensor information integration system, and a monitoring and management interface; As the core part of the embodied intelligence system, the embodied intelligence execution system integrates a voice interaction module, a task planning module, and an action generation module to achieve accurate recognition of human instructions, task decomposition, and generation and control of corresponding actions; the sensor information integration system is responsible for integrating and managing information from multiple sensors to provide comprehensive environmental perception support for task planning and execution; the monitoring and management interface provides an interaction interface for users to support real-time monitoring and configuration management of the state of the embodied intelligence system.

2. The embodied intelligent system according to claim 1, wherein In the design and implementation of the voice interaction module, the voice interaction module can detect and activate the voice interaction function, convert the user's voice instructions into commands that the embodied intelligence system can understand, and generate corresponding voice feedback to improve the naturalness and fluency of the interaction; in the design and implementation of the task planning module, the embodied intelligence system realizes a multi-level and deep-semantic instruction parsing and task decomposition mechanism through deep learning and large language models, achieving accurate understanding and execution of complex instructions; the task planning module parses multi-level task instructions into specific execution steps and formulates the optimal task execution plan according to the current environment and resource status; in the design and implementation of the action generation module, the embodied intelligence system realizes accurate execution of robot actions through an overall action planning and control mechanism; according to the task planning results, the action generation module generates executable action instructions, plans the motion trajectory, and controls the robot posture to ensure the smoothness and safety of the actions.

3. The embodied intelligent system according to claim 1, wherein In the design and implementation of the sensor information integration system, the embodied intelligence system realizes the efficient integration and processing of multi-source sensor data through the establishment of a unified data interface layer, a flexible data management strategy, and modular sensor management; among them, in order to effectively manage the information of various sensors, a unified data interface layer is designed to ensure that sensor data collected by different physical devices can be managed and accessed in a unified format; the data interface layer abstracts different underlying data sources, provides a consistent programming interface, and defines a standardized data structure for easy data parsing and use.

4. The embodied intelligent system according to claim 3, wherein According to the characteristics and application requirements of the sensors, a flexible data management strategy is adopted, including continuous publishing mechanism, request-response mechanism, and event-driven mechanism data processing mechanisms; for sensors that require high frequency and real-time performance, the embodied intelligence system supports the management method of the continuous publishing mechanism to meet the requirements of real-time monitoring and quick response; for data with low acquisition frequency or only required under specific conditions, the embodied intelligence system adopts the request-response mechanism and the event-driven mechanism to effectively save system resources and reduce system load.

5. The embodied intelligent system according to claim 4, wherein The embodied intelligence system realizes a flexible data request layer on the server side; when the large model needs sensor data, it interacts with the sensor information integration system through a standardized communication interface to obtain the required data; The data request layer is responsible for processing these requests, receiving and standardizing the sensor data, enabling the large model to directly use this data for reasoning and planning.

6. The embodied intelligent system according to claim 3, wherein The embodied intelligent system adopts a modular design concept, designs independent data processing modules for each type of sensor, and follows a unified interface specification.

7. The embodied intelligent system according to claim 1, wherein In the design and implementation of the monitoring and management interface, the embodied intelligent system provides two major functions: real-time monitoring and system management; the real-time monitoring function enables the interface to dynamically display the operating status, task progress, and key information of environmental data of each module; the system management function allows users to adjust system configuration parameters online and provides control options for task execution.

8. A task execution method for an embodied intelligent system of "machine brain" based on the large model described in claim 1, characterized in that, The specific method is as follows: The user issues an instruction through a natural interaction method, and the voice interaction module is responsible for the recognition and parsing of the instruction; the task planning module generates a task execution plan according to the parsing result; the action generation module controls the robot to execute specific tasks according to the planning plan; at the same time, the sensor information integration system provides perception data to support the dynamic adjustment and optimization of tasks; the operating status and execution results of the embodied intelligent system are fed back to the user in real time through the monitoring and management interface.

9. An electronic device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method described in claim 8.

10. A computer-readable storage medium for storing computer instructions, characterized in that, When the computer instruction is executed by the processor, it implements the steps of the method described in claim 8.

Images