End-side cloud collaborative management system and method for brain-equipped robot

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a brain-equipped robot end-side cloud collaborative management system, which comprises a brain-equipped robot unit, a man-machine interaction client, an APP / applet, a resource management platform and an application market platform, the brain-equipped robot unit comprises a multi-mode sensing module which is used for understanding natural language instructions, screen contents and voice interaction; a large model decision module; the execution control module is used for simulating a keyboard and a mouse to operate computer software; the man-machine interaction client and the APP / applet serve as an interface for interaction; the resource management platform comprises equipment and a user management module; the resource matching algorithm module is used for evaluating and distributing optimal computing resources; the block chain registration module is used for recording ownership and use right information of the task; the sandbox execution environment module is used for creating a running environment; the application market platform is used for providing a developer tool chain and a revenue distribution mechanism. The application efficiency of the agent system can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a brain-powered robot-edge-cloud collaborative management system and method. Background Art

[0002] Current intelligent agents and their systems suffer from several specific technical flaws. First, existing automated tools lack the ability to understand and plan tasks autonomously, making them incapable of handling complex, unstructured tasks such as data analysis and chart generation. Second, locally deployed intelligent agent systems struggle to achieve cross-device collaboration and dynamic resource allocation, resulting in low efficiency.

[0003] Furthermore, the closed ecosystem of traditional intelligent agent applications lacks developer incentives and monetization channels, limiting functionality expansion. Existing systems also fail to implement the division of ownership and revenue distribution within human-machine collaborative workflows, hindering large-scale adoption. Furthermore, autonomous operation of intelligent agents deployed on ordinary computers presents privacy, security, and performance bottlenecks. Task issuance and result verification require manual intervention, increasing complexity and uncertainty. These limitations restrict the application scenarios of existing intelligent agent systems, leading to low efficiency and an inability to meet the high demands of modern enterprises and individuals for intelligent office and automation.

[0004] From the above, we can see that how to improve the application efficiency of intelligent agent systems still needs to be solved. Summary of the Invention

[0005] In order to improve the application efficiency of the intelligent system, the present application provides a brain-powered robot end-edge-cloud collaborative management system and method.

[0006] In the first aspect, the present application provides a brain-powered robot end-edge-cloud collaborative management system, which adopts the following technical solutions:

[0007] A brain-powered robot end-edge-cloud collaborative management system, including four parts: a brain-powered robot unit, a resource management platform, a human-computer interaction client, an APP / applet, and an application market platform;

[0008] The brain-powered robotic unit is used to autonomously operate a computer to complete tasks. It includes a multimodal perception module for understanding natural language instructions, screen content, and voice interaction; a large model decision module for analyzing task requirements, breaking down and planning tasks, and generating detailed execution plans; and an execution control module for simulating keyboard and mouse operations on computer software to complete tasks according to the execution plan.

[0009] Human-computer interaction client and APP / applet, which are used for interactive interfaces, including activation, management, control, status query, application download and update of the brain-equipped robot, as well as task issuance, task execution process intervention and task execution result confirmation;

[0010] The resource management platform is used to manage the brain-equipped robots through networking, assign permissions between the brain-equipped robots and their owners, and implement the division of labor and collaboration among multiple brain-equipped robots for dynamic scheduling of computing resources. Specifically, it includes: a device and user management module; a resource matching algorithm module for evaluating and allocating optimal computing resources based on task requirements; a blockchain registration module for recording task ownership and usage rights; and a sandbox execution environment module for creating an independent and secure operating environment for each task.

[0011] The application market platform is used to provide developer tool chains and revenue distribution mechanisms, and implement the revenue distribution mechanism when Brainbot users pay to download and subscribe to applications. Specifically, it includes: Developer SDK, which is used to provide APIs and tool chains, and is used to develop applications for tasks; Application review and release unit, which is used for automated testing and security verification; Revenue distribution smart contract module, which is used to support revenue models, including pay-per-use, subscription and profit-sharing.

[0012] Optionally, the large model decision module of the brain-equipped robot unit supports a hybrid deployment mode, including:

[0013] Based on the task requirements input by the user through the client or APP / Mini Program, obtain the corresponding task requirement data and determine the corresponding task type;

[0014] If the task type is determined to be a simple immediate task requirement, the local lightweight model is used to parse the task requirement data and generate a corresponding execution plan;

[0015] If the task type is determined to require high computing power, the task requirement data is sent to the edge server, and the medium-sized model on the edge server is used for collaborative reasoning to generate the corresponding execution plan;

[0016] If the task type is determined to be a complex task, the task requirement data will be transmitted to the cloud, and deep processing will be performed using a large cloud model to generate a detailed execution plan, which will then be returned to the brain-powered robot unit.

[0017] Optionally, the large model decision module of the brain-equipped robot unit supports a hybrid deployment mode, including:

[0018] Based on the task requirements input by the user through the client or APP / Mini Program, obtain the corresponding task requirement data and determine the corresponding task type;

[0019] If the task type is determined to be a simple immediate task requirement, the local lightweight model is used to parse the task requirement data and generate a corresponding execution plan;

[0020] If the task type is determined to require high computing power, the task requirement data is sent to the edge server, and the medium-sized model on the edge server is used for collaborative reasoning to generate the corresponding execution plan;

[0021] If the task type is determined to be a complex task, the task requirement data will be transmitted to the cloud, and deep processing will be performed using a large cloud model to generate a detailed execution plan, which will then be returned to the brain-powered robot unit.

[0022] Optionally, the specific steps of the application market platform include:

[0023] Register as a developer through the app market platform, establish a corresponding development environment, and use the developer SDK to develop applications for specific tasks;

[0024] Obtain the target application submitted by the developer to the Robot App Market, and control the application review and release module to perform automated testing and security verification on the submitted target application.

[0025] After review, the target application is put on the application market platform for users to download. Users purchase and download the target application through the client or APP / mini program, and use the brain-powered robot to complete the instructions.

[0026] Optionally, the execution control module steps of the brain-equipped robot unit include:

[0027] The execution control module receives the execution plan generated by the large model decision module.

[0028] The execution control module simulates keyboard and mouse operations based on the execution plan to automatically complete the corresponding computer software task;

[0029] The execution control module monitors the task execution process in real time and feeds back the execution results to the large model decision module. If an exception occurs during the task execution, the execution control module will make adjustments based on the exception feedback information and continue to execute the task.

[0030] Optionally, the developer SDK of the application market system includes the following specific steps:

[0031] Provides a brain-powered robot control API based on the developer SDK to allow developers to develop applications for specific tasks;

[0032] Call the large model fine-tuning toolchain in the developer SDK and use it to enable developers to perform customized training on large models.

[0033] After the developer develops the application using the developer SDK, the application developed by the developer is obtained and integrated into the brain-equipped robot unit.

[0034] In a second aspect, the present application provides a method for collaborative management of a brain-powered robot end-edge-cloud, which adopts the following technical solutions:

[0035] A method for collaborative management of a brain-powered robot, edge, and cloud, comprising:

[0036] Based on the task requirements input by the user through the client or APP / Mini Program, the multimodal perception module understands natural language instructions, screen content and voice interaction to obtain the corresponding task requirement data, parses the task requirement data and generates the corresponding structured task tree;

[0037] The large-scale decision-making module analyzes task requirements and generates a detailed execution plan. Based on a structured task tree, the main task corresponding to the task requirement data is decomposed into multiple subtasks. The resource matching algorithm module evaluates the computing resources required for each subtask based on the task requirement data and generates the corresponding optimal scheduling plan. The sandbox execution environment module creates an independent and secure operating environment for each subtask and assigns corresponding operating permissions. The resource management platform monitors the execution process of each subtask in real time and dynamically adjusts resources through the resource matching algorithm module.

[0038] Based on the execution plan, the execution control module simulates keyboard and mouse operations on the computer software to complete the corresponding task. After the task is completed, a corresponding execution report is generated. Among them, the blockchain registration module updates the ownership record and feeds back the results to the user through the developer SDK.

[0039] In a third aspect, the present application provides a brain-powered robot-edge-cloud collaborative management system, which adopts the following technical solutions:

[0040] A brain-enabled robot end-edge-cloud collaborative management system includes a processor running a program of any one of the above-mentioned brain-enabled robot end-edge-cloud collaborative management methods.

[0041] In a fourth aspect, the present application provides a storage medium, which adopts the following technical solution:

[0042] A storage medium storing a program of any one of the above-mentioned methods for collaborative management of a brain-powered robot end-edge-cloud.

[0043] In summary, this application includes at least one of the following beneficial technical effects:

[0044] This application realizes the automated processing of complex tasks through the collaborative work of the multimodal perception module, large model decision module and execution control module of the brain-based robot unit. This not only reduces the need for manual intervention, but also improves the efficiency and accuracy of task execution, and solves the complexity and uncertainty problems caused by the reliance on manual operation for task issuance and result confirmation in traditional intelligent system. By understanding natural language instructions, screen content and voice interaction, users can easily input task requirements, while the large model decision module is responsible for parsing and generating detailed execution plans to ensure that each subtask can be completed efficiently. The execution control module further simulates keyboard and mouse operations to automatically complete various computer software tasks, greatly improving the degree of automation of task execution, making the application scenarios of the intelligent system more extensive, and meeting the high demands of modern enterprises and individuals for intelligent office and automation.

[0045] The resource management platform effectively addresses the privacy, security, and performance bottlenecks faced by intelligent agents deployed on standard computers by dynamically scheduling computing resources, creating independent secure operating environments, and monitoring task execution in real time. The platform evaluates and allocates optimal computing resources based on task requirements, ensuring that tasks can be executed under optimal conditions. The sandbox execution environment module provides each task with an independent, secure operating environment to prevent data leakage and cross-interference. Furthermore, the blockchain registration module records task ownership and usage rights, enhancing system transparency and security. This design not only optimizes computing resource utilization and avoids performance bottlenecks, but also significantly improves system reliability and efficiency. It enables flexible configuration and cost optimization of intelligent agent computing resources across client, local, and cloud-based invocations, significantly enhancing the application efficiency of the intelligent agent system.

[0046] The application market platform promotes the development and promotion of innovative applications by providing a developer tool chain and a revenue distribution mechanism, building a complete closed loop and ecosystem among intelligent users, intelligent application developers, and intelligent end-edge cloud platforms. Developers can use the API interfaces provided by the SDK to develop applications for specific tasks and ensure the quality of applications through the application market's automated testing and security verification processes. The revenue distribution smart contract module supports multiple revenue models such as pay-per-use, subscription, and profit-sharing, incentivizing developers to participate in ecosystem construction and promoting the growth of the number of applications and the enrichment of functions. This complete ecosystem not only expands the functions and application scenarios of intelligent agents, but also helps users significantly reduce costs and increase efficiency, allowing platforms and application developers to maximize profits, thereby better serving the high demand of modern enterprises and individuals for intelligent office and automation. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a structural block diagram of a brain-based robot-end-edge-cloud collaborative management system according to an exemplary embodiment.

[0048] Figure 2 It is a structural block diagram of a brain-enabled robot unit in a brain-enabled robot end-edge-cloud collaborative management system according to an exemplary embodiment. Figure 3 It is a structural block diagram of a resource management platform in a brain-based robot-end-edge-cloud collaborative management system according to an exemplary embodiment. Figure 4 It is a structural block diagram of an application market platform in the collaborative management of end-edge-cloud with a brain robot according to an exemplary embodiment. Figure 5 It is a flowchart of a method for collaborative management of a brain-powered robot end-edge-cloud according to an exemplary embodiment. DETAILED DESCRIPTION

[0049] Embodiments of the present application are described in detail below, examples of which are illustrated in the accompanying drawings.

[0050] Throughout this specification, reference to the terms "certain embodiments," "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with the embodiment or example is included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0051] The present application embodiment discloses a brain-powered robot end-edge cloud collaborative management system, referring to Figure 1 , including a brain-powered robot unit 200, a human-computer interaction client and APP / applet 210, a resource management platform 220, and an application market platform 230;

[0052] Reference Figure 2It should be noted that the brain-equipped robot unit 200 is a type of intelligent agent used to autonomously operate a computer to complete tasks. The brain-equipped robot can be a standalone hardware unit with a main control, memory, storage, and high computing power (connected to the display output interface and USB interface of the host computer), or it can be a virtual software-equipped robot running on a PC (including a high-performance AIPC). In addition, the standalone hardware brain-equipped robot has one or more built-in hard drives, which can store downloaded brain-equipped robot applications, serve as temporary data storage for the running brain-equipped robot applications, save the process and results of task execution, and store the knowledge base of the brain-equipped robot and its owner. Part of the disk space can also be used as NAS storage for remote access by the brain-equipped robot owner. It includes a multimodal perception module: used to understand natural language instructions, screen content, and voice interaction; a large model decision module for parsing task requirements, decomposing and planning tasks, and generating detailed execution plans; and an execution control module for simulating keyboard and mouse operations on computer software according to the execution plan to complete the task.

[0053] For the multimodal perception module, the task requirements of understanding natural language instructions, screen content, voice interaction, and parsing user input are as follows: the user inputs the task requirements through the client or APP / mini program (for example, "make a quarterly financial report PPT"), and the system receives the user's natural language instructions, screen content or voice input; then uses natural language processing technology to parse the task requirements and extract key information. If the user inputs through voice, the system first converts the voice into text, and the system analyzes the content input by the user to identify the goals and specific requirements of the task.

[0054] For the large model decision module, it is used to parse task requirements and generate a detailed execution plan. The corresponding execution process includes: analyzing each subtask in the structured task tree, determining its specific requirements and priority, and selecting the appropriate model for processing based on the task type (simple immediate task, high computing power task, complex task).

[0055] That is to say, the large model decision module of the brain-equipped robot unit supports a hybrid deployment mode, which specifically includes the following steps: First, the user inputs the task requirements through the client or APP / mini program (for example, "make a quarterly financial report PPT"), the system receives the user's natural language instructions, screen content or voice input, and uses natural language processing technology to parse the task requirements. The system analyzes the content input by the user and identifies the goals and specific requirements of the task. According to the requirements and complexity of the task, the system classifies the task into simple instant tasks, high-computing power tasks or complex tasks. Based on the task requirements input by the user through the client or APP / mini program, the system obtains the corresponding task requirement data and determines the corresponding task type; among them, simple instant tasks: the task requirements are clear and the computing resource requirements are low, and can be completed quickly locally; high-computing power tasks: the tasks require higher computing resources, but do not require complex deep learning models, and are suitable for processing on edge servers; complex tasks: the tasks require a large amount of computing resources and complex deep learning models, and are suitable for processing in the cloud.

[0056] For simple, immediate tasks, the system selects an appropriate local lightweight model to handle the task. These models typically have low computational overhead and can quickly respond to user needs. The system then uses the selected local lightweight model to parse the task requirement data, extracting key information and operational steps. Based on the parsed results, it generates a detailed execution plan, including specific subtasks, operational steps, and dependencies. This plan is then passed to the execution control module, which then simulates keyboard and mouse operations on the computer software to complete the task.

[0057] For tasks requiring high computing power, the system selects an appropriate edge server based on the task requirements. Edge servers are typically located close to users, offering low latency and high computing power. Task request data is then sent to the selected edge server. Upon receiving the task request data, the edge server prepares for processing. The edge server performs collaborative reasoning on the task request data using medium-sized models, which have strong computing power and can handle more complex task requirements. Based on the reasoning results, the edge server generates a detailed execution plan and returns it to the brain-powered robot unit. This plan is then passed to the execution control module, which then simulates keyboard and mouse operations on the computer software to complete the task.

[0058] To handle complex tasks, the system selects an appropriate large-scale cloud-based model based on the task requirements. These models typically possess powerful computing capabilities and a wealth of training data, enabling them to handle complex tasks. The task requirement data is then transmitted to the cloud. Upon receiving the task requirement data, the cloud server prepares to process it. The cloud server uses large-scale models capable of handling complex tasks to perform in-depth processing on the task requirement data and generates a detailed execution plan. Based on the results of this in-depth processing, the cloud server generates a detailed execution plan containing multiple subtasks, operational steps, and dependencies to ensure efficient task completion. The cloud server returns the generated execution plan to the brain-powered robot unit, which then passes it to the execution control module to simulate keyboard and mouse operations on the computer software to complete the task.

[0059] Through the above detailed execution steps, the large model decision module of the brain-powered robot unit can flexibly select local, edge server or cloud processing according to different task requirements, thereby achieving efficient task execution. This hybrid deployment mode not only improves the efficiency and flexibility of task processing, but also ensures the reliability and security of the system, meeting the high demands of modern enterprises and individuals for intelligent office and automation.

[0060] Human-computer interaction client and APP / applet 210, which is used for interactive interface, including activation, management, control, status query, application download and update of the brain-equipped robot, as well as task issuance, task execution process intervention and task execution result confirmation;

[0061] The machine interaction client and APP / mini-program 200 (hereinafter referred to as the "client") serve as a bridge between the user and the brain-embedded robot system, providing a comprehensive interactive interface that supports the activation, management, control, status query, application download and update of the brain-embedded robot, as well as task issuance, intervention during execution, and result confirmation. The following is a detailed explanation of the execution steps of each functional module:

[0062] Activation: When a user first uses the client, they need to activate the service by registering an account and logging in. During the activation process, the client will guide the user through the necessary settings, such as binding an email address or mobile phone number and setting a password. Once activated, the user can begin using all available features. Management: Users can manage existing Brain Robots through the client. For example, users can view a list of all currently connected Brain Robots and perform basic configurations on them, such as setting working hours and adjusting priorities.

[0063] Control: The client allows users to directly control the operation of the Brain Robot. For example, users can start or pause a specific task, or manually trigger certain urgent tasks. In addition, users can adjust the behavior of the Brain Robot according to their needs, such as choosing a more efficient but resource-intensive mode, or an energy-saving but slightly less efficient mode. Status Query: To ensure the smooth progress of tasks, users can query the operating status of the Brain Robot at any time through the client, including but not limited to details of the currently executing task, remaining time estimate, resource usage, etc. This information helps users make timely decisions, such as whether to add more resources or adjust the order of tasks.

[0064] App Download: The client includes a built-in access to the app market, allowing users to browse and download apps that suit their needs. Each app comes with a detailed description and user reviews to help users make informed choices. Once downloaded, the app is automatically installed on the user's Brainbot, ready for use. Updates: To ensure system stability and security, the client regularly checks for updated versions of installed apps. Once a new version is found, the user will be prompted to update. The update process is typically automated, requiring users to confirm the update without worrying about data loss or other issues.

[0065] Task issuance: Users can issue new task instructions to the brain-equipped robot through the client. There are various input methods, including natural language instructions, screen content recognition and voice interaction. Users only need to briefly describe the task objectives, such as "produce a quarterly financial report PPT", and the client will convert this instruction into specific task requirements and send it to the brain-equipped robot unit for processing.

[0066] Execution process intervention: During the task execution process, if the user finds any deviation or needs for adjustment, they can intervene in real time through the client; for example, the user can add new subtasks midway, modify existing task parameters, or even pause the current task to await further instructions; this flexibility greatly enhances the controllability of task execution.

[0067] Result confirmation: After the task is completed, the client will receive a result report returned by the brain-powered robot. The result report lists in detail the entire process of task execution, including the problems encountered and their solutions. The user can confirm whether the task has achieved the expected results based on the report content. If there is any dissatisfaction, the user can choose to re-execute the task or adjust the parameters and try again.

[0068] Through the detailed steps described above, the human-computer interaction client and app / mini-program not only simplify the user's operation process but also improve the transparency and controllability of task execution, making the entire intelligent agent system more closely aligned with actual needs and enhancing user experience and satisfaction. This also promotes the widespread application of intelligent agent systems in more fields, meeting the needs of intelligent office and automation in different scenarios.

[0069] The resource management platform 220 is used to dynamically schedule computing resources, referring to Figure 3 , specifically including: device and user management module; resource matching algorithm module, used to evaluate and allocate optimal computing resources according to task requirements; blockchain registration module, which records the ownership and usage rights information of tasks; sandbox execution environment module, which is used to create an independent and secure operating environment for each task.

[0070] It's important to note that the Device and User Management module is responsible for network management and status monitoring of all robotic units within the system. It dynamically assesses task requirements and intelligently allocates optimal computing resources, whether for local processing, edge servers, or cloud execution, ensuring each task runs in the optimal environment. Furthermore, the Device and User Management module creates an independent, secure sandbox environment to isolate and protect each task, preventing data leakage and mutual interference, thereby ensuring system stability and security.

[0071] Furthermore, the device and user management module handles user permission control and authentication, ensuring that only authorized users can access specific resources or operate the brain-powered robot. It also leverages blockchain technology to record task ownership and usage rights, ensuring these records are transparent and tamper-proof, enhancing the system's credibility. In this way, the device and user management module not only improves system reliability and efficiency but also addresses the privacy and security issues inherent in traditional intelligent agents, supporting modern businesses and individuals in achieving efficient and secure intelligent office and automation needs.

[0072] Among them, the detailed execution steps of the resource matching algorithm module are: receiving task requirement data from the brain-equipped robot unit, including detailed information of each subtask (such as required computing resources, storage resources and network bandwidth), and analyzing the demand characteristics of the task. For example, computing-intensive tasks may require more CPU and GPU resources, while data-intensive tasks require more storage and network bandwidth.

[0073] Conduct a detailed resource requirement analysis for each subtask, including but not limited to: computing resources, determining the required number of CPU cores and GPUs, etc.; storage resources, determining the required memory size and hard disk space, etc.; network bandwidth, determining the required network bandwidth to ensure efficient data transmission.

[0074] A multi-objective optimization algorithm is used to comprehensively consider multiple factors (such as task completion timeliness, computing resource costs, data security level and system load balancing) to generate the optimal resource scheduling plan; based on the current system resource usage, resource allocation is dynamically adjusted to ensure that each subtask can be executed under optimal conditions, and to determine which resources can be shared and which resources need to be allocated independently to avoid resource conflicts and improve overall efficiency.

[0075] The generated optimal scheduling plan is applied to actual resource allocation to ensure that each subtask can obtain the required computing resources. Resource usage can be dynamically monitored and resource allocation can be re-evaluated and adjusted when necessary to cope with changes during task execution.

[0076] Detailed steps for the blockchain registration module: When a user submits a task request through a client or app / mini-program, the blockchain registration module records the task's ownership information, including the task's creation time, creator identity, and a basic description. It also records the task's usage rights, clarifying which users or systems have permission to access and operate the task. This information includes a list of authorized users, permission levels (e.g., read, write, delete), and the expiration date of the permissions.

[0077] It's important to note that all task ownership and usage rights information is stored on the blockchain, ensuring transparency and immutability. A query interface is also provided, allowing stakeholders to view task ownership and usage rights information, enhancing the system's security and credibility. If ownership or usage rights change during task execution, the blockchain registration module promptly updates the relevant information and regularly checks and maintains the records on the blockchain to ensure data integrity and consistency.

[0078] Detailed execution steps of the sandbox execution environment module: Based on task requirements, an independent secure operating environment is created for each subtask on the resource management platform. Each sandbox environment is isolated to prevent mutual interference and data leakage between different tasks.

[0079] Based on the task requirements and permission settings, allocate corresponding computing resources (such as CPU, memory), storage resources (such as hard disk space) and network bandwidth to the sandbox environment. It is necessary to set the operation permissions of the sandbox environment to ensure that only authorized operations can be performed in the environment.

[0080] Deploy task-related files and data to the corresponding sandbox environment, start task execution, and simulate keyboard and mouse operations on computer software to complete the task. All operations are performed in the sandbox environment to ensure system security.

[0081] At the same time, the resource usage of the sandbox environment is monitored in real time, including CPU utilization, memory usage, network bandwidth, etc. If insufficient resources or performance bottlenecks are found, resource allocation is dynamically adjusted to ensure smooth task execution. The security status of the sandbox environment is monitored to prevent malicious attacks or unauthorized operations.

[0082] After the task is completed, the temporary files and data in the sandbox environment are cleaned up to release the occupied computing and storage resources, thereby ensuring that the sandbox environment is in a clean state before the next task is started and avoiding the impact of residual data on new tasks.

[0083] Through the above execution steps, the resource management platform can efficiently and securely handle various complex task requirements, improve the application scenarios and efficiency of the system, and the multi-level resource management and security assurance mechanism not only improves the efficiency and flexibility of task processing, but also ensures the reliability and security of the system, meeting the high demands of modern enterprises and individuals for intelligent office and automation.

[0084] The application market platform 230 is used to provide a developer tool chain and a revenue distribution mechanism. Figure 4 Specifically, it includes: developer SDK, which is used to provide API and tool chain, and is used for application development tasks; application review and release unit, which is used for automated testing and security verification; revenue distribution smart contract module, which is used to support revenue models, including pay-per-use, subscription and profit-sharing.

[0085] Among them, the detailed execution steps of the developer SDK are: developers register as developers through the application market platform, fill in the necessary information and create the corresponding development environment, and developers download and install the developer SDK, which contains a series of API interfaces, tool chains and sample codes to help developers get started quickly.

[0086] Developers can then interact with the Brain Robot system using the APIs provided by the developer SDK. For example, they can use the API to submit tasks, query task status, and manage resources. They can also use the large model fine-tuning toolchain in the SDK to customize the pre-trained large model to meet the needs of specific application scenarios.

[0087] Developers write application code based on specific needs to implement specific task functions, such as data processing and automated report generation. After completing application development, developers integrate the application into the brain-powered robot system to ensure that the application can seamlessly connect with various modules of the system. Finally, developers test the application's functions to ensure that it can operate normally in different scenarios.

[0088] Detailed steps for executing the application review release unit:

[0089] Developers package their developed applications and submit them to the Brain Robot Application Market. During the submission process, they need to provide basic application information (such as name, description, version number) and related files (such as source code, configuration files, etc.); the application review and release unit automatically performs functional testing on the submitted applications to ensure that all functions of the application can work normally.

[0090] The test content includes but is not limited to: whether the application can submit tasks correctly, whether it can accurately obtain task status, whether it can effectively manage computing resources, etc.; automated testing also includes performance testing to ensure that the application can run stably under high load conditions.

[0091] The app is then subject to security verification to check for potential security vulnerabilities or malicious code. Verification includes, but is not limited to, input validation, permission control, and data encryption to ensure the app poses no threat to user data. Security verification also includes code review to ensure compliance with best practices and security standards.

[0092] After the review is completed, the system will feedback the review results to the developer. If the application passes all tests and verifications, the system will notify the developer that the application has passed the review; if the application fails the review, the system will provide detailed feedback information, point out the problems, and suggest that the developer make modifications and resubmit.

[0093] Detailed execution steps for the revenue distribution smart contract module: Developers can choose a suitable revenue model based on the characteristics of their application. Common revenue models include: pay-per-use, where the developer receives a percentage of the fee each time a user uses the app; a subscription model, where the developer receives a fixed percentage of the revenue on a monthly or annual basis after the user subscribes to the app; and a profit-sharing model, where the revenue distribution ratio is dynamically adjusted based on the frequency of application usage and resource consumption.

[0094] Developers set specific profit distribution rules on the application market platform. For example, developers can choose to distribute part of the profits to partners or platform operators. The system automatically generates smart contracts based on the set profit distribution rules to ensure that the profit distribution process is transparent and cannot be tampered with.

[0095] When users purchase and use an app, the system records their usage in real time and calculates the developer's revenue based on the revenue model. The smart contract module automatically distributes revenue to the relevant parties (such as developers, partners, platform operators, etc.) according to the set ratio. By providing transparent revenue records, developers can view detailed revenue sources and distribution through the app market platform.

[0096] Through regular (such as monthly or quarterly) revenue settlement, the developer's earnings will be aggregated and transferred to their designated account; developers can initiate withdrawal requests through the application market platform to withdraw earnings to their own bank accounts or other designated accounts.

[0097] Through the detailed execution steps described above, the application market platform can efficiently support developers in development, testing, release and profitability, improving the application scenarios and efficiency of the entire system; this multi-level support mechanism not only improves developers' development efficiency and application quality, but also promotes the prosperity and development of the application market, meeting the high demands of modern enterprises and individuals for intelligent office and automation.

[0098] In an embodiment of the present application, the execution control module of the brain-equipped robot unit receives an execution plan generated by the large model decision module. The plan includes the various subtasks of the task, specific operation steps and dependencies; then the execution plan is parsed, including the goals of each subtask, the required resources (such as CPU, memory), operation steps (such as opening specific software, importing data, etc.) and dependencies (such as a subtask must be started after another subtask is completed).

[0099] Prepare the corresponding execution environment according to the requirements of the execution plan. For example, if a subtask requires the use of specific software or tools, ensure that these software or tools have been installed and configured.

[0100] Based on the execution plan, keyboard and mouse operations are simulated to automatically complete the corresponding computer software tasks. The specific detailed execution steps include: initializing the operating environment to ensure that all required operating environments are ready, such as starting related software applications (such as Excel, PowerPoint, etc.) and ensuring that they are in the correct state (such as opening a specific workbook or slide); simulating keyboard and mouse operations, according to the operation steps in the execution plan, simulate keyboard and mouse input operations, for example: simulate clicking the "File" menu and selecting "Open" to open the specified file; simulate entering data or formulas into a spreadsheet; simulate clicking the "Save" button to save the current work progress.

[0101] For complex operation steps, a combination of multiple keyboard and mouse operations may be required. For example, when generating a chart, you may need to click the "Insert" menu, select "Chart Type", enter the data range, etc.; automate these complex operations to ensure that each step is completed accurately. If the execution plan contains multiple subtasks, ensure that these subtasks can be executed sequentially or in parallel. For example, some subtasks can be performed simultaneously (such as downloading multiple files at the same time), while other subtasks need to wait for the previous subtask to complete (such as collecting data before generating a report).

[0102] At the same time, the task execution process is monitored in real time, and the execution results are fed back to the large model decision module. The detailed execution steps include: During the task execution process, the status of each subtask is monitored in real time, which includes but is not limited to: whether the subtask has started execution, the progress of the current subtask (such as the percentage completed), and whether the subtask encounters any problems or exceptions.

[0103] The execution results of each subtask must be recorded, including success or failure information. For example, this could include whether a subtask successfully generated the expected chart or encountered issues such as missing data. These results should be fed back to the large model decision module. The model decision module can adjust subsequent task schedules or generate new execution plans based on the actual execution results.

[0104] During task execution, detect any anomalies in real time. Common anomalies include, but are not limited to, missing or incorrectly formatted data, software crashes or unresponsiveness, network interruptions, or other external factors that cause task interruptions.

[0105] When an anomaly is detected, analyze the specific cause. For example, if data is missing, determine which data source is at fault. If a software crash occurs, examine the log files to determine the cause. Take corrective action based on the cause. For example, if data is missing, attempt to obtain data from an alternative data source or prompt the user to manually enter the missing data. If a software crash occurs, attempt to restart the software and restore it to its pre-crash state to continue the task.

[0106] If an unexpected situation prevents some subtasks from being executed as planned, adjust the execution plan to accommodate the new situation, for example, by reallocating resources, adjusting task priorities, or skipping the affected subtasks. After taking corrective actions, continue with the remaining tasks. Ensure that all unfinished subtasks are successfully executed until the entire task is completed.

[0107] The role of the system in the embodiment of the present application can be illustrated by the following case:

[0108] Case 1: A user issues the task of “Complete the quarterly financial report PPT” through the mobile app.

[0109] When a user enters the command "Complete the quarterly financial report PPT" through the mobile app, the system first receives and interprets the task requirements through the multimodal perception module. This module uses natural language processing technology to convert the user's natural language instructions into structured task objectives, identifying the need to generate a quarterly financial report PPT and automatically breaking it down into three subtasks: data collection, chart generation, and layout design. Users can also input requirements through voice or screen content, and the multimodal perception module can accurately understand and convert them into executable instructions for the system, greatly improving the flexibility of task input.

[0110] The large-scale model decision module then intervenes, intelligently categorizing tasks based on complexity: data collection is a simple, immediate task suitable for rapid processing by a local, lightweight model; chart generation requires moderate computing resources and calls upon a medium-sized model on an edge server; and layout design involves complex layouts and deep optimization, relying on a large cloud-based model. The module generates a detailed execution plan, clearly defining the priorities and dependencies of each subtask (e.g., triggering chart generation after data collection is complete), and assigns these tasks to three intelligent robots for collaborative execution. This hybrid deployment model (local + edge + cloud) reduces excessive consumption of cloud resources while ensuring efficient task execution.

[0111] The resource management platform 220 then dynamically schedules resources. The resource matching algorithm module analyzes the requirements of each subtask, allocates local computing resources for data collection, allocates GPU resources of the edge server for chart generation, and allocates high-performance computing power in the cloud for layout design. The blockchain registration module synchronously records the ownership (user identity, creation time) and usage rights (authorization of brain-powered robot units to execute) of the task to ensure that the task ownership is transparent and cannot be tampered with. The sandbox execution environment module creates an independent and secure operating environment for each subtask. For example, the data collection sandbox isolates access to the financial system, the chart generation sandbox restricts network permissions, and the layout design sandbox ensures the independence of the design tools. The resource management platform monitors the resource usage of each sandbox in real time and dynamically adjusts the allocation (such as increasing GPU resources when chart generation is stuck) to avoid performance bottlenecks.

[0112] The execution control module simulates keyboard and mouse operations according to the execution plan. For example, the data collection phase simulates opening the financial system webpage and exporting data to Excel; the chart generation phase simulates selecting data and generating charts in Excel; and the layout design phase simulates applying templates and adjusting layouts in PowerPoint. During execution, the system monitors the task status in real time. If a data format error is detected, causing chart generation to stall, the system automatically replenishes data from an alternative data source and restarts the task. If cloud resources are insufficient, additional GPU resources are dynamically allocated. After the final PPT is generated, the execution results are fed back to the large model decision module. After the task is completed, the sandbox environment is cleaned up and resources are released.

[0113] The collaboration of multimodal perception + large model decision + execution control modules realizes the fully automated processing of complex tasks. Users no longer need to manually operate Excel or PowerPoint, which significantly improves efficiency and accuracy. The resource management platform solves problems such as insufficient resources of ordinary computers and the risk of data leakage through dynamic scheduling and sandbox isolation, ensuring that tasks are efficiently executed in a safe and stable environment.

[0114] Case 2: Developers use SDK to develop PCB design applications. After the application is launched, users pay for its use and share the profits.

[0115] Developers first register an account on the application market platform 230, download and install the developer SDK. The API interface (such as submitting tasks and managing resources) and tool chain (such as large model fine-tuning tools) provided by the SDK help developers quickly develop PCB design applications. For example, developers can call cloud models to optimize circuit board layouts, or customize models by fine-tuning tool chains to adapt to specific design rules. After development is completed, developers integrate the application into the brain-powered robot system and test its functions (such as automatic layout and wiring verification) to ensure compatibility. This process is seamlessly connected through the SDK tool chain.

[0116] After developers submit their apps to the app market, the App Review and Release Unit initiates automated testing and security verification. Functional testing verifies that the app can correctly submit PCB design tasks and manage resources; performance testing ensures stability under high load. Security verification checks the code for vulnerabilities (such as unauthorized hardware access) and confirms that data encryption and permission control meet standards. After passing the review, developers use the revenue distribution smart contract module to set a profit-sharing model, for example, 30% of user fees goes to the developer, and 70% goes to the platform or partner. The smart contract automatically generates a distribution agreement, ensuring the revenue process is transparent and tamper-proof.

[0117] When users purchase and use an app, the system records usage data in real time. For example, if a user pays 100 yuan per use, the smart contract automatically allocates 30 yuan to the developer. If a user subscribes for a 50 yuan monthly fee, the developer receives a proportional share of the proceeds each month. Developers can view earnings details on the platform and regularly initiate withdrawal requests to transfer earnings to their bank accounts. The app market platform continuously collects user feedback, supporting developers in iterating and optimizing their apps, repeatedly submitting new versions, and passing the review process.

[0118] The developer SDK and application review and release unit provide developers with a complete tool chain and quality assurance, lowering the development threshold while ensuring the security and stability of applications; the profit distribution smart contract module encourages developers to participate in ecosystem construction through a transparent profit-sharing mechanism, promoting the prosperity of the application market; the end-edge-cloud collaborative architecture enables developers to flexibly call on computing resources at different levels (such as cloud-based model optimization design), solving the performance limitations of traditional PCB design tools.

[0119] The present application embodiment discloses a method for collaborative management of a robot with a brain, edge and cloud. Figure 5 , including the following steps:

[0120] S100, based on the task requirements input by the user through the client or APP / mini-program, the multimodal perception module understands the natural language instructions, screen content and voice interaction to obtain the corresponding task requirement data, parses the task requirement data and generates a corresponding structured task tree;

[0121] S110: The large model decision module analyzes task requirements and generates a detailed execution plan. Based on the structured task tree, the main task corresponding to the task requirement data is decomposed into multiple subtasks. The resource matching algorithm module evaluates the computing resources required for each subtask based on the task requirement data and generates a corresponding optimal scheduling plan. The sandbox execution environment module creates an independent and secure operating environment for each subtask and assigns corresponding operation permissions. The resource management platform monitors the execution process of each subtask in real time and dynamically adjusts resources when necessary through the resource matching algorithm module.

[0122] S120, based on the execution plan, the execution control module simulates keyboard and mouse operations on the computer software to complete the corresponding task. After the task is completed, a corresponding execution report is generated. Among them, the blockchain registration module updates the ownership record and feeds back the results to the user through the developer SDK.

[0123] An embodiment of the present application also discloses a brain-based robot end-edge-cloud collaborative management system, including a processor running a program of any one of the brain-based robot end-edge-cloud collaborative management methods described above.

[0124] An embodiment of the present application also discloses a storage medium storing a program of the brain-powered robot end-edge-cloud collaborative management method described in any one of the above.

[0125] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A brain-powered robot-edge-cloud collaborative management system, characterized in that: It includes four parts: a brain-powered robot unit, a resource management platform, a human-computer interaction client, an APP / applet, and an application market platform. A brain-powered robotic unit, designed to autonomously operate a computer to complete tasks, includes a multimodal perception module for understanding natural language commands, screen content, and voice interaction; The large model decision module is used to analyze task requirements, decompose and plan tasks, and generate detailed execution plans; An execution control module is used to simulate keyboard and mouse operations on computer software to complete tasks according to the execution plan; Human-computer interaction client and APP / applet, which are used for interactive interfaces, including activation, management, control, status query, application download and update of the brain-equipped robot, as well as task issuance, task execution process intervention and task execution result confirmation; The resource management platform is used to manage the brain-equipped robots through networking, assign permissions between the brain-equipped robots and their owners, and implement the division of labor and collaboration among multiple brain-equipped robots for dynamic scheduling of computing resources. Specifically, it includes: a device and user management module; a resource matching algorithm module for evaluating and allocating optimal computing resources based on task requirements; a blockchain registration module for recording task ownership and usage rights; and a sandbox execution environment module for creating an independent and secure operating environment for each task. The application market platform is used to provide developer tool chains and revenue distribution mechanisms, and implement the revenue distribution mechanism when Brainbot users pay to download and subscribe to applications. Specifically, it includes: Developer SDK, which is used to provide APIs and tool chains, and is used to develop applications for tasks; Application review and release unit, which is used for automated testing and security verification; Revenue distribution smart contract module, which is used to support revenue models, including pay-per-use, subscription and profit-sharing.

2. The brain-powered robot-edge-cloud collaborative management system according to claim 1 is characterized in that: The large model decision module of the brain-powered robot unit supports hybrid deployment modes, including: Based on the task requirements input by the user through the client or APP / Mini Program, obtain the corresponding task requirement data and determine the corresponding task type; If the task type is determined to be a simple immediate task requirement, the local lightweight model is used to parse the task requirement data and generate a corresponding execution plan; If the task type is determined to require high computing power, the task requirement data is sent to the edge server, and the medium-sized model on the edge server is used for collaborative reasoning to generate the corresponding execution plan; If the task type is determined to be a complex task, the task requirement data will be transmitted to the cloud, and deep processing will be performed using a large cloud model to generate a detailed execution plan, which will then be returned to the brain-powered robot unit.

3. The brain-powered robot-edge-cloud collaborative management system according to claim 2 is characterized in that: The resource management platform also includes a resource scheduling strategy based on a multi-objective optimization algorithm, and further includes: The task requirement data is parsed and a corresponding structured task tree is generated, and the main task corresponding to the task requirement data is decomposed into multiple subtasks. Perform resource requirement assessment for each subtask, including computing resources, storage resources, and network bandwidth. A multi-objective optimization algorithm is used to simultaneously evaluate task completion timeliness, computing resource costs, data security level, and system load balancing, and an optimal scheduling plan is generated based on the corresponding evaluation results.

4. The brain-powered robot-edge-cloud collaborative management system according to claim 1, characterized in that: The specific steps of the application market platform include: Register as a developer through the app market platform, establish a corresponding development environment, and use the developer SDK to develop applications for specific tasks; Obtain the target application submitted by the developer to the Robot App Market, and control the application review and release module to perform automated testing and security verification on the submitted target application. After review, the target application is put on the application market platform for users to download. Users purchase and download the target application through the client or APP / mini program, and use the brain-powered robot to complete the instructions.

5. The brain-powered robot-edge-cloud collaborative management system according to claim 1, characterized in that: The execution control module steps of the brain-powered robot unit include: The execution control module receives the execution plan generated by the large model decision module. The execution control module simulates keyboard and mouse operations based on the execution plan to automatically complete the corresponding computer software task; The execution control module monitors the task execution process in real time and feeds back the execution results to the large model decision module. If an exception occurs during the task execution, the execution control module will make adjustments based on the exception feedback information and continue to execute the task.

6. The brain-powered robot-edge-cloud collaborative management system according to claim 1, characterized in that: The developer SDK of the application market system specifically includes the following steps: Provides a brain-powered robot control API based on the developer SDK to allow developers to develop applications for specific tasks; Call the large model fine-tuning toolchain in the developer SDK and use it to enable developers to perform customized training on large models. After the developer develops the application using the developer SDK, the application developed by the developer is obtained and integrated into the brain-equipped robot unit.

7. A method for collaborative management of a robot with a brain, edge, and cloud, characterized in that: include: Based on the task requirements input by the user through the client or APP / Mini Program, the multimodal perception module understands natural language instructions, screen content and voice interaction to obtain the corresponding task requirement data, parses the task requirement data and generates the corresponding structured task tree; The large-scale decision-making module analyzes task requirements and generates a detailed execution plan. Based on a structured task tree, the main task corresponding to the task requirement data is decomposed into multiple subtasks. The resource matching algorithm module evaluates the computing resources required for each subtask based on the task requirement data and generates the corresponding optimal scheduling plan. The sandbox execution environment module creates an independent and secure operating environment for each subtask and assigns corresponding operating permissions. The resource management platform monitors the execution process of each subtask in real time and dynamically adjusts resources through the resource matching algorithm module. Based on the execution plan, the execution control module simulates keyboard and mouse operations on the computer software to complete the corresponding task. After the task is completed, a corresponding execution report is generated. Among them, the blockchain registration module updates the ownership record and feeds back the results to the user through the developer SDK.

8. A brain-powered robot-edge-cloud collaborative management system, characterized in that: It includes a processor running a program of the brain-based robot end-edge-cloud collaborative management method as described in any one of claim 7.

9. A storage medium, characterized in that: A program storing the method for collaborative management of the end-edge-cloud of a brain-powered robot as described in any one of claims 7 is stored.

Images