AI task assistant remote execution method and system based on desktop cloud sandbox

Abstract

The invention discloses an AI task assistant remote execution method and system based on a desktop cloud sandbox, belongs to the technical field of artificial intelligence, and aims to solve the technical problem of how to release a local computer desktop, eliminate potential safety hazards of a local system and further improve the utilization rate of desktop cloud when an AI task assistant executes a task. According to the technical scheme, a physical decoupling architecture based on interaction, planning and execution and a task execution mechanism based on planning, execution and observation are combined with the standardized packaging of a remote control instruction set and the fusion design of task visualization and a remote takeover channel, so that a user can submit tasks through a local computer browser; and the AI task assistant autonomously completes task planning, remotely calls a desktop cloud built-in tool, observes a closed-loop process of an execution result, and realizes task progress visualization and desktop cloud remote takeover at the same time.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a method and system for remote execution of AI task assistants based on a desktop cloud sandbox. Background Technology

[0002] With the rapid development of artificial intelligence technology, AI task assistants are gradually becoming core tools for improving office efficiency and intelligence. These assistants can achieve natural language understanding, autonomous task planning, and automatic task execution, significantly reducing the cost of human intervention and improving response speed and decision-making quality. However, existing AI task assistants usually execute tasks locally, which poses problems such as occupying the local computer desktop and threatening local system security.

[0003] With the increasing maturity of desktop cloud technology, cloud PCs have become a mature virtualized office solution. By centrally deploying users' desktop environments in the cloud, they achieve lightweight terminal devices, centralized management of data assets, and efficient cross-regional collaboration. However, users generally already have their own local computers before accessing cloud PCs, leading to problems such as low utilization rates and resource waste.

[0004] With the accelerated advancement of digital transformation, single technologies are no longer sufficient to meet enterprises' comprehensive needs for security, flexibility, and intelligence.

[0005] Therefore, how to release the local computer desktop and eliminate local system security risks when the AI ​​task assistant performs tasks, thereby improving the utilization rate of desktop cloud, is a technical problem that urgently needs to be solved. Summary of the Invention

[0006] The technical objective of this invention is to provide a remote execution method and system for an AI task assistant based on a desktop cloud sandbox, in order to solve the problem of how to release the local computer desktop, eliminate local system security risks, and thereby improve the utilization rate of the desktop cloud when the AI ​​task assistant executes tasks.

[0007] The technical task of this invention is achieved as follows: a remote execution method for an AI task assistant based on a desktop cloud sandbox. This method is based on a physical decoupling architecture of interaction, planning, and execution, and a task execution mechanism based on planning, execution, and observation. It combines the standardized encapsulation of remote control instruction sets with the integrated design of task visualization and remote takeover channels. This enables users to submit tasks through a local computer browser, and the AI ​​task assistant to autonomously complete task planning, remotely call built-in desktop cloud tools, and observe execution results in a closed-loop process. At the same time, it enables visualization of task progress and remote takeover by the desktop cloud.

[0008] As a preferred option, the physical decoupling architecture based on interaction, planning, and execution includes a user interaction layer, a task planning layer, and a task execution layer.

[0009] The user interaction layer runs on the user's local computer browser and is used to receive the user's natural language input and present the task status.

[0010] The task planning layer runs in a cloud-based performance AI service cluster environment and is used to transform high-level semantic requirements into executable action sequences.

[0011] The task execution layer runs in a cloud computer virtual machine and is used to execute operation commands in a remote desktop environment.

[0012] As a preferred approach, the task execution mechanism based on planning, execution, and observation achieves a closed-loop process of task planning, task execution, and result observation through a task planner, a task executor, and a result observer.

[0013] The task planner is used to break down the task list according to user needs.

[0014] The task executor is used to read the task list, invoke remote start to execute tasks, and update task progress;

[0015] The results observer is used to determine whether the problem has been resolved: if not, the task is replanned; if resolved, a report is generated.

[0016] Preferably, the remote control instruction set includes browser operation instruction set, file operation instruction set, application management operation instruction set, application interface operation instruction set, terminal command operation instruction set, and code execution operation instruction set;

[0017] The browser operation command set includes browser_close, browser_resize, browser_console_messages, browser_handle_dialog, browser_file_upload, browser_pre ss_key, browser_navigate, browser_navigate_back, browser_navigate_fo rward, browser_network_requests, browser_pdf_save, browser_take_scre enshot, browser_click, browser_drag, browser_hover, browser_type, br There are 22 standard interfaces in total: owser_select_option, browser_tab_list, browser_tab_new, browser_tab_select, browser_tab_close, and browser_wait_for;

[0018] The file operation command set includes eight standard interfaces: fs_create_dir, fs_edit_file, fs_get_file_info, fs_read_file, fs_list_dir, fs_move_file, fs_search_file, and fs_write_file.

[0019] The application management operation command set includes six standard interfaces: app_get_installed, app_start, app_stop_by_pname, app_stop_by_pid, app_stop_by_cmd, and app_list_visible.

[0020] The application interface operation instruction set includes eight standard interfaces: app_ui_list_window, app_ui_get_active_window, app_ui_activate_window, app_ui_maximize_window, app_ui_minimize_window, app_ui_restore_window, app_ui_close_window, and app_ui_resize_window.

[0021] The terminal command operation instruction set includes one standard interface: command_execute.

[0022] The code execution instruction set includes one standard interface: code_run.

[0023] Furthermore, the integrated design of task visualization and remote takeover channels is as follows:

[0024] Task execution status visualization: Using MQTT protocol streaming and WebSocket protocol technology, the task execution status is pushed to the user's local computer browser in real time, in the following format:

[0025]

[0026] Cloud desktop visualization: Using VNC and WebSocket protocol streaming technologies, the user can view the cloud desktop status in real time on their local computer's browser, as shown below:

[0027]

[0028] Task blocking and remote takeover: When encountering a login website or verification code input operation that requires manual intervention, prompt the user to take over the cloud computer and block the execution of the AI ​​task. After the manual operation is completed, prompt the user to continue execution.

[0029] More specifically, task blocking and remote takeover are defined as follows:

[0030] Construct a state set in the following form:

[0031] Construct an event set in the form: ε = {Intervention Required, User Takeover, Continue Execution};

[0032] Construct a state transition function in the following form:

[0033]

[0034] A remote execution system for an AI task assistant based on a desktop cloud sandbox, the system comprising:

[0035] The physical decoupling architecture unit is used to decouple the interaction, planning, and execution between the user's local computer browser, the cloud AI service cluster, and the cloud computer virtual machine;

[0036] The task execution mechanism unit is used to realize a closed-loop process in which users submit tasks through their local computer browsers, the AI ​​task assistant autonomously completes task planning, remotely calls the desktop cloud built-in tools, and observes the execution results.

[0037] Standardized encapsulation unit, used to standardize the encapsulation of remote control instruction sets of 46 standardized interfaces in 6 major categories;

[0038] The integrated design unit is used to integrate task visualization and remote takeover channels, enabling task progress visualization and remote takeover via desktop cloud.

[0039] As a preferred embodiment, the physical decoupling architecture unit includes a user interaction layer, a task planning layer, and a task execution layer;

[0040] The user interaction layer runs on the user's local computer browser and is used to receive the user's natural language input and present the task status.

[0041] The task planning layer runs in a cloud-based performance AI service cluster environment and is used to transform high-level semantic requirements into executable action sequences.

[0042] The task execution layer runs in a cloud computer virtual machine and is used to execute operation commands in a remote desktop environment;

[0043] The task execution mechanism unit includes a task planner, a task executor, and a result observer;

[0044] The task planner is used to break down the task list according to user needs.

[0045] The task executor is used to read the task list, invoke remote start to execute tasks, and update task progress;

[0046] The results observer is used to determine whether the problem has been resolved: if not, the task is replanned; if resolved, a report is generated.

[0047] Preferably, the remote control instruction set includes browser operation instruction set, file operation instruction set, application management operation instruction set, application interface operation instruction set, terminal command operation instruction set, and code execution operation instruction set;

[0048] The browser operation command set includes browser_close, browser_resize, browser_console_messages, browser_handle_dialog, browser_file_upload, browser_pre ss_key, browser_navigate, browser_navigate_back, browser_navigate_fo rward, browser_network_requests, browser_pdf_save, browser_take_scre enshot, browser_click, browser_drag, browser_hover, browser_type, br There are 22 standard interfaces in total: owser_select_option, browser_tab_list, browser_tab_new, browser_tab_select, browser_tab_close, and browser_wait_for;

[0049] The file operation command set includes eight standard interfaces: fs_create_dir, fs_edit_file, fs_get_file_info, fs_read_file, fs_list_dir, fs_move_file, fs_search_file, and fs_write_file.

[0050] The application management operation command set includes six standard interfaces: app_get_installed, app_start, app_stop_by_pname, app_stop_by_pid, app_stop_by_cmd, and app_list_visible.

[0051] The application interface operation instruction set includes eight standard interfaces: app_ui_list_window, app_ui_get_active_window, app_ui_activate_window, app_ui_maximize_window, app_ui_minimize_window, app_ui_restore_window, app_ui_close_window, and app_ui_resize_window.

[0052] The terminal command operation instruction set includes one standard interface: command_execute.

[0053] The code execution instruction set includes one standard interface: code_run.

[0054] More preferably, the integrated design unit includes a task execution status visualization module, a cloud computer desktop visualization module, and a task blocking and remote takeover module;

[0055] The task execution status visualization module uses MQTT protocol streaming and WebSocket protocol technology to push the task execution status to the user's local computer browser in real time, in the following format:

[0056]

[0057] The cloud desktop visualization module uses VNC and WebSocket protocols to allow users to view the cloud desktop status in real time on their local computer's browser, as shown below:

[0058]

[0059] The task blocking and remote takeover module is used to prompt the user to take over the cloud computer and block the execution of the AI ​​task when encountering login websites or verification code input operations that require manual intervention. After the manual intervention is completed, the user is prompted to continue execution. The specific construction process of the task blocking and remote takeover module is as follows:

[0060] ① Construct a set of states in the following form:

[0061] ② Construct an event set in the form: ε = {Intervention required, User takeover, Continue execution};

[0062] ③ Construct the state transition function, in the following form:

[0063]

[0064] The AI ​​task assistant remote execution method and system based on desktop cloud sandbox of the present invention has the following advantages:

[0065] (I) This invention decouples the user interaction layer, task planning layer, and task execution layer, and combines this with a method of standardizing remote control instruction sets. This enables users to submit tasks through their local computer browser, and the AI ​​task assistant to autonomously complete task planning, remotely call the desktop cloud's built-in tools, and observe the execution results in a closed-loop process. It also provides the ability to visualize task progress and remotely take over the desktop cloud, thereby freeing up the local computer desktop, eliminating local system security risks, and improving the utilization rate of the desktop cloud. This solves the problems of traditional AI task assistants occupying the local computer desktop and threatening local system security when executing tasks locally.

[0066] (ii) This invention organically combines AI task assistant with desktop cloud technology, enabling the submission of tasks through a local computer browser and the execution of tasks using a remote cloud computer as a sandbox environment. This not only frees up the local computer desktop and eliminates local system security risks, but also improves the utilization rate of cloud computers.

[0067] (III) Traditional AI task assistants often require long-term use of local CPU, memory, and graphical interface resources when performing complex operations (such as automated script execution and multi-step data processing), preventing users from performing other office tasks in parallel. Compared with existing technologies, this invention migrates the task execution environment to a remote desktop cloud sandbox, allowing only a lightweight browser interaction layer to be retained locally, while all computationally intensive operations are completed in the cloud. This not only significantly reduces the load on local devices and extends hardware lifespan, but also enables low-configuration terminals to run highly complex AI-assisted tasks smoothly, greatly improving the availability and flexibility of terminal devices, completely freeing up local computing resources, and improving the efficiency of terminal device use.

[0068] (iv) During local execution, AI task assistants may call scripts from unknown sources, access sensitive files, or initiate network requests, posing security risks such as malicious code injection and privilege escalation. Compared with existing technologies, this invention constructs a secure and isolated task execution environment, eliminating local system security risks. Specifically, it uses a "desktop cloud sandbox" as the task execution container, and all operations are performed in a highly isolated virtual environment, completely decoupled from the local operating system. Even if the task contains potentially threatening behaviors (such as simulated clicks or automatic form submissions), it will not affect the real host system. At the same time, the sandbox supports on-demand destruction and snapshot recovery mechanisms, further strengthening the system's anti-attack capabilities and fundamentally eliminating the security risks brought about by AI automation.

[0069] (V) This invention innovatively proposes a three-tiered decoupled architecture of "user interaction layer - task planning layer - task execution layer," enabling the AI ​​task assistant to possess a complete intelligent link from understanding instructions to autonomous planning and then to remote execution. Users only need to submit natural language or structured task requests in a browser, and the system can automatically parse the target, break down sub-tasks, select appropriate toolchains, and drive the built-in applications in the remote desktop cloud to implement them one by one. During execution, the AI ​​can observe the interface feedback in real time and dynamically adjust subsequent actions, forming a closed-loop control logic of "perception - decision-making - execution - feedback," which greatly improves the task success rate and intelligence level, realizes closed-loop control of the entire task process, and enhances the AI's autonomous decision-making and execution capabilities.

[0070] (vi) To address the poor compatibility issues of traditional automation tools due to differences in operating systems and software versions, this invention designs a universal remote control instruction set standard, covering browser operations, file operations, application management operations, application interface operations, terminal command operations, and code execution primitives. This instruction set serves as a communication protocol between the task planning layer and the execution layer, shielding the differences in underlying heterogeneous environments. This allows the AI ​​task assistant to seamlessly call tool components from different desktop cloud instances. This standardized interface design not only reduces development and maintenance costs but also provides a reusable technical foundation for future expansion into more automation scenarios. Through the standardized remote control instruction set, unified scheduling capabilities for multiple tools across platforms are achieved.

[0071] (vii) Although the task is executed fully automatically by AI, the user's need for process transparency and emergency intervention cannot be ignored. Compared with the existing technology, this invention integrates functions such as real-time screen stream push, operation log recording and progress bar display to build an intuitive task progress dashboard in the local browser. Users can view the current execution steps, time statistics and intermediate result screenshots at any time, truly achieving "what you see is what you get". More importantly, this invention supports a one-click "remote takeover" mode. When the AI ​​encounters an anomaly or needs manual confirmation, the user can immediately obtain desktop cloud session control, manually complete key operations and then hand them back to the AI ​​to continue execution, balancing automation efficiency and human controllability. That is, this invention has visual monitoring and remote takeover functions, ensuring the user's understanding and intervention capabilities.

[0072] (viii) Many enterprises often face the problem of idle resources and insufficient utilization after deploying desktop cloud. Compared with existing technologies, this invention cleverly integrates the execution needs of AI task assistants with existing desktop cloud platforms, so that virtual desktops originally used for employees' daily office work can be scheduled as AI automated running nodes during idle periods, realizing the transformation from "one person, one machine" to "human and machine sharing, dynamic and static combination" resource mode. Through intelligent scheduling and load balancing mechanisms, multiple AI tasks can be executed in parallel in different sandbox instances, maximizing the utilization of cloud computing resources. This not only reduces the cost of adding dedicated servers, but also conforms to the trend of efficient digital transformation, effectively revitalizes desktop cloud resources, and promotes the intensive utilization of enterprise IT infrastructure. Attached Figure Description

[0073] The invention will be further described below with reference to the accompanying drawings.

[0074] Appendix Figure 1 This is a schematic diagram of a physical decoupling architecture based on interaction, planning, and execution.

[0075] Appendix Figure 2 This is a schematic diagram of a task execution mechanism based on planning, execution, and observation. Detailed Implementation

[0076] The following detailed description of the remote execution method and system for AI task assistant based on desktop cloud sandbox of the present invention is provided with reference to the accompanying drawings and specific embodiments.

[0077] Example 1:

[0078] This embodiment provides a remote execution method for an AI task assistant based on a desktop cloud sandbox. This method is based on a physical decoupling architecture of interaction, planning, and execution, and a task execution mechanism based on planning, execution, and observation. It combines the standardized encapsulation of remote control instruction sets with the integrated design of task visualization and remote takeover channels. This enables users to submit tasks through a local computer browser, and the AI ​​task assistant to autonomously complete task planning, remotely call built-in desktop cloud tools, and observe execution results in a closed-loop process. At the same time, it enables visualization of task progress and remote takeover by the desktop cloud.

[0079] As attached Figure 1 As shown, the physical decoupling architecture based on interaction, planning, and execution in this embodiment includes a user interaction layer, a task planning layer, and a task execution layer;

[0080] The user interaction layer runs on the user's local computer browser and is used to receive the user's natural language input and present the task status.

[0081] The task planning layer runs in a cloud-based performance AI service cluster environment and is used to transform high-level semantic requirements into executable action sequences.

[0082] The task execution layer runs in a cloud computer virtual machine and is used to execute operation commands in a remote desktop environment.

[0083] As attached Figure 2 As shown, the task execution mechanism based on planning, execution, and observation in this embodiment realizes a closed-loop process of task planning, task execution, and result observation through a task planner, a task executor, and a result observer.

[0084] The task planner is used to break down the task list according to user needs.

[0085] The task executor is used to read the task list, invoke remote start to execute tasks, and update task progress;

[0086] The results observer is used to determine whether the problem has been resolved: if not, the task is replanned; if resolved, a report is generated.

[0087] The remote control instruction set in this embodiment includes browser operation instruction set, file operation instruction set, application management operation instruction set, application interface operation instruction set, terminal command operation instruction set and code execution operation instruction set, as shown in Table 1, Table 2, Table 3, Table 4, Table 5 and Table 6.

[0088] Table 1 Browser Operation Command Set

[0089]

[0090]

[0091] Table 2 File Operation Command Set

[0092]

[0093] Table 3 Application Management Operation Command Set

[0094]

[0095] Table 4 Application Interface Operation Command Set

[0096]

[0097] Table 5 Terminal Command Operation Instruction Set

[0098] category instruction describe Terminal command operations command_execute Execute terminal commands

[0099] Table 6 Code Execution Operation Instruction Set

[0100] category instruction describe Code execution operation code_run Run the code; supports Node.js and Python.

[0101] The integration design of task visualization and remote takeover channel in this embodiment is as follows:

[0102] (1) Visualization of task execution status: Using MQTT protocol streaming and WebSocket protocol technology, the task execution status is pushed to the user's local computer browser in real time, in the following form:

[0103]

[0104] (2) Cloud Desktop Visualization: Using VNC protocol streaming and WebSocket protocol technology, the cloud desktop status can be viewed in real time in the user's local computer browser, as follows:

[0105]

[0106] (3) Task blocking and remote takeover: When encountering a login website or verification code input operation that requires manual intervention, prompt the user to take over the cloud computer and block the execution of the AI ​​task. After the manual operation is completed, prompt the user to continue execution.

[0107] The task blocking and remote takeover in step (3) of this embodiment are as follows:

[0108] ① Construct a set of states in the following form:

[0109] ② Construct an event set in the form: ε = {Intervention required, User takeover, Continue execution};

[0110] ③ Construct the state transition function, in the following form:

[0111]

[0112] Example 2:

[0113] This embodiment provides a remote execution system for an AI task assistant based on a desktop cloud sandbox. The system includes:

[0114] The physical decoupling architecture unit is used to decouple the interaction, planning, and execution between the user's local computer browser, the cloud AI service cluster, and the cloud computer virtual machine;

[0115] The task execution mechanism unit is used to realize a closed-loop process in which users submit tasks through their local computer browsers, the AI ​​task assistant autonomously completes task planning, remotely calls the desktop cloud built-in tools, and observes the execution results.

[0116] Standardized encapsulation unit, used to standardize the encapsulation of remote control instruction sets of 46 standardized interfaces in 6 major categories;

[0117] The integrated design unit is used to integrate task visualization and remote takeover channels, enabling task progress visualization and remote takeover via desktop cloud.

[0118] The physical decoupling architecture unit in this embodiment includes a user interaction layer, a task planning layer, and a task execution layer;

[0119] The user interaction layer runs on the user's local computer browser and is used to receive the user's natural language input and present the task status.

[0120] The task planning layer runs in a cloud-based performance AI service cluster environment and is used to transform high-level semantic requirements into executable action sequences.

[0121] The task execution layer runs in a cloud computer virtual machine and is used to execute operation commands in a remote desktop environment.

[0122] The task execution mechanism unit in this embodiment includes a task planner, a task executor, and a result observer;

[0123] The task planner is used to break down the task list according to user needs.

[0124] The task executor is used to read the task list, invoke remote start to execute tasks, and update task progress;

[0125] The results observer is used to determine whether the problem has been resolved: if not, the task is replanned; if resolved, a report is generated.

[0126] The remote control instruction set in this embodiment includes a browser operation instruction set, a file operation instruction set, an application management operation instruction set, an application interface operation instruction set, a terminal command operation instruction set, and a code execution operation instruction set.

[0127] The browser operation command set includes browser_close, browser_resize, browser_console_messages, browser_handle_dialog, browser_file_upload, browser_pre ss_key, browser_navigate, browser_navigate_back, browser_navigate_fo rward, browser_network_requests, browser_pdf_save, browser_take_scre enshot, browser_click, browser_drag, browser_hover, browser_type, br There are 22 standard interfaces in total: owser_select_option, browser_tab_list, browser_tab_new, browser_tab_select, browser_tab_close, and browser_wait_for;

[0128] The file operation command set includes eight standard interfaces: fs_create_dir, fs_edit_file, fs_get_file_info, fs_read_file, fs_list_dir, fs_move_file, fs_search_file, and fs_write_file.

[0129] The application management operation command set includes six standard interfaces: app_get_installed, app_start, app_stop_by_pname, app_stop_by_pid, app_stop_by_cmd, and app_list_visible.

[0130] The application interface operation instruction set includes eight standard interfaces: app_ui_list_window, app_ui_get_active_window, app_ui_activate_window, app_ui_maximize_window, app_ui_minimize_window, app_ui_restore_window, app_ui_close_window, and app_ui_resize_window.

[0131] The terminal command operation instruction set includes one standard interface: command_execute.

[0132] The code execution instruction set includes one standard interface: code_run.

[0133] The fusion design unit in this embodiment includes a task execution status visualization module, a cloud computer desktop visualization module, and a task blocking and remote takeover module.

[0134] The task execution status visualization module uses MQTT protocol streaming and WebSocket protocol technology to push the task execution status to the user's local computer browser in real time, in the following format:

[0135]

[0136] The cloud desktop visualization module uses VNC and WebSocket protocols to allow users to view the cloud desktop status in real time on their local computer's browser, as shown below:

[0137]

[0138] The task blocking and remote takeover module is used to prompt the user to take over the cloud computer and block the execution of the AI ​​task when encountering login websites or verification code input operations that require manual intervention. After the manual intervention is completed, the user is prompted to continue execution. The specific construction process of the task blocking and remote takeover module is as follows:

[0139] ① Construct a set of states in the following form:

[0140] ② Construct an event set in the form: ε = {Intervention required, User takeover, Continue execution};

[0141] ③ Construct the state transition function, in the following form:

[0142]

[0143] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for remotely executing an AI task assistant based on a desktop cloud sandbox, characterized in that, This method is based on a physical decoupling architecture of interaction, planning, and execution, and a task execution mechanism based on planning, execution, and observation. It combines the standardized encapsulation of remote control instruction sets with the integrated design of task visualization and remote takeover channels to achieve a closed-loop process in which users submit tasks through a local computer browser, and the AI ​​task assistant autonomously completes task planning, remotely calls the desktop cloud's built-in tools, and observes the execution results. At the same time, it realizes task progress visualization and remote takeover by the desktop cloud.

2. The remote execution method for an AI task assistant based on a desktop cloud sandbox according to claim 1, characterized in that, The physical decoupling architecture based on interaction, planning, and execution includes a user interaction layer, a task planning layer, and a task execution layer. The user interaction layer runs on the user's local computer browser and is used to receive the user's natural language input and present the task status. The task planning layer runs in a cloud-based performance AI service cluster environment and is used to transform high-level semantic requirements into executable action sequences. The task execution layer runs in a cloud computer virtual machine and is used to execute operation commands in a remote desktop environment.

3. The remote execution method for an AI task assistant based on a desktop cloud sandbox according to claim 1, characterized in that, The task execution mechanism based on planning, execution, and observation realizes a closed-loop process of task planning, task execution, and result observation through a task planner, a task executor, and a result observer. The task planner is used to break down the task list according to user needs. The task executor is used to read the task list, invoke remote start to execute tasks, and update task progress; The results observer is used to determine whether the problem has been resolved: if not, the task is replanned; if resolved, a report is generated.

4. The remote execution method for an AI task assistant based on a desktop cloud sandbox according to claim 1, characterized in that, The remote control command set includes browser operation command set, file operation command set, application management operation command set, application interface operation command set, terminal command operation command set, and code execution operation command set; The browser operation command set includes browser_close, browser_resize, browser_console_messages, browser_handle_dialog, browser_file_upload, browser_pre ss_key, browser_navigate, browser_navigate_back, browser_navigate_fo rward, browser_network_requests, browser_pdf_save, browser_take_scre enshot, browser_click, browser_drag, browser_hover, browser_type, br There are 22 standard interfaces in total: owser_select_option, browser_tab_list, browser_tab_new, browser_tab_select, browser_tab_close, and browser_wait_for; The file operation command set includes eight standard interfaces: fs_create_dir, fs_edit_file, fs_get_file_info, fs_read_file, fs_list_dir, fs_move_file, fs_search_file, and fs_write_file. The application management operation command set includes six standard interfaces: app_get_installed, app_start, app_stop_by_pname, app_stop_by_pid, app_stop_by_cmd, and app_list_visible. The application interface operation instruction set includes eight standard interfaces: app_ui_list_window, app_ui_get_active_window, app_ui_activate_window, app_ui_maximize_window, app_ui_minimize_window, app_ui_restore_window, app_ui_close_window, and app_ui_resize_window. The terminal command operation instruction set includes one standard interface: command_execute. The code execution instruction set includes one standard interface: code_run.

5. The remote execution method for an AI task assistant based on a desktop cloud sandbox according to any one of claims 1 to 4, characterized in that, The integration design of task visualization and remote takeover channel is as follows: Task execution status visualization: Using MQTT protocol streaming and WebSocket protocol technology, the task execution status is pushed to the user's local computer browser in real time, in the following format: Cloud desktop visualization: Using VNC protocol streaming and WebSocket protocol technology, the user can view the cloud desktop status in real time in their local computer browser, as shown below: Task blocking and remote takeover: When encountering a login website or verification code input operation that requires manual intervention, prompt the user to take over the cloud computer and block the execution of the AI ​​task. After the manual operation is completed, prompt the user to continue execution.

6. The remote execution method for an AI task assistant based on a desktop cloud sandbox according to claim 5, characterized in that, Task blocking and remote takeover are detailed below: Construct a state set in the following form: Construct an event set in the form: ε = {Intervention Required, User Takeover, Continue Execution}; Construct a state transition function in the following form:

7. A remote execution system for an AI task assistant based on a desktop cloud sandbox, characterized in that, The system includes: The physical decoupling architecture unit is used to decouple the interaction, planning, and execution between the user's local computer browser, the cloud AI service cluster, and the cloud computer virtual machine; The task execution mechanism unit is used to realize a closed-loop process in which users submit tasks through a local computer browser, the AI ​​task assistant autonomously completes task planning, remotely calls the desktop cloud built-in tools, and observes the execution results; Standardized encapsulation unit, used to standardize the encapsulation of remote control instruction sets of 46 standardized interfaces in 6 major categories; The integrated design unit is used to integrate task visualization and remote takeover channels, enabling task progress visualization and remote takeover via desktop cloud.

8. The AI ​​task assistant remote execution system based on desktop cloud sandbox according to claim 7, characterized in that, The physical decoupling architecture unit includes a user interaction layer, a task planning layer, and a task execution layer; The user interaction layer runs on the user's local computer browser and is used to receive the user's natural language input and present the task status. The task planning layer runs in a cloud-based performance AI service cluster environment and is used to transform high-level semantic requirements into executable action sequences. The task execution layer runs in a cloud computer virtual machine and is used to execute operation commands in a remote desktop environment; The task execution mechanism unit includes a task planner, a task executor, and a result observer; The task planner is used to break down the task list according to user needs. The task executor is used to read the task list, invoke remote start to execute tasks, and update task progress; The results observer is used to determine whether the problem has been resolved: if not, the task is replanned; if resolved, a report is generated.

9. The AI ​​task assistant remote execution system based on desktop cloud sandbox according to claim 7, characterized in that, The remote control command set includes browser operation command set, file operation command set, application management operation command set, application interface operation command set, terminal command operation command set, and code execution operation command set; The browser operation command set includes browser_close, browser_resize, browser_console_messages, browser_handle_dialog, browser_file_upload, browser_pre ss_key, browser_navigate, browser_navigate_back, browser_navigate_fo rward, browser_network_requests, browser_pdf_save, browser_take_scre enshot, browser_click, browser_drag, browser_hover, browser_type, br There are 22 standard interfaces in total: owser_select_option, browser_tab_list, browser_tab_new, browser_tab_select, browser_tab_close, and browser_wait_for; The file operation command set includes eight standard interfaces: fs_create_dir, fs_edit_file, fs_get_file_info, fs_read_file, fs_list_dir, fs_move_file, fs_search_file, and fs_write_file. The application management operation command set includes six standard interfaces: app_get_installed, app_start, app_stop_by_pname, app_stop_by_pid, app_stop_by_cmd, and app_list_visible. The application interface operation instruction set includes eight standard interfaces: app_ui_list_window, app_ui_get_active_window, app_ui_activate_window, app_ui_maximize_window, app_ui_minimize_window, app_ui_restore_window, app_ui_close_window, and app_ui_resize_window. The terminal command operation instruction set includes one standard interface: command_execute. The code execution instruction set includes one standard interface: code_run.

10. The remote execution system for an AI task assistant based on a desktop cloud sandbox according to any one of claims 7 to 9, characterized in that, The integrated design unit includes a task execution status visualization module, a cloud computer desktop visualization module, and a task blocking and remote takeover module; The task execution status visualization module uses MQTT protocol streaming and WebSocket protocol technology to push the task execution status to the user's local computer browser in real time, in the following format: The cloud desktop visualization module uses VNC protocol streaming and WebSocket protocol technology to allow users to view the cloud desktop status in real time in their local computer's browser, as shown below: The task blocking and remote takeover module is used to prompt the user to take over the cloud computer and block the execution of the AI ​​task when encountering login websites or verification code input operations that require manual intervention. After the manual intervention is completed, the user is prompted to continue execution. The specific construction process of the task blocking and remote takeover module is as follows: ① Construct a set of states in the following form: ② Construct an event set in the form: ε = {Intervention required, User takeover, Continue execution}; ③ Construct the state transition function, in the following form:

Images