Task processing method, device, equipment, system, medium and product

Abstract

The invention provides a task processing method, device, equipment, system, medium and product, and relates to the technical field of computers, in particular to the technical fields of artificial intelligence, large language models, cloud computing, cloud terminals, image user interface agents and the like. According to the specific implementation scheme, a user instruction sent by a user terminal for a target client is understood through an intelligent agent service implemented based on a large language model, task information of a corresponding target task is obtained, and a target software identifier of target control software matched with the task information is obtained from a preset knowledge base; and the target task is sent to the cloud terminal platform service, and the target control software is executed through the cloud terminal platform service, so that the target task is executed in the cloud terminal loaded with the target client and corresponding to the user terminal. By adopting the technical scheme, the large model and the cloud terminal are combined, the user task can be accurately completed at the cloud terminal based on the control software, and the task execution efficiency can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to the technical fields of artificial intelligence, large language models, cloud computing, cloud terminals, and image user interface intelligent agents. Background Art

[0002] With the rapid development of artificial intelligence, artificial intelligence has been widely used in various fields. Among them, intelligent agent technology is an important technology in the field of artificial intelligence. Users can interact with the terminal in a natural language dialogue to drive the intelligent agent to complete specific tasks autonomously. Summary of the invention

[0003] The present disclosure provides a task processing method, apparatus, device, system, medium and product.

[0004] According to one aspect of the present disclosure, there is provided a task processing method, comprising:

[0005] Understanding the user instruction sent by the user terminal to the target client through the intelligent agent service, and obtaining the task information of the target task corresponding to the user instruction, wherein the intelligent agent service is implemented based on the large language model;

[0006] Acquire the target software identifier of the target control software matching the task information from a preset knowledge base through the agent service, and send the target software identifier to the cloud terminal platform service;

[0007] The target control software is executed according to the target software identifier through the cloud terminal platform service to execute the target task in the cloud terminal corresponding to the user terminal, wherein the target client is loaded in the cloud terminal.

[0008] According to another aspect of the present disclosure, there is provided a task processing device, comprising:

[0009] A task information determination module, used to understand the user instruction sent by the user terminal to the target client through the intelligent agent service, and obtain the task information of the target task corresponding to the user instruction, wherein the intelligent agent service is implemented based on a large language model;

[0010] A software identification sending module, used to obtain a target software identification of a target control software matching the task information from a preset knowledge base through the agent service, and send the target software identification to a cloud terminal platform service;

[0011] The task processing module is used to execute the target control software according to the target software identifier through the cloud terminal platform service to execute the target task in the cloud terminal corresponding to the user terminal, wherein the cloud terminal is loaded with the target client.

[0012] According to another aspect of the present disclosure, there is provided a task processing system, comprising: an agent service, a cloud terminal platform service, and a cloud terminal;

[0013] The intelligent agent service is implemented based on a large language model, and is used to understand the user instructions sent by the user terminal to the target client, obtain the task information of the target task corresponding to the user instruction, and send the target software identifier of the target control software matching the task information obtained from the preset knowledge base to the cloud terminal platform service;

[0014] The cloud terminal platform service is used to execute the target control software according to the target software identifier sent by the agent service, so as to execute the target task in the cloud terminal;

[0015] The cloud terminal is used to load the target client corresponding to the user terminal and respond to the control of the cloud terminal platform service to execute the target task.

[0016] According to another aspect of the present disclosure, an electronic device is provided, which is configured with an intelligent agent service and / or a cloud terminal platform service, and the electronic device includes:

[0017] at least one processor; and

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform corresponding steps in the method described in the embodiment of the present disclosure.

[0020] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the embodiment of the present disclosure.

[0021] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the method described in any embodiment of the present disclosure is implemented.

[0022] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0024] Figure 1 is a flowchart of a task processing method provided according to an embodiment of the present disclosure;

[0025] Figure 2 is a flowchart of another task processing method provided according to an embodiment of the present disclosure;

[0026] Figure 3 It is a schematic diagram of the principle of a task processing solution provided according to an embodiment of the present disclosure;

[0027] Figure 4 is a flowchart of another task processing method provided according to an embodiment of the present disclosure;

[0028] Figure 5 is a structural diagram of a task processing system provided according to an embodiment of the present disclosure;

[0029] Figure 6 is a structural schematic diagram of a task processing device provided according to an embodiment of the present disclosure;

[0030] Figure 7 It is a block diagram of an electronic device used to implement the task processing method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0032] In order to facilitate the understanding of the technical solutions of the embodiments of the present disclosure, the following examples of applicable application scenarios are introduced. In many scenarios such as office scenarios, game entertainment scenarios, and life service scenarios, there are some requirements for automatically completing tasks based on user accounts and specific clients. For example, in office scenarios, there may be requirements for automatically participating in or exiting online meetings, completing meeting content recordings and summaries, etc.; in game entertainment scenarios, there may be requirements for automatically completing tasks such as game sign-ins and receiving game rewards; in life service scenarios, there may be requirements for tasks such as product or store rating review information aggregation, summaries, and automatic reviews after consumption. In the above scenario-based requirements, it is necessary to rely on specific applications and user accounts, and the data security requirements are high. The operation process in the process of automatic task completion has strong business attributes, and the threshold for understanding and participation is high. For example, different game applications have different ways of playing, and the way of participating in the task system is also different; the participation process of different online conference software is different; the after-sales feedback process of different life service applications is different, and so on.

[0033] Figure 1 The present invention provides a flowchart of a task processing method according to an embodiment of the present invention. The present invention can be applied to automatically execute the corresponding task of the client according to the user's natural language instruction. The method can be executed by a task processing device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 , the method specifically includes the following:

[0034] S101, understanding a user instruction sent by a user terminal to a target client through an agent service, and obtaining task information of a target task corresponding to the user instruction, wherein the agent service is implemented based on a large language model;

[0035] S102, obtaining a target software identifier of a target control software matching the task information from a preset knowledge base through the agent service, and sending the target software identifier to a cloud terminal platform service;

[0036] S103: executing the target control software according to the target software identifier through the cloud terminal platform service to execute the target task in the cloud terminal corresponding to the user terminal, wherein the cloud terminal is loaded with the target client.

[0037] Among them, the agent service is a service implemented based on a large language model. The sample data volume and model parameters of the large language model (LLM) (hereinafter referred to as the large model) can be at a massive level, for example, it can have billions or even hundreds of billions of parameters, which are trained on large-scale text data or data of other modalities, and can be used for various natural language processing tasks. It has good text generation capabilities and can be applied to language translation and question-answering systems. For example, the large language model can adopt an N-layer Transformer network structure with an encoder (Encoder) and a decoder (Decoder). It can be understood that the large language model can also be a model of other network structures, which is not limited here. The large language model involves large model base technology, large model retrieval-augmented generation (RAG) technology, large model artificial intelligence agent (AGENT) technology, and large model workflow technology. The following introduces the various capabilities of the large model for the above-mentioned technologies.

[0038] Among them, for the large model base technology, it has strong language understanding and generation capabilities, can understand the input text content, and generate accurate, fluent, grammatical and logical text replies. Knowledge reserve and generalization capabilities can perform a certain degree of logical reasoning and analysis, and can derive reasonable conclusions based on known information and rules, analyze the cause and effect relationship and logical order of problems, etc. Knowledge reserve and generalization capabilities can use knowledge graphs to query, reason and update knowledge, provide users with more accurate and comprehensive knowledge services, and can also use common sense knowledge to reason and judge some questions to improve the rationality and credibility of answers. Sentiment analysis capabilities can analyze and judge the emotional tendencies in information such as text or voice. Intent understanding capabilities can accurately understand the user's intentions and needs. Multi-round dialogue and context understanding capabilities can remember the previous dialogue content and context information, understand the user's current question intention based on the context, and provide more coherent and accurate answers. As a basic framework, it has good scalability and can adapt to different application scenarios and needs through further training, fine-tuning or combination with other technologies. Multimodal information processing capabilities involve the combined understanding and generation of text and images, the ability to understand image content and associate it with text information, and the combined processing of text and audio, the ability to process audio information and convert and associate it with text. Explanability and transparency capabilities enable large models to generate corresponding explanations for their decisions and answers so that users can understand their reasoning process and basis, and can also display their internal working mechanisms and decision-making processes.

[0039] Among them, for the large model RAG technology, the large model has the ability to integrate knowledge, accurately answer and dynamically update. Among them, the knowledge integration ability includes external knowledge introduction and multi-source information fusion. External knowledge introduction refers to the ability to retrieve information related to user questions from external data sources (such as databases, document collections, and web pages), and integrate this information with the knowledge of the large model itself to provide users with more accurate, comprehensive and real-time answers. Multi-source information fusion refers to the ability to simultaneously integrate knowledge from multiple different sources and in different formats, including structured data and unstructured data, and to fuse and process this information to extract key content and key points, providing rich materials for generating answers. Accurate answering ability verifies and supplements the answers of the large model by retrieving external knowledge to enhance the credibility of the answers. For complex problems, it can use the retrieved knowledge for in-depth analysis and reasoning, find the key to the problem, and provide detailed and accurate solutions. It can decompose the problem into multiple sub-problems, retrieve relevant knowledge separately and conduct comprehensive analysis, and finally form a complete answer. For example, when dealing with some problems that require multi-step reasoning or comprehensive multi-faceted factors, RAG can provide high-quality answers with the help of external knowledge and the reasoning ability of the large model. Dynamic updating capability means that when information in external knowledge sources is updated, RAG can retrieve the latest knowledge in a timely manner and apply it to the answer to maintain the timeliness of the answer.

[0040] For large-model AGENT technology, the goal-oriented task execution capability can autonomously plan and execute a series of tasks and actions according to a given goal to achieve the goal. It can decompose a complex goal into multiple subtasks and gradually complete these subtasks in a certain order and logic. The environmental perception and interaction capability can perceive the surrounding environment and information, and make corresponding decisions and actions based on this information. The learning and adaptive capability can continuously accumulate experience and knowledge based on the interaction with the environment and the execution of tasks, and adjust its own behavior strategies and decision-making methods to improve the efficiency and quality of task execution. For example, after performing similar tasks many times, AGENT can summarize better solutions and methods, and can complete the task more quickly and accurately the next time it is executed. The tool calling and collaboration capability can call external tools and resources, such as databases, search engines, and software applications, to obtain more information and support to complete complex tasks.

[0041] For large-model workflow technology, it has process automation capabilities, task coordination and management capabilities, intelligent decision-making support capabilities, and flexible customization and adaptation capabilities.

[0042] Cloud terminals can use virtualization technology to run terminal operating systems, applications and other resources in the cloud. Users can use cloud terminals by using physical devices (such as mobile phones or tablets, etc., hereinafter referred to as user terminals) to access the Internet. The cloud terminal in the disclosed embodiment can specifically be a cloud phone. The cloud terminal runs in a private cloud, and the data is stored in isolation in the private cloud, which has high security. The cloud terminal also has strong compatibility, can run various native applications, and can support automatic control by the cloud terminal platform service, which can achieve non-stop service capabilities.

[0043] The cloud terminal platform service can be understood as the business service corresponding to the cloud terminal, which can realize the control of the cloud terminal.

[0044] In the disclosed embodiment, at least one of the above capabilities of the big model can be used in combination with the control of the cloud terminal by the cloud terminal platform service to realize the automatic execution of user tasks.

[0045] Optionally, the intelligent agent service and the cloud terminal platform service can be deployed in the same electronic device or in different electronic devices, without any specific limitation. The electronic device can be a server or a server cluster.

[0046] In the disclosed embodiment, the user can use natural language (such as text or voice, etc., not specifically limited) to input user instructions for the target client in the user terminal, and the user terminal forwards the user instructions to the intelligent agent service, or pre-processes the original user instructions (such as converting the user instructions in voice form into user instructions in text form), and then sends the pre-processed user instructions to the intelligent agent service. The target client can be understood as the client on which the automated user task that the user expects to achieve depends, and the client can be an application (Appication, APP) client or a web client, etc., not specifically limited. Optionally, the target client can be a client based on a graphical user interface (Graphical User Interface, GUI). For example, the user expects to automatically complete the review of the restaurant that has been consumed in a certain life service client. The user instruction can be, for example, "give a five-star review to the A store last night in XX takeaway", where "XX takeaway" is a life service client, and the "XX takeaway" is the target client. The intelligent agent service can use the above-mentioned capabilities of the large language model to understand the user instructions based on natural language and obtain the task information of the target task corresponding to the user instruction. Among them, the target task can be understood as a user task that the user wants to complete automatically by inputting user instructions. For example, in the above example, in XX takeaway, giving a five-star review to store A last night, the target task can be to automatically post a five-star review in XX takeaway for the order that the user consumed at store A last night. The task information can be the relevant information describing the target task generated by the intelligent service through understanding the user instructions.

[0047] In the disclosed embodiment, corresponding control software can be pre-written for different types of tasks, which is recorded as preset control software. The software identifier of each preset control software can be stored in the preset knowledge base, and the software identifier can be, for example, a tag. The intelligent agent service can automatically find the preset control software (recorded as the target control software) that matches the task information according to the task information, and obtain the software identifier of the target control software stored in the preset knowledge base (recorded as the target software identifier), and then send the target software identifier to the cloud terminal platform service. After receiving the target software identifier, the cloud terminal platform service can determine the target control software to be executed according to the target software identifier, and obtain or call the target control software to execute the target task in the cloud terminal corresponding to the user terminal by executing the target software. The cloud terminal corresponding to the user terminal is loaded with a target client. Since the cloud terminal runs in a private cloud, the user account can be safely used to log in to the target client to complete the target task.

[0048] The task processing method provided by the embodiment of the present disclosure understands the user instructions sent by the user terminal to the target client through the intelligent agent service realized based on the large language model, obtains the task information of the target task corresponding to the user instruction, obtains the target software identification of the target control software matching the task information from the preset knowledge base, and sends the target software identification to the cloud terminal platform service, and executes the target control software according to the target software identification through the cloud terminal platform service to execute the target task in the cloud terminal corresponding to the user terminal and loaded with the target client. By adopting the above technical scheme, the ability of the large language model is used to understand the user instructions and match the identification of the control software to be executed, and the cloud terminal platform service executes the corresponding control software according to the identification to automatically complete the user task in the cloud terminal, and the large model and the cloud terminal are combined. Based on the control software, the user program in the cloud terminal can be accurately controlled, and the private cloud security of the cloud terminal is relied on to ensure the privacy of the user, and it can run uninterruptedly without obtaining the authority of the user's physical terminal. According to the pre-written control software, the user program in the cloud phone is controlled, rather than the large model simulating the user operation according to the user instruction, the operation accuracy and stability can be guaranteed, and the task execution efficiency can be improved.

[0049] Figure 2 It is a flowchart of another task processing method provided according to an embodiment of the present disclosure. This embodiment is optimized based on the optional solutions in the above embodiments, and optimizes the steps of obtaining the target software identifier and executing the target control software according to the target software identifier. Among them, the preset knowledge base includes a preset mapping relationship, and the preset mapping relationship includes the corresponding relationship between the software function description information of the preset control software in the preset software warehouse and the software identifier of the preset control software. Figure 3 is a schematic diagram of a task processing solution provided according to an embodiment of the present disclosure, which can be combined with Figure 3 Understand the embodiments of the present disclosure.

[0050] like Figure 2 As shown, the method may include:

[0051] S201. Understanding the user instruction sent by the user terminal to the target client through the intelligent agent service, and obtaining the task information of the target task corresponding to the user instruction.

[0052] like Figure 3 As shown, the user can send user instructions to the agent service (large model agent) through the user terminal.

[0053] S202: Determine, through the agent service, the similarity between the task information and the software function description information in the preset mapping relationship in the preset knowledge base.

[0054] For example, Figure 3 As shown, the developer can develop the preset control software in advance. After the development is completed, the developer can add the corresponding software function description according to the functions that the preset control software can realize, and submit it (such as through the cloud terminal platform service) to the preset knowledge base. The preset knowledge base can directly store the submitted software function description or store it after preset processing to obtain software function description information, and establish a corresponding relationship between the software function description information and the software identifier of the corresponding preset control software to obtain a preset mapping relationship. Among them, the preset processing can, for example, be format conversion, so that the software function descriptions submitted by different developers are stored in a unified format in the preset knowledge base, which is convenient for matching with task information.

[0055] Optionally, both the task information and the software function description information may be represented in vector form, such as a task information vector and a software function description information vector, and the similarity between the task information and the software function description information may be determined by calculating the vector similarity.

[0056] S203. Determine, through the agent service, the software identifier corresponding to the software function description information whose similarity meets the preset similarity requirement as the target software identifier of the target control software matching the task information.

[0057] Exemplarily, the preset similarity requirement may be, for example, that the similarity is greater than a preset threshold, which can improve search efficiency; the preset similarity requirement may be, for example, that the similarity is the highest, which can improve search accuracy and success rate.

[0058] For example, S202 and S203 can be implemented using the RAG technology of the large model to improve the search efficiency and accuracy of the target software identification of the target control software. Figure 3 As shown, the intelligent agent service can obtain the target software identification from the preset knowledge base through RAG technology.

[0059] S204. Send the target software identifier to the cloud terminal platform service through the intelligent agent service.

[0060] like Figure 3 As shown, the intelligent agent service will obtain the target software identifier from the preset knowledge base and send it to the cloud terminal platform service.

[0061] S205: Obtain target control software from a preset software warehouse according to the target software identifier through the cloud terminal platform service, and execute the target control software to execute the target task in the cloud terminal corresponding to the user terminal.

[0062] like Figure 3As shown, after obtaining the target software identifier, the cloud terminal platform service obtains the corresponding target control software from the preset software warehouse, and interacts with the cloud terminal in the private cloud by executing the target control software to execute the target task in the cloud terminal.

[0063] The task processing method provided by the embodiment of the present disclosure stores in advance in the preset software warehouse the correspondence between the software function description information and the software identifier of the preset control software in the preset software warehouse, and after obtaining the task information corresponding to the user instruction by using the intelligent body service, determines the identifier of the control software to be executed by the similarity between the task information and the software function description information, and obtains the corresponding control software from the preset software warehouse according to the identifier and executes it by the cloud terminal platform service, thereby automatically completing the user task in the cloud terminal, and effectively utilizes the ability of the large model to retrieve external knowledge to efficiently and accurately determine the target control software for completing the current target task from a large number of pre-written preset control software, and combines with the cloud terminal to accurately control the user program in the cloud terminal based on the control software, which can further improve the accuracy, efficiency and effect of task execution, thereby improving the user experience.

[0064] In an optional embodiment, the cloud terminal platform service establishes a long connection based on a command line tool with the cloud terminal; wherein, the target control software is executed according to the target software identifier by the cloud terminal platform service to execute the target task in the cloud terminal corresponding to the user terminal, including: the cloud terminal platform service determines the target control software according to the target software identifier, and sends the control instructions based on the command line tool in the target control software to the cloud terminal corresponding to the user terminal based on the long connection, so that the cloud terminal executes the target task by responding to the control instructions. The advantage of such a setting is that by establishing a long connection with the cloud terminal, the cloud terminal platform service does not need to establish a connection with the cloud terminal for each client separately, which reduces overhead, improves interaction efficiency, and controls the cloud terminal through control instructions based on command line tools, thereby improving control accuracy.

[0065] Exemplarily, the command line tool may be, for example, Android Debug Bridge (ADB). ADB-based control instructions may be written in the preset control software, and during the execution of the target control software, the cloud terminal platform service sends ADB-based control instructions to the cloud terminal based on a long connection to complete the execution control of the target task.

[0066] In an optional embodiment, the task information includes subtask information of one or more subtasks of the target execution order of the target task; wherein, obtaining the target software identifier of the target control software matching the task information from the preset knowledge base through the intelligent agent service, and sending the target software identifier to the cloud terminal platform service, includes: obtaining the target software identifiers of the target control software matching the subtask information respectively from the preset knowledge base through the intelligent agent service, wherein the number of the target software identifiers matches the number of the subtask information; sending the target software identifier and the target execution order to the cloud terminal platform service through the intelligent agent service; wherein, executing the target control software according to the target software identifier through the cloud terminal platform service includes: executing the target control software according to the target software identifier in accordance with the target execution order through the cloud terminal platform service. The advantage of this setting is that the AGENT technology of the large model can be used to autonomously decompose the target task into multiple sub-tasks and determine the execution order of multiple sub-tasks, so that the preset control software can achieve more fine-grained functions, and use the intelligent agent service to automatically assemble multiple preset control software to complete user tasks in an anthropomorphic way. Compared with the method of specifically writing corresponding control software for a complete user task, it can make the execution of user tasks more flexible and improve the task execution effect.

[0067] For example, as in the example above, the target task is to automatically post a five-star review for the order that the user consumed at store A last night in XX takeaway. The split subtasks may include, for example: opening the XX takeaway client, finding the order that the user consumed at store A last night, opening the order review page, entering a five-star review, and posting the review. The intelligent agent service can determine the target software identifier of the corresponding target control software for the task information of each subtask, and send it to the cloud terminal platform service, so that the cloud terminal platform service can execute the target control software corresponding to each subtask in turn, thereby completing the target task.

[0068] In an optional implementation, the preset software warehouse includes at least one preset control software applicable to multiple clients. The advantage of such a setting is that, on the basis of developing preset control software for fine-grained functions, the same preset control software can be reused in different types of clients, thereby reducing development costs and improving development efficiency.

[0069] For example, as in the above example, opening the XX takeaway client is actually starting a subtask of a certain client. For different clients, the startup methods in the cloud terminal are usually the same or similar, so they can be reused; for another example, after opening the client, some clients will display a splash screen advertisement. If the next step needs to be executed, there may be a subtask of closing the splash screen advertisement, and the actual execution logic is usually to identify the advertisement logo in the upper right corner of the interface and click the close button, so it can be reused.

[0070] Figure 4 1 is a flowchart of another task processing method provided according to an embodiment of the present disclosure. This embodiment is optimized based on the optional solutions in the above embodiments, and further optimizes the steps of obtaining the target software identifier and executing the target control software according to the target software identifier. Figure 4 As shown, the method may include:

[0071] S401. Understanding the user instructions sent by the user terminal to the target client through the intelligent agent service, and obtaining task information of the target task corresponding to the user instruction, wherein the task information includes subtask information of one or more subtasks in the target execution order of the target task.

[0072] S402. For each subtask information, the similarity between the current subtask information and the software function description information in the preset mapping relationship in the preset knowledge base is determined by the intelligent agent service, and the software identifier corresponding to the software function description information whose similarity meets the preset similarity requirement is determined as the target software identifier of the target control software that matches the current subtask information.

[0073] For example, in the case where there are multiple subtask information, the agent service determines the similarity between each subtask information and the software function description information in the preset mapping relationship in the preset knowledge base, and then determines the target software identifier of the target control software that needs to be executed to complete the corresponding subtask. The number of target software identifiers can be equal to the number of subtask information.

[0074] S403, sending the target software identifier and the target execution sequence to the cloud terminal platform service through the intelligent agent service.

[0075] S404, obtaining the target control software corresponding to each subtask from the preset software warehouse according to the target software identifier through the cloud terminal platform service, and executing the target control software according to the target execution order, so as to execute each subtask in the target task in sequence in the cloud terminal corresponding to the user terminal.

[0076] Exemplarily, a long connection based on ADB is established between the cloud terminal platform service and the cloud terminal. According to the target execution order, the ADB-based control instructions in the current target control software are sent to the cloud terminal corresponding to the user terminal based on the long connection, so that the cloud terminal executes each subtask in the target task in sequence by responding to the control instructions.

[0077] The task processing method provided by the embodiment of the present disclosure divides the target task corresponding to the user instruction into multiple sub-tasks to be executed sequentially through the intelligent body service, and for each sub-task information, determines the similarity between the current sub-task information and the software function description information in the preset mapping relationship in the preset knowledge base, and determines the software identifier corresponding to the software function description information whose similarity meets the preset similarity requirement as the target software identifier of the target control software matching the current sub-task information, and sends it to the cloud terminal platform service for sequential execution of each target control software, so that the intelligent body service can automatically assemble multiple preset control software to complete the user task in an anthropomorphic manner, improve the flexibility and accuracy of user task execution, and further improve the task execution effect.

[0078] In an optional embodiment, the target client is a client based on a graphical user interface; wherein the method further comprises: obtaining a screenshot stream of the user interface in the cloud terminal through the cloud terminal platform service; performing a similarity comparison between the screenshots in the screenshot stream and the preset images corresponding to the target control software through the cloud terminal platform service, and determining the execution status of the target task based on the comparison result, wherein at least one execution status is used to feedback to the user terminal. The advantage of such a setting is that, for a GUI-based client, by performing a similarity comparison between the screenshot stream of the user interface and the preset image, the task execution status can be accurately determined, thereby improving the accuracy and efficiency of task execution.

[0079] Exemplarily, the cloud terminal platform service can obtain the screenshot stream in the cloud terminal based on a long connection, and maintain a circular buffer, retaining the most recent screenshots of a preset duration (such as 5 minutes), the screenshot frequency can be greater than or equal to the preset frequency (such as 1 frame per second), and the screenshots can be I-frame key frames or intra-frame coded frames, etc. The preset image can be a standard interface image corresponding to various task execution states preset by the developer in the preset control software. For example, for the task of closing the splash screen advertisement, the preset image can be the first interface image after closing the splash screen advertisement. For another example, for the task of publishing an evaluation, the preset image can be the interface image of the evaluation publishing success page, etc.

[0080] Optionally, the similarity comparison for the screenshots may include a similarity comparison based on a single image, and may also include a similarity comparison based on an image sequence (which may also be understood as a video). For example, for a single image, the similarity between the screenshot and the preset image may be calculated based on the OpenCV template matching algorithm, etc. Before the similarity comparison is performed, it is determined whether the two images to be compared correspond to the same screen resolution. If they are different, adjustments are made, such as adjusting to the same image size. Optionally, a rectangular selection box may also be set (such as a rectangular selection box defined by the abscissa value, ordinate value, width value, and height value with the vertex in the upper left corner of the image as the origin of the coordinate system), and a portion of the image content is respectively intercepted from the screenshot and the preset image according to the rectangular selection box for similarity comparison, and a similarity value such as between 0 and 1 is obtained (recorded as a single image similarity). For an image sequence, the image sequence similarity may be calculated based on the single image similarity by applying the dynamic time warping (DTW) algorithm, and the comparison range is the number of image frames corresponding to the preset image sequence (a plurality of continuous preset images pre-set in the preset control software), and the image sequence similarity (such as between 0 and 1) is returned. Using image sequence similarity to determine the task status can improve the accuracy of task status determination compared to using single image similarity to determine the task status.

[0081] Among them, at least one execution state may include the final execution state of the target task, and may also include the execution state of each subtask, etc. At least one execution state may be sent from the cloud terminal platform service to the agent service, and fed back to the user terminal by the agent service.

[0082] The present disclosure also provides a task processing system. The system structure can be referred to as Figure 3 The system includes: an intelligent agent service, a cloud terminal platform service, a preset knowledge base and a cloud terminal; wherein the intelligent agent service is implemented based on a large language model, and is used to understand the user instruction sent by the user terminal to the target client, obtain the task information of the target task corresponding to the user instruction, and send the target software identifier of the target control software matched with the task information obtained from the preset knowledge base to the cloud terminal platform service; the cloud terminal platform service is used to execute the target control software according to the target software identifier sent by the intelligent agent service, so as to execute the target task in the cloud terminal; the cloud terminal is used to load the target client corresponding to the user terminal, and respond to the control of the cloud terminal platform service to execute the target task.

[0083] For example, the relevant concepts and working process of the task processing system can be referred to the relevant introduction in the task processing method above, which will not be repeated here.

[0084] The task processing system provided by the disclosed embodiment utilizes the capability of a large language model to understand user instructions and match them to the identification of the control software that needs to be executed. The cloud terminal platform service executes the corresponding control software according to the identification to automatically complete the user task in the cloud terminal. The large model and the cloud terminal are combined. The user program in the cloud terminal can be accurately controlled based on the control software. The private cloud security of the cloud terminal is relied upon to ensure user privacy security. The system can run uninterruptedly without obtaining the authority of the user's physical terminal. The user program in the cloud phone is controlled according to the pre-written control software, rather than the large model simulating the user operation according to the user instruction. This ensures the accuracy and stability of the operation and improves the efficiency of task execution.

[0085] In an optional implementation, the cloud terminal platform service includes a first cloud terminal platform service deployed in a first computer room and a second cloud terminal platform service deployed in a second computer room; the first computer room is a centralized computer room, and the second computer room is a computer room where the cloud terminal is deployed; the first cloud terminal platform service is used to interact with the intelligent service; the first cloud terminal platform service and the second cloud terminal platform service interact based on a public network; the second cloud terminal platform service is used to interact with the cloud terminal based on an intranet. The advantage of such a setting is that the cloud terminal platform services are reasonably split and deployed in different computer rooms respectively. Platform services with relatively low performance requirements but requirements for centralized storage can be deployed in a centralized computer room, while platform services with high performance and network environment requirements can be deployed in the same computer room as the cloud terminal. Interaction is carried out through the intranet, which effectively improves the interaction efficiency and reliability on the basis of giving full play to the advantages of different entity resources.

[0086] In an optional embodiment, the first cloud terminal platform service includes a task manager, a development platform and a first preset software warehouse; the second cloud terminal platform service includes a basic service, a framework layer, an executor, a scheduler and a second control software warehouse; wherein the task manager is used to call the scheduler based on the received target software identifier; the development platform is used to provide development tools to the developer of the preset control software, and publish the preset control software developed by the developer based on the development tools to the first control software warehouse, wherein the preset control software in the first control software warehouse is synchronized to the second control software warehouse through the code warehouse management software; the scheduler is used to create a running container of the target control software in response to the call of the task manager, and deploy the target control software obtained from the second control software warehouse to the running container; the executor is used to provide the running environment of the running container and execute the target control software; the framework layer is used to provide the framework code corresponding to the development tool; the basic service is used to interact with the cloud terminal. By adopting the above technical solution, the first cloud terminal platform service and the second cloud terminal platform service in the task system are reasonably configured, so that the cloud terminal platform service can better cooperate with the intelligent body service and the cloud terminal to efficiently execute user tasks, and by providing development tools and framework codes to developers, the development efficiency of the preset control software is improved, thereby improving the overall operating efficiency and timeliness of the task processing system, timely meeting the user's diverse and constantly updated user task automatic execution needs, and can agilely adapt to the client's update iterations.

[0087] Figure 5 is a structural diagram of a task processing system provided according to an embodiment of the present disclosure, such as Figure 5 As shown, the system includes an intelligent agent service, a first cloud terminal platform service, a second cloud terminal platform service and a cloud terminal. The first cloud terminal platform service is deployed in the first computer room, the intelligent agent service can also be deployed in the first computer room, and the second cloud terminal platform service and the cloud terminal are deployed in the second computer room. The terminal user can interact with the intelligent agent service through the user interface (UI) layer. The user interface layer includes a question-and-answer front end and a task execution front end. The question-and-answer front end can be a general large model dialogue front end. The task execution front end is used to initiate cloud tasks based on question-and-answer in order to complete GUI tasks, such as completing product likes.

[0088] The task manager (TaskManager) in the first cloud terminal platform service may include a task add, delete, modify and check (CURD) module, a task status management module and a task log management module. The task CURD module is used to implement the creation, update, query and deletion of tasks. The task status management module is used to maintain the basic state machine of task execution and can support the developer to customize the sub-state. The task log management module is used to provide the executor log return and viewing capabilities. The development platform (Platform) may include an editor (Editor), a cloud terminal management module (ARM Manager) and a knowledge base publishing module, which can provide a visual platform tool in the form of a plug-in based on vs-code, etc., for preset control software development, cloud machine management and code publishing. Among them, the editor can provide intelligent prompts, annotation generation code capabilities and code completion capabilities based on the vs-code plug-in; the cloud terminal management module can connect to the cloud terminal account system to realize the management of the cloud terminal, such as device list, screenshot, push stream and cloud terminal control functions; the knowledge base publishing module is used to provide functional descriptions based on the functions of the preset control software, and submit the large model RAG, that is, publish it to the preset knowledge base. The developer develops the preset control software through the development platform and publishes it to the first control software warehouse. The preset control software in the first control software warehouse is synchronized to the second control software warehouse in the second cloud terminal platform service through the code warehouse management software.

[0089] The dispatcher in the second cloud terminal platform service is used to create containers, pull tags, and build images. The container allocator receives task meta information from the task manager, completes container allocation and task detail environment variable writing, and encapsulates the container operation application programming interface (Application Programming Interface, API). The tag pull module pulls the target control software code from the second control software warehouse according to the input tag and deploys it to the container. The executor provides the control software operating environment, completes application initialization, execution, and resource cleanup in the form of docker, can achieve network isolation, and provide a safe operating environment. The framework layer (Framework) can provide a multi-language (for example, including Python) framework and basic code for developers to integrate and call. It can run in the executor container and provide at least one of the following APIs: task configuration (config_initd), used to read task configuration information from the executor environment variables; resource initialization (resource_initd), used to complete resource initialization based on configuration information, including ADB links, screenshots, and screenshot streams; assertions (assert), used to feedback the execution process and results of the preset control software; notifications (notify), used for external request interaction, which can realize network isolation restrictions; logging (logger), used to record targeted logs and collect them to the task manager in a unified manner. Figure 5 The dynamic link library in can also belong to the framework layer, which is used to encapsulate high-performance screenshot comparison (ScreenCompare) algorithm module and screenshot stream comparison (StreamCompare) algorithm module, and provide resource initialization, reading and command sending and receiving capabilities. The resource manager can be called when the preset control software performs initialization, and is used to obtain ADB links and screenshot streams, and complete the slicing of sample videos. The command line tool terminal is used to provide ADB command calling API. Basic services are used to encapsulate the ARM cloud machine foundation and provide APIs in the form of services, including screenshots and ADB link pools, etc. Among them, the screenshot stream storage can provide real-time screenshot streams in the form of services based on sockets (such as unix-sockets). The command line tool link pool can establish a stable ADB long connection with the cloud machine in the same computer room (including virtual machines, running the operating system and client in the cloud terminal through virtual machines), and can encapsulate the ADB protocol to provide ADB connection capabilities, proxy ADB, and provide REST-API to synchronously execute ADB commands.

[0090] In an optional implementation, the development tool includes a large language model; the development platform is used to generate a recommended code corresponding to the software development instructions input by the developer based on the natural language based on the framework code using the large language model in the development tool, and the recommended code is used for the developer to develop the preset control software. The advantage of this setting is that the recommended code can be automatically generated by using the capabilities of the large language model and the framework code, which assists the developer in conveniently developing the preset control software and improves development efficiency.

[0091] It should be noted that the large language model in the development tool is different from the large language model in the agent service. During the development of the preset control software by the developer, the natural language-based software development instructions can be input through the development tool. The large language model in the development tool can understand the software development instructions and obtain the development requirement information. According to the development requirement information, the RAG technology is used to obtain the matching target code from the framework code (including the code in the dynamic link library), and the target code is automatically assembled to generate the recommended code, and the recommended code is fed back to the developer through the development tool. The developer can directly use the recommended code or modify the recommended code to obtain the preset control software corresponding to the software development instruction, effectively improving the convenience and efficiency of the preset control software development.

[0092] Figure 6 The structure diagram of a task processing device provided according to an embodiment of the present disclosure is applicable to automatically executing the corresponding task of the client according to the user's natural language instructions. The device can be implemented in hardware and / or software and can be configured in an electronic device. Figure 6 , the task processing device 600 includes:

[0093] The task information determination module 601 is used to understand the user instruction sent by the user terminal to the target client through the intelligent agent service, and obtain the task information of the target task corresponding to the user instruction, wherein the intelligent agent service is implemented based on the large language model;

[0094] A software identification sending module 602 is used to obtain a target software identification of a target control software matching the task information from a preset knowledge base through the agent service, and send the target software identification to a cloud terminal platform service;

[0095] The task processing module 603 is used to execute the target control software according to the target software identifier through the cloud terminal platform service to execute the target task in the cloud terminal corresponding to the user terminal, wherein the cloud terminal is loaded with the target client.

[0096] Optionally, the task information determination module and the software identification sending module can be integrated into a physical device used to deploy intelligent agent services, and the task processing module can be integrated into a physical device used to deploy cloud terminal platform services.

[0097] The task processing device provided by the embodiment of the present disclosure utilizes the capability of the large language model to understand user instructions and match them to the identification of the control software that needs to be executed. The cloud terminal platform service executes the corresponding control software according to the identification to automatically complete the user task in the cloud terminal. The large model and the cloud terminal are combined. The user program in the cloud terminal can be accurately controlled based on the control software. The private cloud security of the cloud terminal is relied upon to ensure the privacy of the user. The device can run uninterruptedly without obtaining the authority of the user's physical terminal. The user program in the cloud phone is controlled according to the pre-written control software, rather than the large model simulating the user operation according to the user instruction. This can ensure the accuracy and stability of the operation and improve the efficiency of task execution.

[0098] In an optional implementation, the preset knowledge base includes a preset mapping relationship, and the preset mapping relationship includes a correspondence between software function description information of the preset control software in the preset software warehouse and a software identifier of the preset control software;

[0099] Wherein, the software identification sending module includes a similarity determination unit, a software identification determination unit and a software identification sending unit;

[0100] The similarity determination unit is used to determine the similarity between the task information and the software function description information in the preset mapping relationship in the preset knowledge base through the agent service;

[0101] A software identification determination unit, configured to determine, through the agent service, the software identification corresponding to the software function description information whose similarity meets the preset similarity requirement as the target software identification of the target control software matching the task information;

[0102] A software identification sending unit, used to send the target software identification to the cloud terminal platform service through the agent service;

[0103] Wherein, the task processing module is used to obtain the target control software from the preset software warehouse according to the target software identifier through the cloud terminal platform service, and execute the target control software.

[0104] In an optional implementation, the task information includes subtask information of one or more subtasks in the target execution order of the target task;

[0105] Wherein, the software identification sending module includes:

[0106] an identification determination unit, configured to obtain, from a preset knowledge base through the agent service, target software identifications of target control software that respectively match the subtask information, wherein the number of the target software identifications matches the number of the subtask information;

[0107] A sending unit, used to send the target software identifier and the target execution sequence to the cloud terminal platform service through the agent service;

[0108] Wherein, the task processing module is used to execute the target control software according to the target software identifier and the target execution order through the cloud terminal platform service.

[0109] In an optional implementation, the preset software warehouse includes at least one preset control software applicable to multiple clients.

[0110] In an optional implementation manner, the cloud terminal platform service establishes a long connection based on a command line tool with the cloud terminal;

[0111] Among them, the task processing module is used for the cloud terminal platform service to determine the target control software according to the target software identifier, and based on the long connection, send the control instructions based on the command line tool in the target control software to the cloud terminal corresponding to the user terminal, so that the cloud terminal executes the target task by responding to the control instructions.

[0112] In an optional implementation, the target client is a client based on a graphical user interface;

[0113] Wherein, the device further comprises:

[0114] A screenshot stream acquisition module, used to acquire a screenshot stream of the user interface in the cloud terminal through the cloud terminal platform service;

[0115] An execution status determination module is used to compare the screenshots in the screenshot stream with the preset images corresponding to the target control software through the cloud terminal platform service, and determine the execution status of the target task based on the comparison results, wherein at least one execution status is used to feedback to the user terminal.

[0116] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0117] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0118] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0119] like Figure 7 As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0120] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0121] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as the task processing method. For example, in some embodiments, the task processing method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the task processing method described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the task processing method in any other appropriate manner (e.g., by means of firmware).

[0122] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0123] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0124] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0125] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0126] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0127] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services. The server may also be a server of a distributed system, or a server combined with a blockchain.

[0128] Artificial intelligence is a discipline that studies how to use computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.). It includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, machine learning / deep learning technology, big data processing technology, knowledge graph technology, and other major directions.

[0129] Cloud computing refers to a technology system that uses network access to elastically scalable shared physical or virtual resource pools. Resources can include servers, operating systems, networks, software, applications, and storage devices, and can be deployed and managed on demand and in a self-service manner. Cloud computing technology can provide efficient and powerful data processing capabilities for technical applications such as artificial intelligence and blockchain, as well as model training.

[0130] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution provided by this disclosure can be achieved, and this document does not limit this.

[0131] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A task processing method, comprising: Understanding the user instruction sent by the user terminal to the target client through the intelligent agent service, and obtaining the task information of the target task corresponding to the user instruction, wherein the intelligent agent service is implemented based on the large language model; Acquire the target software identifier of the target control software matching the task information from a preset knowledge base through the agent service, and send the target software identifier to the cloud terminal platform service; The target control software is executed according to the target software identifier through the cloud terminal platform service to execute the target task in the cloud terminal corresponding to the user terminal, wherein the target client is loaded in the cloud terminal.

2. The method according to claim 1, wherein: The preset knowledge base includes a preset mapping relationship, and the preset mapping relationship includes a corresponding relationship between software function description information of the preset control software in the preset software warehouse and the software identifier of the preset control software; The step of obtaining the target software identifier of the target control software matching the task information from a preset knowledge base through the agent service includes: Determine, by the agent service, the similarity between the task information and the software function description information in the preset mapping relationship in the preset knowledge base; Determining, by the agent service, the software identifier corresponding to the software function description information whose similarity meets the preset similarity requirement as the target software identifier of the target control software matching the task information; The executing the target control software according to the target software identifier through the cloud terminal platform service includes: The target control software is obtained from the preset software warehouse according to the target software identifier through the cloud terminal platform service, and the target control software is executed.

3. The method according to claim 1, wherein: The task information includes subtask information of one or more subtasks in the target execution order of the target task; The step of obtaining a target software identifier of a target control software matching the task information from a preset knowledge base through the agent service and sending the target software identifier to the cloud terminal platform service includes: Acquire, through the agent service, target software identifiers of target control software respectively matching the subtask information from a preset knowledge base, wherein the number of the target software identifiers matches the number of the subtask information; Sending the target software identifier and the target execution sequence to the cloud terminal platform service through the agent service; The executing the target control software according to the target software identifier through the cloud terminal platform service includes: The target control software is executed according to the target software identifier and the target execution order through the cloud terminal platform service.

4. The method according to claim 3, wherein: The preset software warehouse includes at least one preset control software applicable to multiple clients.

5. According to the method of claim 1, the cloud terminal platform service establishes a long connection with the cloud terminal based on a command line tool; in, The executing the target control software according to the target software identifier through the cloud terminal platform service to execute the target task in the cloud terminal corresponding to the user terminal includes: The cloud terminal platform service determines the target control software according to the target software identifier, and sends the control instructions based on the command line tool in the target control software to the cloud terminal corresponding to the user terminal based on the long connection, so that the cloud terminal executes the target task by responding to the control instructions.

6. The method according to claim 1, wherein the target client is a client based on a graphical user interface; in, The method further comprises: Obtaining a screenshot stream of the user interface in the cloud terminal through the cloud terminal platform service; The screenshots in the screenshot stream are compared with preset images corresponding to the target control software for similarity through the cloud terminal platform service, and the execution status of the target task is determined according to the comparison result, wherein at least one execution status is used to feed back to the user terminal.

7. A task processing system, comprising: Intelligent agent services, cloud terminal platform services, preset knowledge base and cloud terminal; The intelligent agent service is implemented based on a large language model, and is used to understand the user instructions sent by the user terminal to the target client, obtain the task information of the target task corresponding to the user instruction, and send the target software identifier of the target control software matching the task information obtained from the preset knowledge base to the cloud terminal platform service; The cloud terminal platform service is used to execute the target control software according to the target software identifier sent by the agent service, so as to execute the target task in the cloud terminal; The cloud terminal is used to load the target client corresponding to the user terminal and respond to the control of the cloud terminal platform service to execute the target task.

8. The system according to claim 7, wherein: The cloud terminal platform service includes a first cloud terminal platform service deployed in a first computer room and a second cloud terminal platform service deployed in a second computer room; the first computer room is a centralized computer room, and the second computer room is a computer room where the cloud terminal is deployed; The first cloud terminal platform service is used to interact with the agent service; The first cloud terminal platform service and the second cloud terminal platform service interact based on a public network; The second cloud terminal platform service is used to interact with the cloud terminal based on an intranet.

9. The system according to claim 8, wherein: The first cloud terminal platform service includes a task manager, a development platform and a first preset software warehouse; the second cloud terminal platform service includes a basic service, a framework layer, an executor, a scheduler and a second control software warehouse; Wherein, the task manager is used to call the scheduler based on receiving the target software identifier; The development platform is used to provide a development tool to a developer of the preset control software, and publish the preset control software developed by the developer based on the development tool to the first control software warehouse, wherein the preset control software in the first control software warehouse is synchronized to the second control software warehouse through the code warehouse management software; The scheduler is used to create a running container of the target control software in response to a call of the task manager, and deploy the target control software obtained from the second control software warehouse into the running container; The executor is used to provide an operating environment for the operating container and execute the target control software; The framework layer is used to provide framework codes corresponding to the development tool; The basic service is used to interact with the cloud terminal.

10. The system according to claim 9, wherein: The development tool includes a large language model; The development platform is used to generate recommended codes corresponding to the software development instructions input by the developer based on natural language based on the framework code using the large language model in the development tool, and the recommended codes are used by the developer to develop the preset control software.

11. A task processing device, comprising: A task information determination module, used to understand the user instruction sent by the user terminal to the target client through the intelligent agent service, and obtain the task information of the target task corresponding to the user instruction, wherein the intelligent agent service is implemented based on a large language model; A software identification sending module, used to obtain a target software identification of a target control software matching the task information from a preset knowledge base through the agent service, and send the target software identification to a cloud terminal platform service; The task processing module is used to execute the target control software according to the target software identifier through the cloud terminal platform service to execute the target task in the cloud terminal corresponding to the user terminal, wherein the cloud terminal is loaded with the target client.

12. The device according to claim 11, wherein The preset knowledge base includes a preset mapping relationship, and the preset mapping relationship includes a corresponding relationship between software function description information of the preset control software in the preset software warehouse and the software identifier of the preset control software; Wherein, the software identification sending module includes a similarity determination unit, a software identification determination unit and a software identification sending unit; The similarity determination unit is used to determine the similarity between the task information and the software function description information in the preset mapping relationship in the preset knowledge base through the agent service; A software identification determination unit, configured to determine, through the agent service, the software identification corresponding to the software function description information whose similarity meets the preset similarity requirement as the target software identification of the target control software matching the task information; A software identification sending unit, used to send the target software identification to the cloud terminal platform service through the agent service; Wherein, the task processing module is used to obtain the target control software from the preset software warehouse according to the target software identifier through the cloud terminal platform service, and execute the target control software.

13. The device according to claim 11, wherein: The task information includes subtask information of one or more subtasks in the target execution order of the target task; Wherein, the software identification sending module includes: an identification determination unit, configured to obtain, from a preset knowledge base through the agent service, target software identifications of target control software that respectively match the subtask information, wherein the number of the target software identifications matches the number of the subtask information; A sending unit, used to send the target software identifier and the target execution sequence to the cloud terminal platform service through the agent service; Wherein, the task processing module is used to execute the target control software according to the target software identifier and the target execution order through the cloud terminal platform service.

14. The device according to claim 13, wherein: The preset software warehouse includes at least one preset control software applicable to multiple clients.

15. The device according to claim 11, wherein the cloud terminal platform service establishes a long connection with the cloud terminal based on a command line tool; in, The task processing module is used for the cloud terminal platform service to determine the target control software according to the target software identifier, and based on the long connection, send the control instructions based on the command line tool in the target control software to the cloud terminal corresponding to the user terminal, so that the cloud terminal executes the target task by responding to the control instructions.

16. The device according to claim 11, wherein the target client is a client based on a graphical user interface; in, The device also includes: A screenshot stream acquisition module, used to acquire a screenshot stream of the user interface in the cloud terminal through the cloud terminal platform service; An execution status determination module is used to compare the screenshots in the screenshot stream with the preset images corresponding to the target control software through the cloud terminal platform service, and determine the execution status of the target task based on the comparison results, wherein at least one execution status is used to feedback to the user terminal.

17. An electronic device, the electronic device being configured with an intelligent agent service and / or a cloud terminal platform service, the electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform corresponding steps in any one of the methods of claims 1-6.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-6.

19. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images