UI task scheduling method and device, storage medium and program product

Abstract

The embodiment of the invention provides a UI task scheduling method and device, a storage medium and a program product. In the embodiment of the invention, description information of a UI page is obtained, a UI task matched with the page is generated by utilizing an AI-based task generation model, and performance scores of nodes are dynamically calculated in combination with multi-dimensional resource use information of the calculation nodes; furthermore, according to the execution complexity of the UI task and the performance score of the computing node, the task is intelligently scheduled to the adaptive target computing node, each UI task is ensured to be executed on the optimal computing node, and the utilization rate of system resources is maximized, so that balanced allocation of resources is realized, and the execution efficiency and the dynamic adjustment capability of the UI task are improved.

Description

Technical Field

[0001] The present application relates to the field of automation technology, and in particular to a UI task scheduling method, device, storage medium and program product. Background Art

[0002] With the continuous advancement of information technology and the widespread popularity of Internet applications, automated UI (User Interface) tasks play an increasingly important role in fields such as software testing, data collection, and business process automation. From simple web page clicks to complex interactive operations, automated UI tasks are designed to simulate the interaction between human users and graphical interfaces, thereby achieving efficient task processing and optimizing workflows.

[0003] However, as the business continues to expand and the amount of tasks increases day by day, there are significant differences in the execution complexity and resource requirements of UI tasks in different scenarios. Some tasks may require a large amount of computing resources to complete complex interactive operations, while other tasks are relatively simple and have lower resource requirements. This difference leads to an imbalance in resource allocation: some computing nodes may approach or exceed their performance limits due to high-load tasks, while other nodes are idle or inefficiently running, failing to fully utilize their performance potential, further limiting the dynamic adjustment capabilities of UI task execution. Summary of the invention

[0004] Multiple aspects of the present application provide a UI task scheduling method, device, storage medium and program product to improve system resource utilization and dynamic adjustment capability of UI task execution.

[0005] An embodiment of the present application provides a UI task scheduling method, including: obtaining description information of at least one UI page, each UI page refers to a page that relies on at least one UI task to perform interactive operations on the UI page, and different UI tasks have different execution complexities; based on the description information of at least one UI page, calling an AI-based task generation model to generate UI tasks to obtain at least one UI task corresponding to each UI page; calculating a performance score of each computing node based on multi-dimensional resource usage information of each computing node in a computing cluster; for each scheduled UI task, selecting a target computing node that matches the execution complexity of the UI task from each computing node based on the performance score of each computing node, and scheduling the UI task to the target computing node for execution.

[0006] The embodiment of the present application also provides a computing platform, including: a processor and a memory, the memory is used to store a computer program, when the computer program is executed by the processor, the processor can implement each step of the UI task scheduling method provided in the embodiment of the present application.

[0007] The embodiment of the present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor is enabled to implement the various steps in the UI task scheduling method provided in the embodiment of the present application.

[0008] The embodiment of the present application also provides a computer program product, including a computer program / instruction. When the computer program / instruction is executed by a processor, the processor is enabled to implement the various steps in the UI task scheduling method provided in the embodiment of the present application.

[0009] In an embodiment of the present application, a UI task scheduling method is provided, which obtains the description information of the UI page, generates a UI task adapted to the page by using an AI-based task generation model, and dynamically calculates the performance score of the computing node in combination with the multi-dimensional resource usage information of the computing node; further, according to the execution complexity of the UI task and the performance score of the computing node, the task is intelligently scheduled to the adapted target computing node to ensure that each UI task can be executed on the optimal computing node, maximize the utilization of system resources, thereby achieving balanced resource allocation, improving the execution efficiency of UI tasks and dynamic adjustment capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0011] Figure 1 A flowchart of a UI task scheduling method provided for an exemplary embodiment of the present application;

[0012] Figure 2 A structural diagram of a UI task scheduling method provided by an exemplary embodiment of the present application;

[0013] Figure 3 A schematic diagram of a process of generating a UI task based on an AI-based task generation model provided by an exemplary embodiment of the present application;

[0014] Figure 4a A flowchart of a UI task scheduling method provided for an exemplary embodiment of the present application;

[0015] Figure 4b A schematic diagram of a structure for calculating the performance score of each computing node provided by an exemplary embodiment of the present application;

[0016] Figure 4c A schematic diagram of the structure of a thread pool provided for an exemplary embodiment of the present application;

[0017] Figure 5 A schematic diagram of the structure of an electronic device provided for an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0019] In view of the technical problem that in the prior art, computing nodes approach or exceed performance limits under high-load tasks, while other nodes are idle or inefficiently running, and cannot give full play to their performance potential, and at the same time, when the task volume surges or the task complexity increases, the existing system is difficult to respond quickly and dynamically adjust, further limiting the system's ability to cope with dynamic changes, in an embodiment of the present application, a UI task scheduling method is provided, which obtains description information of the UI page, generates UI tasks adapted to the page using an AI-based task generation model, and dynamically calculates the performance score of the node in combination with the multi-dimensional resource usage information of the computing node; further, according to the execution complexity of the UI task and the performance score of the computing node, the task is intelligently scheduled to the adapted target computing node to ensure that each UI task can be executed on the optimal computing node, thereby maximizing system resource utilization, thereby achieving balanced resource allocation, improving UI task execution efficiency and dynamic adjustment capabilities.

[0020] The following is combined with Figure 1 , describe in detail the technical solutions provided by each embodiment of the present application.

[0021] Figure 1 The following is a flow chart of a UI task scheduling method provided by an exemplary embodiment of the present application. Figure 1 As shown, the method includes:

[0022] 11. Obtain description information of at least one UI page;

[0023] 12. According to the description information of at least one UI page, call the AI-based task generation model to generate UI tasks to obtain at least one UI task corresponding to each UI page;

[0024] 13. Calculate the performance score of each computing node based on the multi-dimensional resource usage information of each computing node in the computing cluster;

[0025] 14. For each UI task that is scheduled, select a target computing node that matches the execution complexity of the UI task from among the computing nodes based on the performance scores of each computing node, and schedule the UI task to run on the target computing node.

[0026] In the embodiment of the present application, the device responsible for executing each step of the UI task scheduling method provided in the above embodiment can be implemented as an electronic device, and the UI task scheduling method can be implemented on a terminal device or a server device. The specific type of the electronic device is not limited in the embodiment of the present application. Among them, the terminal device refers to a computing device directly used by the user. The embodiment of the present application does not limit the specific form of the terminal device. For example, the terminal device can be a mobile device (such as a smart phone, a tablet computer), or a fixed device (such as a desktop computer, a workstation), or an embedded device (such as a smart TV, an in-vehicle information system); the server device refers to a high-performance device for providing computing resources and services. The embodiment of the present application does not limit the specific form of the server device. For example, the server device can be a physical server, a virtual server, or a distributed server cluster.

[0027] In the embodiments of the present application, a UI page refers to a graphical interface for users to interact with the system, such as a web page or an application interface. Each UI page relies on at least one UI task to perform interactive operations on the UI page, and each UI page contains multiple page elements, such as buttons, text boxes, menus, icons, etc., through which users can interact with the system. In automated tasks, the UI page is the basis for the execution of UI tasks. The scheduling of UI tasks depends on the description information of the UI page, and the execution complexity of different UI tasks is different.

[0028] In the embodiments of the present application, the execution complexity of a UI task refers to the computing resources, time and complexity of interactive operations required during the execution of the UI task. For example, low-complexity tasks require fewer resources and a shorter execution time, such as simple click operations or text input tasks; medium-complexity tasks require certain computing resources and time, such as form filling or data verification tasks; and high-complexity tasks require a large amount of computing resources and a longer execution time, such as large-scale data processing or complex logic verification tasks.

[0029] In the embodiment of the present application, the information included in the description information of the UI page is not limited. The description information of the UI page is a detailed description of the structure and content of the UI page, including the page element hierarchy information and / or ID, type, location, interaction attributes, etc. of the UI page elements. For example, for the login page, the description information of the UI page includes the hierarchical relationship and interaction logic of elements such as the user input box, password input box, and login button. The description information of the UI page is the basis for generating UI tasks. The UI task generation model will generate corresponding UI task steps based on the description information of the UI task, and the UI task can be specifically executed through the computing node.

[0030] In the embodiment of the present application, the computing node is the basic component unit in the computing cluster, and is a computing unit used to perform UI tasks. The type and form of the computing node are not limited. For example, the computing node can be a physical server, a virtual machine, or a container instance. The type of computing node can be flexibly selected according to the needs of UI tasks and the system architecture. For example, in a cloud computing environment, a computing node can be an elastically scalable virtual machine instance. In a containerized deployment, a computing node can be the smallest deployment unit in a Kubernetes cluster. Among them, the embodiment of the present application does not limit the number of computing nodes. The number of computing nodes can be dynamically adjusted according to the needs of the computing cluster. For example, there can be 3 computing nodes to handle relatively simple UI tasks; there can also be dozens or even hundreds of computing nodes to support daily operations and data analysis.

[0031] In the embodiment of the present application, the multi-dimensional resource usage refers to the usage of various resources of the computing node during operation, including the usage of CPU, memory, IO and thread pool. The performance score of each computing node can be calculated based on the multi-dimensional resource usage information of each computing node in the computing cluster. The role of the multi-dimensional resource usage information is to comprehensively evaluate the performance status of the computing node and provide data support for UI task scheduling. The performance score of each computing node reflects the current load status and remaining resource capacity of each computing node. The higher the performance score, the better the performance of the computing node and the ability to handle more complex tasks.

[0032] In the embodiment of the present application, in order to ensure that the UI task can be executed under the best conditions, the system makes an adaptive selection based on the execution complexity of the UI task and the performance score of each computing node. For example, for a high-complexity task, the computing node with the highest performance score can be selected to ensure efficient execution; while for a low-complexity task, it is assigned to a node with a slightly lower performance score but sufficient to complete the task, which not only ensures the smooth execution of the task, but also promotes the effective use of resources.

[0033] In the embodiment of the present application, there is no limitation on the strategy for adapting according to the execution complexity of the UI task and the performance score of each computing node. For example, the adapted strategy may be a maximum performance score strategy, which selects the computing node with the highest performance score to ensure that the task can be executed on the optimal computing node; the adapted strategy may also be a performance score threshold strategy, which selects computing nodes with performance scores exceeding a preset threshold to ensure that the task can be executed on nodes that meet the performance requirements. The embodiment of the present application does not limit the threshold of the performance score threshold strategy, for example, the threshold is set to 80 points, and nodes with scores exceeding 80 points are selected as target computing nodes; the threshold may also be set to 90 points, and nodes with scores exceeding 90 points are selected as target computing nodes.

[0034] In an embodiment of the present application, the triggering methods of UI tasks include manual triggering by the user and automatic triggering by a browser plug-in. These two triggering methods are respectively applicable to different scenarios to ensure that the UI task can be executed at the appropriate time. Among them, manual triggering by the user means that the user actively triggers the execution of the UI task according to actual needs. This method is suitable for scenarios where the user needs to clearly control the timing of task execution. For example, the user manually clicks the "Start Test" button in the test environment, and the system generates and executes the corresponding UI task according to the user's instructions. Automatic triggering by a browser plug-in means that the system automatically triggers the execution of the UI task according to specific events or conditions. This method is suitable for scenarios where tasks need to be automatically executed according to user behavior or system status. For example, after the user enters the user name and password on the login page, the system automatically detects the login event and triggers the corresponding UI task to verify the correctness of the login operation.

[0035] It should be noted that there are various forms of interactive operations with components on the UI page, and the functions or state changes in the target application can be triggered through interactive operations. The embodiments of the present application do not limit the specific forms of interactive operations. For example, the form of interactive operations can be that the user directly clicks (including single click, double click), slides (including up, down, left, right slides), long presses, zooms (zooms in, zooms out) and other operations on the component on the touch screen with a finger, or the user inputs text, numbers or other symbols through a physical keyboard or a virtual keyboard on the screen, or the user operates through physical buttons on the device, such as a power button, a volume button, a shutter button, etc.

[0036] Figure 2 The following is a schematic diagram of a UI task scheduling method provided by an exemplary embodiment of the present application. Figure 2As shown, the structure of the method includes an application layer 201, an API layer 202, a logic layer 203, and a resource layer 204. Each layer has its specific functions and components, working together to achieve efficient UI task execution. Among them, the application layer 201 is the front-end part of the system, responsible for interacting with users. The API layer 202 provides a series of interfaces for the front-end or other services to call, process task logic and data transmission. The logic layer 203 contains the core business logic and processing flow of the system, including the generation of UI tasks, task life cycle, task priority, task monitoring, automation tools, information acquisition (multi-dimensional resource usage information of each computing node in the computing cluster), and scheduling and scheduling logic of UI tasks. Among them, the task life cycle is responsible for managing the entire life cycle of UI tasks from generation to execution. The resource layer 204 manages the computing resources of the system, such as CPU, memory, storage, etc., provides the necessary computing resources for UI tasks, and ensures that UI tasks can be executed smoothly. As shown Figure 2 As shown, the resource layer 204 includes a container, a browser, and a driver. The container in the resource layer 204 is used to isolate and manage the execution environment of the UI task (i.e., the browser in the resource layer 204), providing a lightweight virtualization environment to ensure isolation and portability between tasks; the driver is used to control the browser or automation tool, interact with the browser or automation tool, and perform specific UI task operations. Among them, the API layer 202, the logic layer 203, and the resource layer 204 are all monitored by the monitoring module in real time for the running status and execution of the UI task, so that users can promptly discover and solve problems when calling UI tasks.

[0037] like Figure 2 As shown, the application layer 201 is responsible for UI tasks or user test scenarios submitted by users through manual triggering by users and / or automatically triggered by browser plug-ins. Among them, the application layer 201 includes user test scenarios and browser plug-ins. Among them, the user test scenario is the starting point of UI task scheduling, and the various interactive scenarios encountered by users when actually using the application. For example, after the user enters the user name and password on the login page and clicks the login button, the system automatically generates a UI task to simulate this operation. The browser plug-in is used to capture the description information of the user operation and the UI page in the browser, and pass it to the application layer 201 for processing. For example, the browser plug-in captures the user's click operation on the web page and sends the description information of the relevant UI page to the application layer 201.

[0038] like Figure 2As shown, the API layer 202 acts as an intermediate bridge, responsible for receiving requests from the application layer 201 and forwarding them to the logic layer for further processing, and is also responsible for feeding back the results returned by the logic layer 203 to the application layer 201. The API layer provides a series of interfaces for operations such as task generation, task scheduling, and task monitoring. In this process, "gateway distribution" distributes the request to the corresponding module of the logic layer for processing according to the content of the request, ensuring that the request can be accurately delivered to the corresponding business logic module. For example, the UI task generation API generates UI tasks based on the description information of the UI page, and the task scheduling API distributes the tasks to the computing nodes.

[0039] like Figure 2 As shown, at the logic layer 203, according to the description information of at least one UI page, the AI-based task generation model is called to generate UI tasks to obtain at least one UI task corresponding to each UI page, including: for each UI page, according to the description information of the UI page, a model prompt word is generated. In the embodiment of the present application, the specific type of the task generation model is not limited, that is, the task generation model refers to any model that can generate a corresponding output (such as a UI task) based on a given input (such as the description information of the UI page). For example, the task generation model can be a large language model that understands and generates natural language by learning a large amount of text data; it can also be a customized model specially designed for processing and generating specific types of tasks; it can also be a reinforcement learning model that interacts with the environment and gradually optimizes its generation strategy based on feedback.

[0040] In an embodiment of the present application, the model prompt word includes a first prompt word and a second prompt word, and the first prompt word is used to instruct the task generation model to obtain the page element level information of the UI page. Among them, one implementation of the first prompt word that can produce the above-mentioned effect is "page element level information". The page element level information of the UI page refers to the hierarchical relationship between the various elements (such as buttons, text boxes, pictures, etc.) on the UI page, which helps the model understand the page layout and the relative positions of the elements. For example, the UI page is a login page, and the first prompt word may include: User name input box: type is a text input box, located at the top of the page, level 1; Password input box: type is a password input box, located below the username input box, level 2; Login button: type is a button, located below the password input box, level 3.

[0041] In the above embodiment, after the task generation model obtains the page element hierarchy information of the UI page, the task generation model obtains the page element hierarchy information of the UI page according to the first prompt word, and generates the UI task according to the set task logic based on the page element hierarchy information according to the second prompt word. The second prompt word is used to instruct the task generation model to generate the UI task according to the set task logic based on the page element hierarchy information. The set task logic refers to the operation process or rules defined for different types of UI tasks, such as clicking a button, filling out a specific form, or navigating to another UI page. Different task logics correspond to different UI tasks, that is, even when facing the same page element hierarchy information, if different task logics are used, the UI tasks ultimately generated will be different.

[0042] Figure 3 A flowchart of a task generation model based on AI to generate UI tasks is provided for an exemplary embodiment of the present application. Figure 3 As shown, after obtaining the UI page in step ①, step ② generates a model prompt word according to the description information of the UI page extracted from the UI page, including: obtaining the page element level information of the UI page from the description information of the UI page, and using the page element level information as the first prompt word; obtaining the use guidance information of the automation tool and the set task logic, generating the second prompt word, based on the model prompt word generated in step ②, providing the corresponding API simulation set task logic through step ③, and then using the guidance information to guide the task generation model to call the corresponding API provided by the automation tool according to the page element level information to simulate the set task logic to generate the UI task through step ④. Among them, the model prompt word includes a first prompt word and a second prompt word, which are respectively used to guide the task generation model to obtain the page element level information and generate the UI task. The generation of the first prompt word is obtained by extracting the page element level information from the description information of the UI page. For the description of the page level information, please refer to the above embodiment, which will not be repeated here. The generation of the second prompt word includes the use guidance information of the automation tool and the set task logic. Among them, the usage guidance information contains detailed guidance on how the automation tool is called, including but not limited to the code address or identifier of the automation tool used to determine the calling location, and the grammatical structure describing the automation tool, that is, the introduction document of the automation tool, so that the task generation model can correctly identify and call the appropriate API to simulate the interactive operation process between the user and the UI interface.

[0043] like Figure 2As shown, the logic layer 203 includes an automation tool, which refers to any software or framework that can provide a variety of APIs for simulating user interaction operations to perform UI tasks. The embodiment of the present application does not limit the type of automation tool. For example, the automation tool can be Selenium, or it can be Puppeteer, or it can be Cypress. Among them, the API of the automation tool is an interface provided by the automation tool, which can be used to simulate various user interaction operations. By calling the API of the automation tool, the task generation model can perform corresponding interaction operations according to the set task logic, thereby completing the creation of UI tasks. For example, the API of the automation tool can be a click operation API, or it can be an input operation API, or it can be a sliding operation API. The embodiment of the present application does not limit the specific type and implementation method of the API of the automation tool.

[0044] In addition, after the task generation model calls the corresponding API provided by the automation tool according to the page element hierarchy information to simulate the set task logic to generate UI tasks, you can also use Figure 3 Step ⑤ shown in the figure returns the UI task to the user, so that the user can perform step ⑥ to modify the UI task generated by the task generation model. After the user modifies, the model prompt words can be regenerated in step ⑦ according to the UI task modified by the user, and the task logic set by the corresponding API simulation can be provided in step ⑧, and the model prompt words can be input into the task generation model to perform step ⑨ to generate a UI task including the user modification process. For example, the user can adjust the UI task steps, delete redundant operations or add necessary verification steps to ensure that the UI task can be completed more efficiently. The task generation model can automatically generate UI tasks, but it may not fully capture all subtle requirements or complexities in specific scenarios. For example, some UI elements may require special processing methods, or there may be some unforeseen exceptions. By allowing users to participate in the modification process, not only the accuracy and applicability of the generated tasks can be improved, but also the flexibility of the system and the user experience can be enhanced to optimize the accuracy and execution efficiency of the UI tasks. In this way, users can fine-tune the generated UI tasks according to actual needs, further improving the quality and applicability of the UI tasks.

[0045] In an optional embodiment, the task generation model includes: an input layer, a feature extraction layer, a task logic reasoning layer and an output layer. Through the collaborative work of the above four layers, the model prompt word is input into the task generation model, and the page element level information of the UI page is obtained according to the first prompt word. According to the second prompt word, the UI task is generated according to the set task logic based on the page element level information, including: the model prompt word is input into the input layer of the task generation model for parsing, and the first prompt word and the second prompt word are extracted. Among them, the input layer receives the model prompt word containing the UI page description information, and these model prompt words can be a text description or a structured data format. The input layer parses the model prompt word and identifies the first prompt word and the second prompt word in the model prompt word. The first prompt word involves the page element level information, such as element type, position, level relationship, etc.; and the second prompt word is designed to generate the UI task according to a specific task logic based on this information. After identifying the first prompt word and the second prompt word, the input layer classifies and marks the first prompt word and the second prompt word so that the subsequent layers can process the description information of the corresponding part of the UI page according to their respective responsibilities. For example, the first prompt word will be marked for feature extraction layer processing, while the second prompt word will be marked for task logic reasoning layer processing. Through the above input layer operations, the task generation model can effectively understand and classify input information, provide accurate data support for subsequent steps, and thus achieve efficient and accurate UI task generation.

[0046] In an embodiment of the present application, based on the first prompt word and the second prompt word obtained in the input layer, the first prompt word is input into the feature extraction layer for multi-dimensional feature extraction to obtain page element level features. Wherein, the page element level feature refers to the structured information extracted from the UI page to describe the attributes of the page elements and their relationships. The page element level features include at least element type features, element position features, and element relationship features. Wherein, the element type feature refers to the type of UI page element, such as buttons, input boxes, text labels, drop-down menus, pictures, links, etc.; the element position feature refers to the specific position information of the element in the page, including coordinates (such as position coordinates), layout levels (such as the relationship between parent containers and child containers), and the relative position of the element in the page (such as center, left, right, etc.); the element relationship feature refers to the logical relationship or interactive relationship between elements, such as parent-child relationship (such as a button is located in a container), sibling relationship (such as multiple input boxes are located in the same form) or functional association (such as a button is bound to an input box). The feature extraction layer first locates the target UI page based on the first prompt word, and then extracts the element type (such as buttons, input boxes, etc.), position features (such as coordinates and layout levels), and element relationship features (such as parent-child container relationships or interactive correlations) from the UI page, integrates the element type features, element position features, and element relationship features into structured data (such as vectors or graph structures), and passes them to the task logic reasoning layer for generating specific UI tasks. For example, for the first prompt word "user login page", the feature extraction layer will identify elements such as the user name input box, password input box, and submit button on the page, extract their location and relationship information, and represent them as structured data to provide a basis for subsequent task generation. In this way, the feature extraction layer can deeply understand and parse the specific layout of the UI page, providing accurate and rich data support for the subsequent task logic reasoning layer, thereby ensuring that the final generated UI task is both accurate and meets actual needs.

[0047] In an embodiment of the present application, the page element level features and the second prompt word are input into the task logic reasoning layer for task logic reasoning to generate an operation sequence corresponding to the task logic set in the second prompt word. Among them, the second prompt word describes the natural language instructions of the task logic, for example, when the user clicks the submit button, the username and password are verified. The task logic reasoning layer first parses the second prompt word to extract the key steps and logical relationships of the task. For example, parsing when the user clicks the submit button, the verification of the username and password can be decomposed into the following steps: 1. The user clicks the submit button 2. The system verifies the username and password. In the task logic reasoning layer, according to the parsed task logic, the task logic reasoning layer searches for the corresponding elements in the page element level features, for example, searching for the elements corresponding to the "submit button" (through the element type feature and the position feature), and searching for the elements corresponding to the "user name input box" and the "password input box". Subsequently, a specific operation sequence is generated according to the task logic and the page element level features. An operation sequence refers to a series of ordered operation steps generated to achieve a specific UI task. Each operation step can include user interaction actions (such as clicks, input) or system response actions (such as verification, jumps). The operation sequence is the output result of the task logic reasoning layer and is used to guide the execution of UI tasks. For example, the operation sequence is 1. The user clicks the submit button (specific location coordinates) 2. The system obtains the value of the username input box (specific location coordinates) 3. The system obtains the value of the password input box (specific location coordinates) 4. The system verifies whether the username and password match 5. If the verification is successful, jump to the home page; if it fails, an error message is displayed. Finally, the task logic reasoning layer passes the above operation sequence to the input layer for the next step. Through the above steps, the task logic reasoning layer can effectively transform the abstract task logic into a specific and executable operation sequence, which not only improves the efficiency and accuracy of automated testing, but also provides developers with a clear task execution guide, which is convenient for adjustment and optimization according to actual needs.

[0048] In an embodiment of the present application, based on the above-mentioned operation sequence, the operation sequence input and output layer is subjected to task formatting to generate an executable UI task. Wherein, formatting refers to arranging and converting data or information according to specific rules or structures so that it conforms to a certain standardized format for subsequent processing, storage or execution. First, the output layer converts the actions and parameters in the operation sequence into a unified format, then adds necessary context information (such as page name, element identifier, etc.) for each operation, converts the formatted operation sequence into a specific task instruction, and finally, outputs the generated UI task instruction for execution by the user or the system. Through the above-mentioned process, from the preliminary parsing model prompt words to the final generation of executable UI tasks, the whole process ensures the accuracy and operability of the UI task, not only improves the efficiency of UI task calls, but also provides developers with a clear task execution guide, which is convenient for adjustment and optimization according to actual needs. This method makes the complex UI task generation become systematic and standardized, meeting the needs under various application scenarios.

[0049] In an optional embodiment, the performance score of each computing node is calculated based on the multi-dimensional resource usage information of each computing node in the computing cluster, including: determining multiple weight information corresponding to multiple resource dimensions in the computing cluster according to the target attribute of the computing cluster; the target attribute points to the most important target resource dimension in the computing cluster, and the weight information corresponding to the target resource dimension is the largest. Among them, the target attribute reflects the core task requirements or management goals of the cluster, and the selection of the target attribute depends on the computing characteristics of the cluster. The weight information is used to indicate the importance of each resource dimension to the performance score, and the size of the weight depends on the target attribute and the computing characteristics of the cluster. The weight corresponding to the target attribute is the largest, and the weight information of other resource dimensions decreases in turn according to their importance. The weight value can be dynamically adjusted according to the actual operation of the cluster. For example, if the cluster is mainly used for CPU-intensive tasks (such as scientific computing and image processing), the target attribute is CPU, and the weight information corresponding to the CPU is the largest.

[0050] In an embodiment of the present application, based on multiple weight information, the multi-dimensional resource usage information of each computing node in the computing cluster is weighted and summed to obtain the performance score of each computing node, and the multi-dimensional resource usage information refers to the resource usage information corresponding to multiple resource dimensions. Among them, the performance score is a quantitative assessment of the overall performance of the computing node, which is obtained by weighted summing the usage information of each resource dimension. The resource usage information corresponding to multiple resource dimensions refers to the usage of the computing node in multiple resource dimensions. Multiple dimensions can include CPU (central processing unit), memory, IO (input / output) and thread pool.

[0051] In addition, the steps of weighted summation are determining weight information, standardizing scores, and weighted summation. Determining weight information is to determine the weight information of multiple resource dimensions according to target attributes; standardizing scores is to convert the usage information of multiple resource dimensions into standardized scores (such as scores in reverse order); and weighted summation is to multiply the standardized scores by the weight information and then add them together to obtain the performance score of the computing node. Figure 4b A schematic diagram of a structure for calculating the performance score of each computing node provided by an exemplary embodiment of the present application, such as Figure 4b As shown, based on the multi-dimensional resource usage information of each computing node in the computing cluster, formula (1) is used for weighted summation to calculate the performance score of each computing node.

[0052] y=a1×x1+a2×x2+a3×x3+a4×x4 (1)

[0053] In formula (1), y represents the performance score of the computing node; x1, x2, x3, and x4 represent the usage information of CPU, memory, IO, and thread pool respectively; a1, a2, a3, and a4 represent the weight information of resource dimensions CPU, memory, IO, and thread pool respectively. This performance score not only takes into account the actual performance of each computing node in different resource dimensions, but also emphasizes the importance of key resources by assigning different weight information, thereby ensuring that the performance score can accurately reflect the actual performance status of the computing node and help managers better understand and optimize the resource configuration and scheduling strategy of the cluster.

[0054] In an optional embodiment, the multiple resource dimensions include: CPU, memory, IO, and thread pool. Figure 4b A schematic diagram of a structure for calculating the performance score of each computing node provided by an exemplary embodiment of the present application. Figure 4bAs shown, multiple resource dimensions of the computing nodes contained in the cluster are collected, including CPU, memory, IO, and thread pool. Among them, CPU is the core computing resource of the computing node, responsible for executing instructions and processing data. CPU usage indicates the occupancy of CPU resources, which can be expressed as a percentage (such as 70%). High CPU usage indicates that the computing node is processing a large number of computing tasks; memory is a temporary storage resource of the computing node, used to store running programs and data. Memory usage indicates the occupancy of memory resources, which can be expressed as a percentage (such as 50%). High memory usage indicates that the computing node is processing a large amount of data or running memory-intensive tasks; IO refers to data transmission between the computing node and external devices (such as disks and networks). Disk I / O indicates disk read and write performance, which can be expressed as a percentage or a specific value (such as 30%). High disk I / O indicates that the computing node is processing a large number of file read and write or data storage tasks; thread pool is a collection of threads in the computing node used to manage and schedule tasks. The thread pool indicator indicates the usage of the thread pool, which can be expressed as a percentage (such as 80%). High thread pool indicators indicate that the computing node is processing a large number of concurrent tasks.

[0055] In an embodiment of the present application, according to the target attribute of the computing cluster, multiple weight information corresponding to multiple resource dimensions in the computing cluster is determined, including: when the target attribute is mainly task logic, the thread pool dimension is determined as the target resource dimension, the thread pool resource dimension is determined to correspond to the first weight information, and the CPU, memory and IO resource dimensions are determined to be the second weight information, the third weight information and the fourth weight information respectively. The second weight information, the third weight information and the fourth weight information are respectively less than the first weight information. Among them, Figure 4b A schematic diagram of a structure for calculating the performance score of each computing node provided by an exemplary embodiment of the present application. Figure 4b As shown, multiple weight information corresponding to multiple resource dimensions in the computing cluster is determined according to the target attribute. When the target attribute is mainly task logic, that is, the main optimization direction of the system is to ensure that the task can be executed efficiently, the thread pool is determined as the target resource dimension. The thread pool dimension directly affects the task scheduling and execution efficiency. For example, the first weight information corresponding to the thread pool dimension is the largest, assuming it is a4. The other three dimensions CPU, memory, and IO correspond to the second weight information (a1), the third weight information (a2), and the fourth weight information (a3), respectively, and these three weights are all less than the first weight information. The weight information is distributed as follows: the first weight information corresponding to the thread pool resource dimension is 85%, the second weight information corresponding to the CPU resource dimension is 10%, the third weight information corresponding to the memory resource dimension is 3%, and the fourth weight information corresponding to the IO resource dimension is 2%.

[0056] Optionally, in the case where the target attribute is mainly system resources, based on the relationship between the monitored UI task running status and the CPU, memory and IO resource dimensions, one of the resource dimensions of CPU, memory and IO resource dimensions is determined as the target resource dimension, the target resource dimension is determined as the fifth weight information, and the remaining three resource dimensions other than the target resource dimension are determined as the sixth weight information, the seventh weight information and the eighth weight information, respectively, and the sixth weight information, the seventh weight information and the eighth weight information are respectively less than the fifth weight information. Among them, in the case where the target attribute is mainly system resources, that is, the main optimization direction of the system is to ensure the effective use of hardware resources and the stability of the system, at this time, based on the relationship between the monitored UI task running status and the CPU, memory and IO resource dimensions, determine which resource dimension among the multiple resource dimensions is used as the target resource dimension. Collect performance data about the UI task runtime through monitoring tools, analyze the relationship between these data and the CPU, memory and IO resource dimensions, and select one of the resource dimensions as the target resource dimension based on the analysis results. For example, if it is found that the bottleneck of most UI tasks is that the CPU usage is too high, then the CPU is set as the target resource dimension, and the fifth weight information corresponding to the CPU resource dimension is the largest, assuming it is a1', and the other three resource dimensions (memory, IO, thread pool) correspond to the sixth weight information (a2'), the seventh weight information (a3') and the eighth weight information (a4'), respectively, and these three weights are all less than the fifth weight information. The weight information is allocated as follows: the first weight information corresponding to the CPU resource dimension is 80%, the second weight information corresponding to the memory resource dimension is 10%, the third weight information corresponding to the IO resource dimension is 5%, and the fourth weight information corresponding to the thread pool resource dimension is 5%. This flexibility and pertinence make the scheduling method of the UI task suitable for a variety of computing scenarios. Maximize the task processing speed and system resource utilization, thereby achieving improved UI task execution efficiency and dynamic adjustment capabilities while achieving balanced resource allocation.

[0057] For example, a computing cluster has three computing nodes, and the resource usage information of the three computing nodes is as follows:

[0058] Compute Node CPU (usage) Memory (usage) IO Thread pool (metric) Compute Node 1 70％ 50％ 30％ 80％ Compute Node 2 90％ 60％ 40％ 70％ Compute Node 3 50％ 40％ 20％ 90％

[0059] According to the scoring rules in reverse order, the scores are:

[0060] CPU: Compute node 2 (100), compute node 1 (50), compute node 3 (0);

[0061] Memory: Compute node 2 (100), compute node 1 (50), compute node 3 (0);

[0062] IO: Compute node 2 (100), compute node 1 (50), compute node 3 (0);

[0063] Thread pool: compute node 3 (100), compute node 1 (50), compute node 2 (0).

[0064] When the target attribute of the computing cluster is determined to be mainly task logic, the weight information corresponding to the multiple resource dimensions in the computing cluster is determined as follows: the first weight information corresponding to the thread pool resource dimension is 85%, the second weight information corresponding to the CPU resource dimension is 10%, the third weight information corresponding to the memory resource dimension is 3%, and the fourth weight information corresponding to the IO resource dimension is 2%. The performance score of the computing node is calculated according to formula (1):

[0065] Computation node 1: y1 = 0.85 × 50 + 0.10 × 50 + 0.03 × 50 + 0.02 × 50 = 42.5 + 5 + 1.5 + 1 = 50;

[0066] Calculation node 2: y2 = 0.85 × 0 + 0.10 × 100 + 0.03 × 100 + 0.02 × 100 = 0 + 10 + 3 + 2 = 15;

[0067] Calculation node 3: y3 = 0.85×100+0.10×0+0.03×0+0.02×0 = 85+0+0+0=85.

[0068] Finally, the computing node with the highest performance score is computing node 3 (y3=85), so computing node 3 is the optimal node. Figure 4b A schematic diagram of a structure for calculating the performance score of each computing node provided by an exemplary embodiment of the present application, such as Figure 4b As shown, the optimal computing node 3 is bound to the UI task and executes the UI task.

[0069] In addition, the usage of the thread pool dimension is determined by comprehensively considering the number of UI tasks, error rate, GC (Garbage Collection) time and heap memory usage. Among them, the number of UI tasks reflects the load of the thread pool; the error rate reflects the reliability of task execution; GC time refers to the time spent on performing garbage collection operations. You can use monitoring tools (such as JVisualVM, JConsole or GCEasy, etc.) to view the detailed information of GC activities, including the duration, frequency and amount of memory recovered for each GC; heap memory usage refers to the proportion of the current heap memory that has been allocated to UI tasks to the total available heap memory. The heap memory usage is monitored through built-in tools or third-party monitoring tools. The heap memory usage includes the used heap memory size, the maximum heap memory size, and the unused heap memory size. Through the scoring rules of the thread pool dimension, you can quickly evaluate the running status of the thread pool, and you can also take corresponding optimization measures based on the results of the thread pool dimension scoring. For example, if the score is poor, you need to increase resources (such as memory or CPU) or optimize the task design; if the score is medium, you need to further monitor and analyze potential problems; if the score is good, you can continue to maintain the current configuration.

[0070] Figure 4c A schematic diagram of the structure of a thread pool provided by an exemplary embodiment of the present application. Figure 4c As shown, the thread pool includes an execution pool, a waiting pool, and a retry pool. The number of UI tasks is used to count the number of UI tasks in the execution pool, the waiting pool, and the retry pool; the error rate is used to count the number of timed UI tasks, resultless UI tasks, and error UI tasks. Formula (2) is used to calculate the number of UI tasks on the computing node, and formula (3) is used to calculate the error rate of the computing node.

[0071] count=count1+count2+count3 (2)

[0072] Wherein, count in formula (2) represents the number of UI tasks on the computing node; count1, count2 and count3 represent the number of UI tasks in the execution pool, the waiting pool and the retry pool respectively.

[0073]

[0074] In formula (3), timeout represents the number of timed-out UI tasks, unresult represents the number of UI tasks with no results, error represents the number of erroneous UI tasks, and count represents the number of UI tasks on the computing node calculated by formula (2).

[0075] Among them, the score of the computing node thread pool dimension is determined according to the scoring rule of the thread pool dimension: if the error rate is less than 1% and the GC time is less than 100ms and the heap memory usage is less than 50%, the score of the computing node thread pool dimension is determined to be good (100); if the error rate is greater than 10% or the GC time is greater than 500ms or the heap memory usage is greater than 80%, the score of the computing node thread pool dimension is determined to be poor (0); in other cases, the score of the computing node thread pool dimension is determined to be medium (50). By comprehensively considering the number of UI tasks, error rate, GC time and heap memory usage, the running status of the thread pool can be comprehensively evaluated, the health status of the thread pool can be dynamically reflected, and the system can be helped to adjust resource allocation in time to avoid resource waste or task accumulation, so that the performance score of the computing node can be determined more accurately, and the stability and efficiency of the overall system can be improved.

[0076] In an optional embodiment, for each UI task scheduled, the performance score of each computing node is calculated. Figure 4a The following is a flow chart of a UI task scheduling method provided by an exemplary embodiment of the present application. Figure 4a As shown, before selecting the target computing node that is adapted to the execution complexity of the UI task from each computing node, it also includes: determining the priority of each UI task, and placing each UI task in a priority queue that is adapted to the priority of the UI task in multiple priority queues; scheduling the UI tasks in multiple priority queues in order from high to low. Among them, the priority of the UI task is determined by the user according to the importance and urgency of the UI task. The priority of the UI task can be divided into high priority, medium priority, and low priority. Among them, high priority represents urgent and important UI tasks that need to be executed immediately; medium priority tasks represent important tasks but can be executed later; low priority represents ordinary tasks that can be executed when resources are idle. Determine the priority according to the business logic, user needs or system policy of the task. For example, user interaction tasks (such as clicking a button) have a higher priority, while background data processing tasks may have a lower priority.

[0077] In the embodiment of the present application, according to the priority of the UI task, it is placed in the corresponding priority queue. The priority queue is a data structure used to store and manage UI tasks of different priorities. For example, the high priority queue stores high priority tasks, the medium priority queue stores medium priority tasks, and the low priority queue stores low priority tasks. Each priority queue is arranged according to the arrival time or priority order of the task, and the tasks in the high priority queue are scheduled first. Among them, the scheduling engine is the component responsible for managing and scheduling the execution of UI tasks, such as Figure 4aAs shown in the figure, the scheduling engine schedules tasks in order of priority queues from high to low. It takes tasks from the high priority queue and selects an appropriate target computing node for it. If the high priority queue is empty, it schedules the medium priority queue; it takes tasks from the medium priority queue and selects an appropriate target computing node for it. If the medium priority queue is empty, it schedules the low priority queue; it takes tasks from the low priority queue and selects an appropriate target computing node for it based on the performance score. This method can improve the efficiency of task scheduling and the overall performance of the system, and is suitable for scenarios that need to process tasks of multiple priorities.

[0078] Figure 4c A schematic diagram of the structure of a thread pool provided by an exemplary embodiment of the present application. Figure 4c As shown, the target computing node includes a thread pool, which includes an execution pool, a waiting pool, and a retry pool. Among them, the execution pool is the core component of the thread pool, which is responsible for actually running and processing UI tasks. It contains a group of active worker threads, which are assigned to execute UI tasks in the current task queue. The key to the execution pool is that it can handle multiple tasks at the same time and optimize resource usage and improve efficiency through reasonable thread management; the waiting pool is used to store UI tasks that cannot be executed immediately because the execution pool is full. The waiting pool is a buffer of the execution pool, ensuring that even under high load conditions, newly arrived UI tasks will not be discarded or lost, but queued in an orderly manner waiting for execution opportunities; the retry pool is specifically used to handle UI tasks that fail to execute or time out for the first time. The retry pool provides one or more opportunities for retrying UI tasks that fail to execute or time out for the first time, instead of simply discarding them.

[0079] In an optional embodiment, the UI task is scheduled to run on the target computing node, including: determining whether the number of UI tasks currently executed by the execution pool is greater than or equal to a set quantity threshold. Wherein, the quantity threshold refers to the maximum number of UI tasks allowed to be executed simultaneously in the execution pool, which is a preset value used to control the number of concurrently executed tasks to avoid system resource exhaustion or performance degradation due to excessive concurrency. By setting the quantity threshold, the workload of the thread pool can be effectively managed to ensure the stability and response speed of the system. In the embodiment of the present application, the specific value of the quantity threshold is not limited, and the quantity threshold can be 5, 20, or 100. The number of UI tasks currently executed by the execution pool refers to the number of UI tasks currently active (being executed) in the execution pool that the system monitors and counts in real time. This value reflects the load and resource usage of the current computing node.

[0080] Optionally, determine whether the number of UI tasks currently executed by the execution pool is greater than or equal to a set number threshold; if not, allocate an execution thread to the UI task from the execution pool, and use the execution thread to execute the UI task; if so, add the UI task to the waiting pool to wait for the number of UI tasks currently executed in the execution pool to be less than the set number threshold before entering the execution pool for execution. In other words, when a new UI task arrives, the system first checks the number of UI tasks currently being executed in the execution pool. If the number of UI tasks currently being executed is less than the set number threshold, it means that the execution pool has enough resources to process the new UI task, then allocate an available execution thread to the UI task and start executing it immediately; if the threshold is reached, the task is temporarily placed in the waiting pool, waiting in the waiting pool for the tasks in the execution pool to be completed and for idle threads to be available, then the front task is taken out of the waiting pool, and moved to the execution pool for processing.

[0081] Optionally, if the result of the execution thread executing the UI task is failure or timeout, the UI task is added to the retry pool for the retry pool to re-execute the UI task; if the UI task fails to execute in the retry pool, the UI task is discarded. Among them, for UI tasks that fail to execute or time out, they will not be discarded directly, but the UI task will be transferred to the retry pool, and the retry pool will try to execute the UI task again. If it still fails to execute successfully in the retry pool, the UI task will be discarded. The thread pool realizes efficient scheduling and execution of UI tasks through the collaborative work of the execution pool, the waiting pool and the retry pool. The execution pool is responsible for executing tasks, the waiting pool acts as a buffer to avoid overload, and the retry pool provides a fault-tolerant mechanism. This design can effectively improve the efficiency and reliability of task execution, and is suitable for scenarios where a large number of UI tasks need to be processed, maximizing the task processing speed and system resource utilization, thereby achieving improved UI task execution efficiency and dynamic adjustment capabilities while balancing resource allocation.

[0082] In addition, it should be noted that, in the case where the embodiments of the present application involve user interaction operations or trigger operations, the user interaction operations or trigger operations involved in the embodiments of the present application include but are not limited to: touch operation, gesture operation, voice operation, head movement operation, eye movement operation and other interactive operations in various ways; among which, touch operation includes but is not limited to: click operation, double-click operation, long press operation, sliding operation, pinch operation or mouse hover operation, etc. Sliding operation includes but is not limited to: straight line sliding, curve sliding, etc.

[0083] The detailed implementation and beneficial effects of each step in the method of this embodiment have been described in detail in the aforementioned embodiments and will not be elaborated here.

[0084] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel, and the sequence numbers of the operations, such as 11, 12, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0085] Figure 5 The exemplary embodiment of the present application provides a schematic diagram of an electronic device structure. Figure 5 As shown, the electronic device includes: a processor 55 and a memory 54.

[0086] The memory 54 is used to store computer programs and can be configured to store various other data to support operations on the computing platform. Examples of such data include instructions for any application or method operating on the electronic device, UI page elements, multi-dimensional resource usage information, AI-based task generation models, etc.

[0087] The memory 54 can be implemented by any type of volatile or nonvolatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0088] The processor 55 is coupled to the memory 54 and is used to execute the computer program in the memory 54, so as to: obtain the description information of at least one UI page, each UI page refers to a page that relies on at least one UI task to perform interactive operations on the UI page, and the execution complexity of different UI tasks is different; according to the description information of at least one UI page, call the AI-based task generation model to generate UI tasks to obtain at least one UI task corresponding to each UI page; according to the multi-dimensional resource usage information of each computing node in the computing cluster, calculate the performance score of each computing node; for each scheduled UI task, according to the performance score of each computing node, select a target computing node that is adapted to the execution complexity of the UI task from each computing node, and schedule the UI task to the target computing node for execution.

[0089] In an optional embodiment, the processor 55 calls an AI-based task generation model to generate UI tasks based on the description information of at least one UI page to obtain at least one UI task corresponding to each UI page, including: for each UI page, generating a model prompt word based on the description information of the UI page, the model prompt word includes a first prompt word and a second prompt word, the first prompt word is used to instruct the task generation model to obtain page element hierarchy information of the UI page, and the second prompt word is used to instruct the task generation model to generate UI tasks according to the set task logic based on the page element hierarchy information, and different task logics correspond to different UI tasks; the model prompt word is input into the task generation model, the page element hierarchy information of the UI page is obtained according to the first prompt word, and the UI task is generated according to the set task logic based on the page element hierarchy information according to the second prompt word.

[0090] In an optional embodiment, the processor 55 generates a model prompt word based on the description information of the UI page, including: obtaining the page element hierarchy information of the UI page from the description information of the UI page, and using the page element hierarchy information as the first prompt word; obtaining the use guidance information of the automation tool and the set task logic, generating a second prompt word, and using the guidance information to guide the task generation model to call the corresponding API provided by the automation tool according to the page element hierarchy information to simulate the set task logic to generate a UI task. The automation tool provides a variety of APIs for simulating interactive operations.

[0091] In an optional embodiment, the task generation model includes: an input layer, a feature extraction layer, a task logic reasoning layer and an output layer. The processor 55 inputs the model prompt word into the task generation model, obtains the page element hierarchy information of the UI page according to the first prompt word, and generates the UI task according to the set task logic based on the page element hierarchy information according to the second prompt word, including: inputting the model prompt word into the input layer of the task generation model for parsing to extract the first prompt word and the second prompt word; inputting the first prompt word into the feature extraction layer for multi-dimensional feature extraction to obtain page element hierarchy features; the page element hierarchy features include at least element type features, element position features and element relationship features; inputting the page element hierarchy features and the second prompt word into the task logic reasoning layer for task logic reasoning to generate an operation sequence corresponding to the task logic set in the second prompt word; inputting the operation sequence into the output layer for task formatting processing to generate an executable UI task.

[0092] In an optional embodiment, the processor 55 calculates the performance score of each computing node based on the multi-dimensional resource usage information of each computing node in the computing cluster, including: determining multiple weight information corresponding to multiple resource dimensions in the computing cluster based on the target attribute of the computing cluster; the target attribute points to the most important target resource dimension in the computing cluster, and the weight information corresponding to the target resource dimension is the largest; based on the multiple weight information, weighted summing the multi-dimensional resource usage information of each computing node in the computing cluster to obtain the performance score of each computing node, the multi-dimensional resource usage information refers to the resource usage information corresponding to multiple resource dimensions.

[0093] In an optional embodiment, the multiple resource dimensions include: CPU, memory, IO and thread pool; the processor 55 determines the multiple weight information corresponding to the multiple resource dimensions in the computing cluster according to the target attribute of the computing cluster, including: when the target attribute is mainly task logic, the thread pool dimension is determined as the target resource dimension, the thread pool resource dimension is determined to correspond to the first weight information, and the CPU, memory and IO resource dimensions are determined to be the second weight information, the third weight information and the fourth weight information respectively. The second weight information, the third weight information and the fourth weight information are respectively smaller than the first weight information; when the target attribute is mainly system resources, based on the relationship between the monitored UI task running status and the CPU, memory and IO resource dimensions, one of the resource dimensions of the CPU, memory and IO resource dimensions is determined as the target resource dimension, the target resource dimension is determined to be the fifth weight information, and the remaining three resource dimensions except the target resource dimension are determined to be the sixth weight information, the seventh weight information and the eighth weight information respectively, and the sixth weight information, the seventh weight information and the eighth weight information are respectively smaller than the fifth weight information.

[0094] In an optional embodiment, the processor 55 selects a target computing node that matches the execution complexity of the UI task from each computing node based on the performance scores of each computing node for each UI task scheduled, and further includes: determining the priority of each UI task and placing each UI task in a priority queue that matches the priority of the UI task among multiple priority queues; and scheduling the UI tasks in the multiple priority queues in order of their priorities from high to low.

[0095] In an optional embodiment, the target computing node includes a thread pool, which includes an execution pool, a waiting pool and a retry pool. The processor 55 schedules the UI task to the target computing node for execution, including: determining whether the number of UI tasks currently executed by the execution pool is greater than or equal to a set number threshold; if not, allocating an execution thread to the UI task from the execution pool, and using the execution thread to execute the UI task; if so, adding the UI task to the waiting pool to wait for the number of UI tasks currently executed in the execution pool to be less than the set number threshold before entering the execution pool for execution; if the result of the execution thread executing the UI task is failure or timeout, adding the UI task to the retry pool for the retry pool to re-execute the UI task; if the execution of the UI task in the retry pool fails, discarding the UI task.

[0096] Further, if Figure 5 As shown, the electronic device also includes: a communication component 56, a display 57, a power component 58, an audio component 59 and other components. Figure 5 Only some components are shown schematically, which does not mean that the electronic device only includes Figure 5 In addition, Figure 5 The components in the dashed box are optional components, not mandatory components, and the specific components depend on the product form of the working node. The working node of this embodiment can be implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone, or an IOT device, or a server-side device such as a conventional server, a cloud server, or a server array. If the working node of this embodiment is implemented as a terminal device such as a desktop computer, a laptop computer, a smart phone, etc., it can include Figure 5 If the working node of this embodiment is implemented as a server device such as a conventional server, a cloud server or a server array, it may not include Figure 5 Components within the dashed box.

[0097] Accordingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed, can implement each step that can be executed by an electronic device in the above method embodiment.

[0098] The above-mentioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0099] The above-mentioned communication component is configured to facilitate wired or wireless communication between the device where the communication component is located and other devices. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G / LTE, 5G and other mobile communication networks, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.

[0100] The above-mentioned display includes a screen, and the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.

[0101] The power supply assembly provides power to various components of the device where the power supply assembly is located. The power supply assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device where the power supply assembly is located.

[0102] The above-mentioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (Microphone, MIC), and when the device where the audio component is located is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting an audio signal.

[0103] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-readable storage media (including but not limited to disk storage, compact disc read-only memory (Compact Disc Read-Only Memory, CD-ROM), optical storage, etc.) containing computer-usable program code.

[0104] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0105] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0106] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0107] In a typical configuration, a computing device includes one or more processors (Central Processing Unit, CPU), input / output interface, network interface and memory.

[0108] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0109] Computer readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0110] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0111] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A UI task scheduling method, characterized in that: include: Obtaining description information of at least one UI page, where each UI page refers to a page that relies on at least one UI task to perform interactive operations on the UI page, and different UI tasks have different execution complexities; According to the description information of the at least one UI page, calling the AI-based task generation model to generate UI tasks, so as to obtain at least one UI task corresponding to each UI page; Calculate the performance score of each computing node based on the multi-dimensional resource usage information of each computing node in the computing cluster; For each UI task that is scheduled, a target computing node that matches the execution complexity of the UI task is selected from each computing node according to the performance score of each computing node, and the UI task is scheduled to run on the target computing node.

2. The method according to claim 1, characterized in that According to the description information of the at least one UI page, calling the AI-based task generation model to generate UI tasks to obtain at least one UI task corresponding to each UI page, including: For each UI page, a model prompt word is generated according to the description information of the UI page, and the model prompt word includes a first prompt word and a second prompt word, the first prompt word is used to instruct the task generation model to obtain the page element level information of the UI page, and the second prompt word is used to instruct the task generation model to generate a UI task according to the set task logic based on the page element level information, and different task logics correspond to different UI tasks; The model prompt word is input into the task generation model, the page element hierarchy information of the UI page is obtained according to the first prompt word, and the UI task is generated according to the set task logic based on the page element hierarchy information according to the second prompt word.

3. The method according to claim 2, characterized in that Generate model prompt words according to the description information of the UI page, including: Acquire page element level information of the UI page from the description information of the UI page, and use the page element level information as the first prompt word; The usage guidance information and set task logic of the automation tool are obtained to generate a second prompt word, wherein the usage guidance information is used to guide the task generation model to call the corresponding API provided by the automation tool according to the page element hierarchy information to simulate the set task logic to generate a UI task, and the automation tool provides a variety of APIs for simulating interactive operations.

4. The method according to claim 2, characterized in that: The task generation model includes: an input layer, a feature extraction layer, a task logic reasoning layer and an output layer. Inputting the model prompt word into the task generation model, acquiring page element level information of the UI page according to the first prompt word, and generating a UI task according to the set task logic based on the page element level information according to the second prompt word, including: Inputting the model prompt word into the input layer of the task generation model for parsing, and extracting the first prompt word and the second prompt word; Inputting the first prompt word into the feature extraction layer to perform multi-dimensional feature extraction to obtain page element level features; the page element level features at least include element type features, element position features and element relationship features; Inputting the page element level feature and the second prompt word into the task logic reasoning layer to perform task logic reasoning to generate an operation sequence corresponding to the task logic set in the second prompt word; The operation sequence is input into the output layer for task formatting to generate an executable UI task.

5. The method according to claim 1, characterized in that Based on the multi-dimensional resource usage information of each computing node in the computing cluster, the performance score of each computing node is calculated, including: Determine, according to the target attribute of the computing cluster, a plurality of weight information corresponding to a plurality of resource dimensions in the computing cluster; the target attribute points to the most important target resource dimension in the computing cluster, and the weight information corresponding to the target resource dimension is the largest; According to the multiple weight information, weighted summation is performed on the multi-dimensional resource usage information of each computing node in the computing cluster to obtain a performance score of each computing node, wherein the multi-dimensional resource usage information refers to the resource usage information corresponding to the multiple resource dimensions.

6. The method according to claim 5, characterized in that The multiple resource dimensions include: CPU, memory, IO and thread pool; according to the target attribute of the computing cluster, multiple weight information corresponding to the multiple resource dimensions in the computing cluster is determined, including: In the case where the target attribute is mainly task logic, the thread pool dimension is determined as the target resource dimension, the thread pool resource dimension is determined to correspond to the first weight information, and the CPU, memory and IO resource dimensions are determined to be the second weight information, the third weight information and the fourth weight information respectively, and the second weight information, the third weight information and the fourth weight information are respectively less than the first weight information; In the case where the target attribute is mainly system resources, based on the relationship between the monitored UI task running status and the CPU, memory and IO resource dimensions, one of the resource dimensions of CPU, memory and IO resource dimensions is determined as the target resource dimension, the target resource dimension is determined as the fifth weight information, and the remaining three resource dimensions except the target resource dimension are determined as the sixth weight information, the seventh weight information and the eighth weight information, respectively, and the sixth weight information, the seventh weight information and the eighth weight information are respectively smaller than the fifth weight information.

7. The method according to any one of claims 1 to 6, characterized in that: For each UI task that is scheduled, before selecting a target computing node that matches the execution complexity of the UI task from each computing node according to the performance score of each computing node, the method further includes: Determine the priority of each UI task, and put each UI task into a priority queue that matches the priority of the UI task among multiple priority queues; The UI tasks in the multiple priority queues are scheduled in sequence according to the priority order of the multiple priority queues from high to low.

8. The method according to any one of claims 1 to 6, characterized in that: The target computing node includes a thread pool, which includes an execution pool, a waiting pool, and a retry pool. Scheduling the UI task to the target computing node for execution includes: Determine whether the number of UI tasks currently executed by the execution pool is greater than or equal to a set number threshold; If not, allocating an execution thread to the UI task from the execution pool, and using the execution thread to execute the UI task; If yes, add the UI task to the waiting pool to wait for the number of UI tasks currently executed in the execution pool to be less than a set number threshold before entering the execution pool for execution; If the result of the execution of the UI task by the execution thread is failure or timeout, the UI task is added to a retry pool so that the retry pool can re-execute the UI task; If the execution of the UI task in the retry pool fails, the UI task is discarded.

9. An electronic device, characterized in that: include: A processor and a memory, wherein the memory is used to store a computer program, and when the computer program is executed by the processor, the processor is enabled to implement the steps in the method according to any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the processor is enabled to implement the steps in the method according to any one of claims 1 to 8.

11. A computer program product, characterized in that The method comprises a computer program / instruction, which, when executed by a processor, enables the processor to implement the steps of the method according to any one of claims 1 to 8.

Images