TASK OPTIMIZATION METHOD, TASK OPTIMIZATION DEVICE, ELECTRONIC DEVICE, AND COMPUTER-READABLE STORAGE MEDIUM
By evaluating tasks to identify optimization needs and performing global scheduling trials, the method enhances task optimization efficiency and success rates on data middle platforms, addressing inefficiencies in current optimization methods.
Patent Information
- Application Number
- JP2024532339
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2021-12-06
- Filing Date
- 2022-10-25
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2042-10-25
AI Technical Summary
Current task optimization methods on data middle platforms are inefficient due to the lack of overall standards, leading to tasks being optimized based on time and resources without considering their impact on the entire system, requiring lengthy trial-and-error processes and reducing efficiency and success rates.
A method involving task evaluation to identify target tasks requiring optimization, adjusting task parameters, and performing global scheduling trials through virtual scheduling to determine optimal parameters, thereby optimizing tasks from a global perspective.
This approach increases the efficiency and success rate of task optimization by quickly identifying tasks needing adjustment, simulating optimizations, and avoiding suboptimal verification, thus bringing the data middle platform system to an ideal operational state.
Smart Images

Figure 2024541601000001_ABST
Abstract
Description
[Technical field]
[0001] This application is filed based on a Chinese patent application bearing application number 202111482034.5 and filing date December 6, 2021, and claims priority to the Chinese patent application, the entire contents of which are incorporated herein by reference.
[0002] TECHNICAL FIELD Embodiments of the present application relate to the field of information technology, and in particular, but not exclusively, to a task optimization method, apparatus, and computer-readable storage medium. [Background technology]
[0003] The data middle platform establishes unified standards at the enterprise level, opens up enterprise data channels, breaks the information island effect, and reduces the costs of repeated construction, which not only can quickly provide data service support to various business departments, but also reduces the burden on the backend and improves the overall work efficiency and quality.
[0004] In the data middle platform, a large number of data processing tasks are executed. These tasks form a huge Directed Acyclic Graph (DAG), some of them process relatively basic data, have many dependencies on subsequent tasks, some have output time requirements, and need to produce results by a specified time. In addition, these tasks are usually developed separately by different teams, and the quality of the tasks is different. In order to ensure the operation of each task, each team usually applies for higher task resources and priorities, but this has no overall standard, which is detrimental to overall efficiency.
[0005] Currently, when optimizing a task, a task with a certain operating time or resource is usually selected as the optimization target, but only looking at the time or resource does not necessarily mean that the task has a problem. In addition, to determine whether the optimization of a task has an impact on the whole, actual testing is usually required, and the whole process requires a long cycle and a large cost of trial and error, which negatively affects the efficiency and success rate of task optimization. Summary of the Invention [Problem to be solved by the invention]
[0006] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.
[0007] The embodiments of the present application provide a task optimization method, a task optimization device, an electronic device, and a computer-readable storage medium. [Means for solving the problem]
[0008] In a first aspect, the present embodiment comprises: evaluating tasks to identify a goal task requiring optimization; adjusting a goal task parameter corresponding to the goal task; executing the target task parameters through virtual scheduling to perform a global scheduling practice run; if the rehearsal effect of the global scheduling rehearsal is as expected, determining a goal task parameter, and executing the goal task based on the goal task parameter.
[0009] In a second aspect, the present embodiment comprises: There is provided a task optimization device including a memory, a processor, and a computer program stored in the memory and operable by the processor, the processor executing the computer program to realize the task optimization method described in the first aspect.
[0010] In a third aspect, the present embodiment comprises: There is provided an electronic device including a memory, a processor, and a computer program stored in the memory and operable by the processor, the processor executing the computer program to realize the task optimization method described in the first aspect.
[0011] In a fourth aspect, the present embodiment comprises: There is provided a computer-readable storage medium storing a computer-executable program for causing a computer to execute the task optimization method according to the first aspect.
[0012] Other features and advantages of the present application will be set forth in the following specification, and in part will be obvious from the specification, or may be learned by the practice of the present application. The objectives and other advantages of the present application may be realized and obtained by the structure particularly pointed out in the written description, claims and drawings. The drawings are used to provide a further understanding of the technical solution of the present application, constitute a part of the specification, and are used to interpret the technical solution of the present application together with the embodiments of the present application, and are not intended to limit the technical solution of the present application. [Brief description of the drawings]
[0013] [Figure 1] 1 is a main flowchart of a task optimization method according to an embodiment of the present application; [Diagram 2] 4 is a sub-flowchart of a task optimization method according to an embodiment of the present application. [Diagram 3] 4 is another sub-flowchart of the task optimization method according to an embodiment of the present application. [Figure 4] 4 is another sub-flowchart of the task optimization method according to an embodiment of the present application. [Diagram 5] 1 is a task DAG graph of initial task evaluation according to one embodiment of the present application. [Figure 6] 1 is a task DAG graph of the first pre-optimization according to an embodiment of the present application. [Figure 7] 1 is a task DAG graph of the second pre-optimization according to an embodiment of the present application. [Figure 8] FIG. 1 is a structural schematic diagram of a task optimization device according to an embodiment of the present application; [Figure 9] 1 is a structural schematic diagram of an electronic device according to an embodiment of the present application; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0014] In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be described in more detail below with reference to the drawings and examples. The specific examples described in this specification are only used to interpret the present application, and are not used to limit the present application.
[0015] In the description of the embodiments of the present application, "plurality" (or "multiple kinds") means two or more, "greater than," "smaller than," "exceeds," etc. are understood to exclude the number, and "greater than," "less than," "within," etc. are understood to include the number. Descriptions such as "first," "second," etc. are provided merely for the purpose of distinguishing technical features, and should not be understood as indicating or implying relative importance or as implicitly indicating the number of technical features shown or the context of the technical features shown.
[0016] The data middle platform establishes unified standards at the enterprise level, opens up enterprise data channels, eliminates information islands, and reduces the costs of repeated construction, which not only can quickly provide data service support to various business departments, but also reduces the burden on the backend and improves the overall work efficiency and quality.
[0017] In the data middle platform, a large number of data processing tasks are executed. These tasks form a huge Directed Acyclic Graph (DAG), some of them process relatively basic data, have many dependencies on subsequent tasks, some have output time requirements, and need to produce results by a specified time. In addition, these tasks are usually developed separately by different teams, and the quality of the tasks is different. In order to ensure the operation of each task, each team usually applies for higher task resources and priorities, but this has no overall standard, which is detrimental to overall efficiency.
[0018] Currently, when optimizing a task, a task with a certain operating time or resource is usually selected as the optimization target, but only looking at the time or resource does not necessarily mean that there is a problem with the task. In addition, to determine whether the optimization of a single task affects the whole, actual testing is usually required, and the whole process requires a long cycle and a large cost of trial and error, which negatively affects the efficiency and success rate of task optimization.
[0019] In response to the current problem of blind optimization, the embodiments of the present application provide a task optimization method, apparatus, and computer-readable storage medium, which evaluates tasks to identify target tasks that need optimization, adjusts target task parameters corresponding to the target tasks, executes the target task parameters by virtual scheduling, performs a global scheduling rehearsal, and if the rehearsal effect of the task rehearsal is as expected, determines target task parameters, and executes the target task based on the target task parameters. Based on this, by evaluating tasks to identify tasks that need optimization, adjusting task parameters, and performing a global scheduling rehearsal by virtual scheduling, the optimal task optimization method can be found quickly and iteratively, which can improve the efficiency and success rate of task optimization and avoid the verification after optimization being substandard due to blind optimization. Therefore, the present application can quickly find the optimal task parameters from a global perspective, which can bring the data output capacity of the entire data middle platform system into an ideal target state.
[0020] As shown in Fig. 1, Fig. 1 is a flowchart of a task optimization method according to an embodiment of the present application. The task optimization method includes, but is not limited to, the following steps 101 to 104.
[0021] Step 101: Evaluate tasks to identify a goal task that requires optimization.
[0022] Step 102: Adjust a target task parameter corresponding to the target task.
[0023] Step 103: The target task parameters are executed through virtual scheduling, and a global scheduling practice run is performed.
[0024] Step 104: If the rehearsal effect of the global scheduling rehearsal is as expected, determine a target task parameter, and execute the target task based on the target task parameter.
[0025] The method can be used to optimize tasks operated on a data middle platform. The tasks are evaluated to identify target tasks that need optimization, the target task parameters corresponding to the target tasks are adjusted, the target task parameters are executed by virtual scheduling, a global scheduling rehearsal is performed, and if the rehearsal effect of the task rehearsal is as expected, the target task parameters are determined, and the target task is executed based on the target task parameters. Based on this, the tasks are evaluated to identify tasks that need optimization, the task parameters are adjusted, and a global scheduling rehearsal is performed by virtual scheduling, so that an optimal task optimization method can be found through rapid iterations, which can improve the efficiency and success rate of task optimization and avoid the verification after optimization being substandard due to blind optimization. Therefore, the present application can quickly find optimal task parameters from a global perspective, so that the data output capability of the entire data middle platform system can be brought to an ideal target state. Note that the identification and adjustment of tasks can be performed by humans or automatically by machine learning.
[0026] A method of evaluating a task to identify a target task that needs optimization may be to collect static and dynamic information of the task, create an evaluation model of the task from the static and dynamic information, and identify a target task that needs optimization from an evaluation result of the evaluation model. Note that the static information is configuration information of the task, such as, but not limited to, input form, output form, resource allocation, task priority, scheduling cycle, etc., and the dynamic information is historical execution information of the task, such as, but not limited to, historical average execution time, average startup delay, average data processing amount, dependency degree, etc.
[0027] The evaluation model of a task can be expressed as S(evaluation dimension of a task)=Exec(influence factor of a task), where S represents the evaluation of several dimensions of a task, and Exec is a fitted task execution model that can estimate the value of each evaluation dimension from several influence factors. For example, the evaluation model of a task created from the static and dynamic information of a task is S(importance, algorithm efficiency, delay, time)=Exec(dependency, algorithm efficiency, priority, resource allocation), where dependency refers to the kinship of tasks, importance is defined as the total number of subsequent tasks that directly or indirectly depend on the task, and algorithm efficiency refers to the amount of data that can be processed per unit time per unit resource.
[0028] By collecting static and dynamic information of tasks, the present application can create a task evaluation model S (task evaluation dimension) = Exec (task influence factor) from the information of each task, where S represents the evaluation for several dimensions of the task, and Exec is a fitted task execution model that can estimate the value of each evaluation dimension from several influence factors. A target task that needs optimization is identified from the evaluation result, and the identification method can be human identification or automatic identification by rules. For a task that is determined to need optimization, the target task parameters of the task execution model are adjusted according to the optimization direction that means the evaluation dimension to be improved. The adjusted task execution model is subjected to a global scheduling practice by virtual scheduling, and if the practice effect is as expected, the adjusted task configuration is applied to the production system, and if not, the parameters of the task execution model are continued to be adjusted for optimization. Based on this, the present application can identify which tasks need optimization from a global perspective, and optimize task parameters by a global scheduling practice by virtual scheduling to improve the operation efficiency of the system.
[0029] As shown in FIG. 2, step 101 may include the following steps 1011 to 1013, but is not limited to these.
[0030] Step 1011: Collect static information and dynamic information of the task, where the static information is configuration information of the task and the dynamic information is historical execution information of the task.
[0031] Step 1012: A task evaluation model is created from the static information and the dynamic information.
[0032] Step 1013: Identify a goal task that requires optimization from the evaluation results of the evaluation model.
[0033] Static information of a task may be collected from configuration information of the task, where the static information may include, but is not limited to, input forms, output forms, resource allocation, task priority, scheduling cycle, etc.; historical execution information of a task may be obtained from execution logs of the task, where dynamic information may include, but is not limited to, historical average execution time, average startup delay, average data processing amount, dependency degree, etc.
[0034] The evaluation dimension and the influencing factor of the task can be determined from the static information and the dynamic information, and the evaluation model of the task can be created from the evaluation dimension and the influencing factor. The evaluation model of the task can be expressed as S (evaluation dimension of the task) = Exec (influencing factor of the task). Here, S represents the evaluation of several dimensions of the task, and Exec is a fitted task execution model that can estimate the value of each evaluation dimension from several influencing factors. For example, the evaluation model of the task created from the static information and the dynamic information of the task is S (importance, algorithm efficiency, delay, time) = Exec (dependency, algorithm efficiency, priority, resource allocation), where dependency refers to the kinship of the task, importance is defined as the total number of subsequent tasks that directly or indirectly depend on the task, and algorithm efficiency refers to the amount of data that can be processed per unit time per unit resource. Based on this, the present application rapidly simulates the optimization effect by creating a task evaluation model and virtual scheduling, and improves the optimization efficiency of the entire task. The multi-dimensional and three-dimensional task evaluation index provides effective guidance for the direction of task optimization.
[0035] In this application, a task evaluation model is created to associate the task evaluation information of user's interest with a simplified execution model. Tasks that need optimization can be quickly identified according to the evaluation information and business goals, and the optimal task optimization method can be quickly iterated to find by adjusting task parameters and performing task dry runs in a virtual scheduling engine, thereby improving the efficiency and success rate of task optimization and avoiding the post-optimization verification being substandard due to blind optimization.
[0036] As shown in FIG. 3, step 102 may include, but is not limited to, steps 1021 and 1022.
[0037] Step 1021: The optimization direction of the target task, which is the evaluation dimension to be improved for the target task, is determined.
[0038] Step 1022: Adjust a goal task parameter corresponding to the goal task according to the optimization direction of the goal task.
[0039] For the tasks determined to need optimization, adjust the parameters of the task execution model according to the optimization direction, where the optimization direction is the evaluation dimension to be improved, and the evaluation dimension may include, but is not limited to, importance, algorithm efficiency, delay, time, etc., for example, if the algorithm efficiency is in the normal range but the time is too long, it is necessary to consider increasing resources, and if the delay is too large, it is necessary to consider optimizing the scheduling logic. The target task parameters corresponding to the target task are adjusted according to the evaluation dimension to be improved of the target task.
[0040] As shown in FIG. 4, step 103 may include, but is not limited to, steps 1031 and 1032.
[0041] Step 1031: Create an execution model of the target task obtained by fitting the influencing factors of the target task.
[0042] Step 1032: Using the execution model, execute the target task parameters through virtual scheduling, and perform a global scheduling rehearsal.
[0043] The influencing factors of a task may include, but are not limited to, dependency, algorithm efficiency, priority, resource allocation, etc., and Exec is a fitting task execution model that can estimate the value of each evaluation dimension from several influencing factors, and the evaluation model of a task is S(importance, algorithm efficiency, delay, time)=Exec(dependence, algorithm efficiency, priority, resource allocation), where dependency refers to the kinship of a task, importance is defined as the total number of subsequent tasks that directly or indirectly depend on the task, and algorithm efficiency refers to the amount of data that can be processed per unit time per unit resource. The execution model is used to execute the target task parameters by virtual scheduling, and a global scheduling rehearsal is performed. Based on this, the present application performs an overall scheduling rehearsal by virtual scheduling through the adjusted execution model, quickly simulating the optimization effect and improving the optimization efficiency of the entire task.
[0044] In this application, a task evaluation model is created to associate the task evaluation information of user's interest with a simplified execution model. Tasks that need optimization can be quickly identified according to the evaluation information and business goals, and the optimal task optimization method can be quickly iterated to find by adjusting task parameters and performing task dry runs in a virtual scheduling engine, thereby improving the efficiency and success rate of task optimization and avoiding the post-optimization verification being substandard due to blind optimization.
[0045] The task optimization method according to the present application will be further described below with reference to the drawings and specific embodiments.
[0046] Taking one simplified task of the data middle platform as an example, if the total resources of the data middle platform are 10, the corresponding steps for task optimization are as follows:
[0047] a. Mainly, static information of the task including input form, output form, resource allocation, task priority, scheduling cycle, etc. is collected from configuration information, and dynamic information, which is historical execution information including historical average execution time, average startup delay, average data processing amount, dependency degree, etc., is obtained from the task execution log.
[0048] b. A task evaluation model is created based on the static and dynamic information of each task, and the overall task DAG graph is shown in Fig. 5.
[0049] c. Tasks that require optimization are identified from the evaluation results. Here, since the goal is defined as optimizing the startup delay of t31, the tasks that require optimization as a result of the identification are t12 and t22.
[0050] d. Optimize t12 first: due to insufficient resources and priority, t12 has to wait to run after t11 completes, resulting in a delay.
[0051] (1) A rehearsal was conducted in which the priority of t12 was adjusted to 100. Although the delays of t12 and t22 became 0, the delays of t11 and t21 increased. As a result, the delay of t31 did not change, so this adjustment was not adopted.
[0052] (2) A dry run was performed by reducing the t12 task resource to 5, but as shown in Figure 6, the t31 task delay was reduced to 3, so this optimization was adopted.
[0053] e. Based on the previous step, optimizing the next efficiency parameter of t12 is difficult, so optimize t22 instead.
[0054] (1) Adjust the task resource of t22 to 6 and reduce the delay of t31 to 1, adopt this optimization and move on to the next step.
[0055] (2) Because the efficiency of the t22 algorithm is low, a trial run is conducted after increasing the algorithm efficiency of t22 to 0.7, and as shown in Figure 7, the delay of the t31 task is eliminated, and this optimization is adopted.
[0056] f. The algorithm optimization performed by decreasing the task resources at t11 and increasing the task resources at t22 after the adjustment is applied to the production system.
[0057] It should be noted that the above steps c, d and e can be performed manually or automatically by a program.
[0058] As shown in FIG. 8, an embodiment of the present application also provides a task optimization apparatus.
[0059] In some embodiments, the task optimization device includes one or more processors and a memory, with one processor and memory illustrated in Figure 8. The processor and memory may be connected via a bus or in other ways, with Figure 8 illustrating an example in which they are connected via a bus.
[0060] The memory can be used as a non-transitory computer-readable storage medium to store non-transitory software programs and non-transitory computer-executable programs, such as the task optimization method in the above embodiment of the present application. The processor executes the non-transitory software programs and programs stored in the memory to realize the task optimization method in the above embodiment of the present application.
[0061] The memory may include a program storage area capable of storing an operating system and / or applications required for at least one function, and a data storage area capable of storing data required to execute the task optimization method in the above-described embodiment of the present application. Furthermore, the memory may include a high-speed random access memory, and may include non-transient memory, such as at least one disk memory device, flash memory device, or other non-transient solid-state memory device. In some embodiments, the memory may include a memory located remotely relative to the processor, and such a remotely located memory may be connected to the task optimization device via a network. Examples of the above-described network include, but are not limited to, the Internet, a corporate intranet, a local area network, a mobile communication network, and combinations thereof.
[0062] The non-transitory software programs and programs required to realize the task optimization method in the above embodiment of the present application are stored in memory, and when executed by one or more processors, perform the task optimization method in the above embodiment of the present application, such as steps 101-104 of the method in FIG. 1, steps 1011-1013 of the method in FIG. 2, steps 1021-1022 of the method in FIG. 3, and steps 1031-1032 of the method in FIG. 4. Evaluate the tasks to identify the target tasks that need optimization, adjust the target task parameters corresponding to the target tasks, execute the target task parameters by virtual scheduling, perform a global scheduling rehearsal, and if the rehearsal effect of the task rehearsal is as expected, determine the target task parameters, and execute the target task based on the target task parameters. Based on this, the tasks can be evaluated to identify the tasks that need optimization, adjust the task parameters, and perform a global scheduling rehearsal by virtual scheduling, thereby quickly iteratively finding the optimal task optimization method, improving the efficiency and success rate of task optimization, and avoiding the verification after optimization being substandard due to blind optimization. Therefore, the present application can quickly find optimal task parameters from a global perspective, thereby bringing the data output capability of the entire data middle platform system into an ideal target state.
[0063] As shown in FIG. 9, an embodiment of the present application also provides an electronic device.
[0064] In some embodiments, the electronic device includes one or more processors and a memory, with one processor and memory illustrated in Figure 9. The processor and memory may be connected via a bus or in other ways, with Figure 9 illustrating an example where they are connected via a bus.
[0065] The memory can be used as a non-transitory computer-readable storage medium to store non-transitory software programs and non-transitory computer-executable programs, such as the task optimization method in the above embodiment of the present application. The processor executes the non-transitory software programs and programs stored in the memory to realize the task optimization method in the above embodiment of the present application.
[0066] The memory may include a program storage area capable of storing an operating system and / or applications required for at least one function, and a data storage area capable of storing data required to execute the task optimization method in the above-described embodiment of the present application. Furthermore, the memory may include a high-speed random access memory, and may include non-transient memory, such as at least one disk memory device, flash memory device, or other non-transient solid-state memory device. In some embodiments, the memory may include a memory located remotely relative to the processor, and such a remotely located memory may be connected to the task optimization device via a network. Examples of the above-described network include, but are not limited to, the Internet, a corporate intranet, a local area network, a mobile communication network, and combinations thereof.
[0067] The non-transitory software programs and programs required to realize the task optimization method in the above embodiment of the present application are stored in memory, and when executed by one or more processors, perform the task optimization method in the above embodiment of the present application, such as the above steps 101-104, steps 1011-1013 of the method in FIG. 2, steps 1021-1022 of the method in FIG. 3, and steps 1031-1032 of the method in FIG. 4. Evaluate the tasks to identify the target tasks that need optimization, adjust the target task parameters corresponding to the target tasks, execute the target task parameters by virtual scheduling, perform a global scheduling rehearsal, and if the rehearsal effect of the task rehearsal is as expected, determine the target task parameters, and execute the target task based on the target task parameters. Based on this, the tasks can be evaluated to identify the tasks that need optimization, adjust the task parameters, and perform a global scheduling rehearsal by virtual scheduling, thereby quickly iteratively finding the optimal task optimization method, thereby improving the efficiency and success rate of task optimization, and avoiding the verification after optimization being substandard due to blind optimization. Therefore, the present application can quickly find optimal task parameters from a global perspective, thereby bringing the data output capability of the entire data middle platform system into an ideal target state.
[0068] Further, the embodiment of the present application also provides a computer-readable storage medium storing a computer-executable program, which, when executed by one or more control processors, for example, one processor in FIG. 8, causes the one or more processors to execute the task optimization method in the above embodiment of the present application, for example, steps 101-104, steps 1011-1013 of the method in FIG. 2, steps 1021-1022 of the method in FIG. 3, and steps 1031-1032 of the method in FIG. 4. Evaluate tasks to identify target tasks that require optimization, adjust target task parameters corresponding to the target tasks, execute the target task parameters by virtual scheduling, perform a global scheduling rehearsal, and if the rehearsal effect of the task rehearsal is as expected, determine target task parameters, and execute the target tasks based on the target task parameters. Based on this, the tasks can be evaluated to identify those that need optimization, the task parameters can be adjusted, and a global scheduling practice run can be performed through virtual scheduling to quickly iteratively find the optimal task optimization method, thereby improving the efficiency and success rate of task optimization and avoiding the post-optimization verification being substandard due to blind optimization. Therefore, the present application can quickly find the optimal task parameters from a global perspective, thereby making the data output capability of the entire data middle platform system into an ideal target state.
[0069] The embodiment of the present application includes the steps of: evaluating tasks to identify target tasks that need optimization; adjusting target task parameters corresponding to the target tasks; executing the target task parameters by virtual scheduling to perform a global scheduling rehearsal; and, if the rehearsal effect of the task rehearsal is as expected, determining target task parameters and executing the target task based on the target task parameters. Based on this, the task is evaluated to identify tasks that need optimization, adjusting the task parameters, and performing a global scheduling rehearsal by virtual scheduling, so as to quickly iteratively find an optimal task optimization method, thereby improving the efficiency and success rate of task optimization, and avoiding the verification after optimization being substandard due to blind optimization. Therefore, the present application can quickly find optimal task parameters from a global perspective, thereby making the data output capacity of the entire data middle platform system into an ideal target state.
[0070] All or part of the steps in the methods and systems disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. Some or all of the physical components may be implemented as software executed by a processor, such as a central processor, digital signal processor, microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer readable medium, which may include computer storage media (or non-transitory media) and communication media (or transitory media). As known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (e.g., computer readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cartridge, magnetic tape, magnetic disk storage or other magnetic storage device, or any other medium that can be used to store the desired information and that can be accessed by a computer. Additionally, communication media typically includes computer readable instructions, data structures, program modules or other data in a modulated data signal, such as a carrier wave or other transmission mechanism, and may include any information delivery media as known to those skilled in the art.
[0071] Although several implementations of the present application have been specifically described above, the present application is not limited to the above embodiments, and a person skilled in the art may make various equivalent modifications or substitutions without departing from the spirit of the present application, and all of these equivalent modifications or substitutions are intended to be included within the scope defined by the claims of the present application.
Claims
1. 1. A task optimization method, comprising: evaluating tasks to identify a goal task requiring optimization; adjusting a goal task parameter corresponding to the goal task; executing the target task parameters through virtual scheduling to perform a global scheduling practice run; if the rehearsal effect of the global scheduling rehearsal is as expected, determining a goal task parameter, and executing the goal task based on the goal task parameter.
2. The step of evaluating the tasks to identify the goal task requiring optimization comprises: collecting static information and dynamic information of the task, the static information being configuration information of the task and the dynamic information being historical execution information of the task; creating an evaluation model of the task from the static information and the dynamic information; and identifying the target task requiring optimization from an evaluation result of the evaluation model.
3. The static information is Task entry form, or Task output form, or Task resource allocation, or the priority of said task, or The method of claim 2 , further comprising at least one of a task schedule cycle.
4. The dynamic information is the historical average execution time of said task, or the average startup delay of said task, or the average data processing volume of said task; or The method of claim 2 , including at least one of the task dependencies.
5. The step of creating the evaluation model of the task from the static information and the dynamic information includes: determining an evaluation dimension of the task and an influence factor of the task from the static information and the dynamic information; and creating the evaluation model from the evaluation dimensions and the influence factors.
6. The step of adjusting a target task parameter corresponding to the target task includes: determining an optimization direction of the target task, which is the evaluation dimension to be improved for the target task; and adjusting the goal task parameters corresponding to the goal task according to an optimization direction of the goal task.
7. The step of executing the target task parameters by virtual scheduling and performing a global scheduling practice run includes: creating an execution model of the target task obtained by fitting the influencing factors of the target task; 7. The method of claim 6, further comprising: executing the target task parameters by virtual scheduling using the execution model to perform a global scheduling rehearsal.
8. A task optimization device comprising: a memory; a processor; and a computer program stored in the memory and operable by the processor, the processor executing the computer program to realize the task optimization method according to any one of claims 1 to 7.
9. An electronic device including a memory, a processor, and a computer program stored in the memory and operable by the processor, the processor executing the computer program to realize the task optimization method according to any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer-executable program for causing a computer to execute the task optimization method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Parameter optimization control method and device and distributed computing system
CN109710395A
Task scheduling method, device, platform, server and storage medium
CN111625331A
Task attribute optimization method and device, server and storage medium
CN112052082A
Optimization of task execution
US20170357530A1
