Efficient task scheduling method and system based on data enumeration structure optimization

Abstract

The invention discloses an efficient task scheduling method and system based on data enumeration structure optimization. The method comprises the steps that task information is collected, distribution characteristics in the task information are calculated, the distribution characteristics are classified, different standardization methods are adopted for the classified distribution characteristics, and standardized task information is obtained; obtaining task feature vectors according to the standardized task information, creating a multi-level index structure, dividing the task feature vectors, and calculating corresponding comprehensive evaluation functions through the divided task feature vectors; setting pruning conditions through the comprehensive evaluation function, pruning the multi-level index structure, generating a task scheduling scheme according to the pruned multi-level index structure, and executing the task scheduling scheme; through dynamic standardization processing and pruning operation, unnecessary task scheduling calculation in the system is reduced while the task attribute distribution difference is reduced, the task scheduling accuracy and efficiency are improved, and the system stability is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of task scheduling optimization, and in particular to an efficient task scheduling method and system based on data enumeration structure optimization. Background Art

[0002] As the scale of modern computing systems continues to expand, the importance of task scheduling in various computing environments has become increasingly prominent, especially in cloud computing, distributed computing, and high-performance computing systems, where the efficiency of task scheduling directly affects the performance of the system. Traditional task scheduling methods mostly rely on simple priority algorithms or static scheduling strategies, which can provide certain scheduling efficiency when the number of tasks is small or the task attributes are relatively simple. However, with the increase in task complexity and the multidimensionality of system resources, traditional scheduling methods have gradually exposed their limitations.

[0003] In the current development, one of the core issues of task scheduling is how to achieve efficient task sorting and resource allocation under high-dimensional task attributes. Most traditional scheduling methods only focus on a single attribute and fail to fully consider the balance of tasks on multiple attributes. This single scheduling strategy often leads to unbalanced allocation of system resources, with some resources being over-occupied and other resources being idle, resulting in a decrease in the overall efficiency of the system. At the same time, in scenarios where the distribution of task attributes varies greatly, the existing scheduling methods fail to introduce effective attribute standardization processing, which makes it difficult for the system to fairly compare the resource requirements and priorities of different tasks, further affecting the accuracy and efficiency of scheduling. In addition, the existing technology also has certain limitations when dealing with task dependencies. When dealing with task dependencies, many scheduling algorithms usually only perform simple sorting through the static dependency chain of tasks, while ignoring the dynamic changes that may occur during task execution. As the system scale expands and the complexity of tasks increases, task dependencies become more complex, and a simple dependency chain sorting mechanism is difficult to flexibly respond to dynamic changes in tasks in complex scenarios, thereby affecting the overall efficiency of task scheduling and system response time. At the same time, under the condition of limited resources, the existing technology lacks a flexible resource scheduling strategy and cannot fully meet the dynamic needs of tasks for multi-dimensional resources.

[0004] Therefore, how to effectively balance the multi-attribute requirements of tasks and improve the overall efficiency of scheduling in a complex multi-dimensional task attribute environment in a more intelligent way has become a difficult problem that needs to be solved urgently. Summary of the invention

[0005] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0006] In view of the above existing problems, the present invention is proposed. Therefore, the present invention provides an efficient task scheduling method based on data enumeration structure optimization to solve the problems mentioned in the background technology.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0008] In a first aspect, the present invention provides an efficient task scheduling method based on data enumeration structure optimization, comprising:

[0009] Collecting task information, calculating distribution features in the task information, classifying the distribution features, and using different standardization methods for the classified distribution features to obtain standardized task information;

[0010] According to the standardized task information, a task feature vector is obtained, a multi-level index structure is created, the task feature vector is divided, and the corresponding comprehensive evaluation function is calculated through the divided task feature vector;

[0011] The pruning conditions are set through the comprehensive evaluation function, the multi-level index structure is pruned, and a task scheduling scheme is generated and executed according to the pruned multi-level index structure to obtain an efficient task scheduling method.

[0012] As a preferred solution of the efficient task scheduling method based on data enumeration structure optimization described in the present invention, wherein: the collecting task information includes:

[0013] The collected task information includes a task set T to be scheduled, and the task set T includes multiple tasks, each task N Contains four task attributes, N represents the total number of tasks;

[0014] The four task attributes are: execution time estimate E i , Priority P i , resource demand vector R i And task dependencies D i .

[0015] As a preferred solution of the efficient task scheduling method based on data enumeration structure optimization described in the present invention, wherein: the distribution characteristics in the task information are calculated, the distribution characteristics are classified, and different standardization methods are used for the classified distribution characteristics to obtain standardized task information, including:

[0016] Calculate the mean of the task attributes in each task, obtain the standard deviation of each task from the mean of the task attributes, and use the standard deviation as the distribution feature corresponding to each task;

[0017] A range value of the distribution feature between each task is set, and the distribution feature is divided into multiple intervals according to the range value, and each interval adopts a different standardization rule.

[0018] As a preferred solution of the efficient task scheduling method based on data enumeration structure optimization described in the present invention, each interval adopts different standardized rules, including:

[0019] If there are no outliers in the current interval range and the current interval range is evenly distributed, linear normalization is used;

[0020] If there are no outliers in the current interval range value, and the current interval range value belongs to a positively skewed distribution, logarithmic standardization is used;

[0021] If there are no outliers in the current interval range value, and the current interval range value belongs to a negatively skewed distribution, exponential normalization is used;

[0022] If there are outliers in the current interval range and they do not meet the above interval range values, quantile normalization is used;

[0023] If there are outliers in the current interval range, the range value of the distribution feature between each task is reset.

[0024] As a preferred solution of the efficient task scheduling method based on data enumeration structure optimization described in the present invention, wherein: according to the standardized task information, a task feature vector is obtained, a multi-level index structure is created, and the task feature vector is divided, including:

[0025] Extract each task attribute from the standardized task information and combine them into a task feature vector V i , the task feature vector V i Represented as the root node of a multi-level index structure;

[0026] The task feature vector V i The resource demand vector R in i As a node of a multi-level index structure, each node contains at least one child node, and the child node represents a resource demand vector R i The resource type in the node selects a unique dimension for each node to divide;

[0027] In the selection dimension, the resource type of the current node is in the resource demand vector R i The data is divided into the dimension with the largest variance;

[0028] In terms of partitioning, the median of the node is selected as the partition point to divide the tasks in the node into left and right sub-nodes.

[0029] As a preferred solution of the efficient task scheduling method based on data enumeration structure optimization described in the present invention, wherein: the corresponding comprehensive evaluation function is calculated by the divided task feature vector, and the pruning condition is set by the comprehensive evaluation function to prune the multi-level index structure, including:

[0030] Define a comprehensive evaluation function and calculate the comprehensive score S of the divided task feature vector;

[0031] Set the pruning threshold Θ. When the comprehensive score S is less than Θ, the task corresponding to the task feature vector after division is considered to be N Under current conditions, priority P i Too low, the resource demand vector R in the original task feature vector i The nodes represented by are pruned until the child nodes become leaf nodes.

[0032] As a preferred solution of the efficient task scheduling method based on data enumeration structure optimization described in the present invention, wherein: generating and executing a task scheduling scheme according to the pruned multi-level index structure includes:

[0033] Re-assign tasks through the pruned multi-level index structure N Sort the tasks in order from high to low. N Perform scheduling and check task dependencies during scheduling i , if the task dependency D i If there is an unfinished scheduled task, skip the task and the unfinished scheduled task, schedule the next task, and record the actual time of the scheduled task in the current process. Estimated execution time i Compare, if the same, do not update the execution time estimate E i If they are different, the actual time of the scheduled task in the current process is used. Replace the execution time estimate E i .

[0034] In a second aspect, the present invention provides an efficient task scheduling system based on data enumeration structure optimization, which includes:

[0035] The task information collection and preprocessing module is configured to collect task information, calculate distribution characteristics in the task information, classify the distribution characteristics, and use different standardization methods for the classified distribution characteristics to obtain standardized task information;

[0036] The task information feature extraction and index structure construction module is configured to obtain a task feature vector according to the standardized task information, create a multi-level index structure, divide the task feature vector, and calculate the corresponding comprehensive evaluation function through the divided task feature vector;

[0037] The task scheduling and scheme generation module is configured to set pruning conditions through the comprehensive evaluation function, prune the multi-level index structure, generate and execute a task scheduling scheme according to the pruned multi-level index structure, and obtain an efficient task scheduling method.

[0038] In a third aspect, the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: the processor implements any step of the above method when executing the computer program.

[0039] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: the computer program implements any step of the above method when executed by a processor.

[0040] Compared with the prior art, the invention has the following beneficial effects:

[0041] 1. Aiming at the problem that the existing technology fails to effectively handle the distribution characteristics of task attributes, the present invention classifies the attribute distribution in the task information and adopts different standardization methods according to different distribution types; the dynamic standardization processing mechanism can ensure that the attributes of tasks in multiple dimensions can be reasonably compared, avoiding the scheduling unfairness caused by the large difference in task attribute distribution, thereby improving the accuracy of scheduling and the fairness between tasks;

[0042] 2. Aiming at the problem of low efficiency of index structure in processing high-dimensional task attributes in the prior art, the present invention creates a multi-level index structure based on task feature vectors, performs detailed division at each node by the resource requirement vector of the task, selects the dimension with the largest data variance as the basis for division, and uses the median as the division point during division; this optimized index structure greatly improves the efficiency of task retrieval and reduces the computational overhead in the scheduling process, especially when processing large-scale task sets, it can significantly improve the speed and accuracy of task allocation;

[0043] 3. By introducing a comprehensive evaluation function, the multi-dimensional attributes of the task, such as execution time, priority and resource requirements, are taken into account, which solves the limitation of the existing scheduling algorithm that only focuses on a single task attribute; through comprehensive scoring, the system can more comprehensively evaluate the scheduling priority of the task; at the same time, combined with the pruning mechanism, low-priority tasks are dynamically pruned in the multi-level index structure, unnecessary task scheduling calculations are reduced, and the overall scheduling efficiency of the system is further improved; compared with the traditional simple sorting mechanism, the comprehensive scoring and pruning strategy of the present invention significantly enhances the flexibility and efficiency of task scheduling;

[0044] 4. The prior art often relies on static dependency chain sorting when processing task dependencies, and fails to dynamically adapt to changes in task dependencies. The present invention performs real-time checks on task dependencies during task scheduling to ensure that subsequent tasks are not scheduled in advance when dependent tasks are not completed. At the same time, by recording the difference between the actual execution time and the estimated execution time of the task, and updating the execution time of the task according to actual conditions, dynamic optimization and self-adjustment of the scheduling plan are achieved, ensuring that task dependencies can adapt to system changes more flexibly, further improving the system's responsiveness and the stability of task execution. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0046] Figure 1 The overall flow chart of an efficient task scheduling method based on data enumeration structure optimization according to an embodiment of the present invention;

[0047] Figure 2 A standardized rule judgment flow chart of an efficient task scheduling method based on data enumeration structure optimization according to an embodiment of the present invention;

[0048] Figure 3 A task resource utilization comparison diagram of an efficient task scheduling method based on data enumeration structure optimization according to an embodiment of the present invention;

[0049] Figure 4 A radar chart of task execution time of an efficient task scheduling method based on data enumeration structure optimization described in one embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0051] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0052] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0053] The present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0054] At the same time, in the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper, lower, inner and outer" are based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first, second or third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0055] In the present invention, unless otherwise clearly specified and limited, the terms "install, connect, connect" should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection or an integral connection; it can also be a mechanical connection, an electrical connection or a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0056] Example 1

[0057] Reference Figure 1 and Figure 2, which is the first embodiment of the present invention, and provides an efficient task scheduling method based on data enumeration structure optimization, comprising:

[0058] S1. Collect task information, calculate distribution features in the task information, classify the distribution features, and use different standardization methods for the classified distribution features to obtain standardized task information;

[0059] Furthermore, the collected task information includes a task set T to be scheduled, and the task set T includes multiple tasks, each task N Contains four task attributes, N represents the total number of tasks;

[0060] Specifically, the task set T is expressed as:

[0061] T={task1,task2,…,task N}

[0062] Specifically, the four task attributes are: Estimated execution time i , Priority P i , resource demand vector R i And task dependencies D i ;

[0063] It should be explained that the execution time estimate E i Represents each task task N The expected execution time of i Represents each task task N The priority of resource demand vector R i Represents each task task N The demand for various resources; task dependencies D i Represents a set of dependent tasks. Each task has corresponding dependent tasks, which can be adjacent tasks or non-adjacent tasks.

[0064] Specifically, the resource demand vector R i It is expressed as:

[0065] R i =[R i1 ,R i2 ,…,R im ]

[0066] Among them, m represents the resource type;

[0067] Furthermore, the mean of the task attributes in each task is calculated, and the standard deviation of each task is obtained from the mean of the task attributes, and the standard deviation is used as the distribution feature corresponding to each task;

[0068] It should be noted that the mean of the task attributes is estimated by the execution time E i , Priority P i , resource demand vector R i And task dependencies D i Get, and the standard deviation is determined by the current task task N The square of the difference from the mean is obtained;

[0069] Furthermore, the range value of the distribution feature between each task is set, and the range value is divided into multiple intervals, and each interval adopts a different normalization rule;

[0070] For further reference, Figure 2 , the standardization rules are as follows:

[0071] Rule 1: If there are no outliers in the current interval range and the current interval range is uniformly distributed, linear normalization is used;

[0072] Rule 2: If there are no outliers in the current interval range and the current interval range belongs to a positively skewed distribution, logarithmic standardization is used;

[0073] Rule 3: If there are no outliers in the current interval range value, and the current interval range value belongs to a negatively skewed distribution, use exponential normalization;

[0074] Rule 4: If there are no outliers in the current interval range and none of the interval range values ​​meet the requirements of Rules 1 to 3, quantile normalization is used;

[0075] Rule 5: If there are outliers in the current interval range, reset the range value of the distribution feature between each task;

[0076] It should be noted that when the amount of data processing is too large, the scheduling system usually exhibits slow response or scheduling rigidity when facing task changes, especially when the task attributes fluctuate, it is difficult to respond quickly; through dynamic interval division and standardization methods, it can respond to task changes more quickly, not only maintaining efficient task scheduling, but also enhancing the flexibility and robustness of the system;

[0077] S2. Obtain a task feature vector based on the standardized task information, create a multi-level index structure, divide the task feature vector, and calculate the corresponding comprehensive evaluation function through the divided task feature vector;

[0078] Furthermore, each task attribute is extracted from the standardized task information and combined into a task feature vector V i , by the task feature vector V i Represented as the root node of a multi-level index structure;

[0079] Specifically, the task feature vector V i It is expressed as:

[0080] V i =[E i ,P i ,R i ,D i ]

[0081] Furthermore, the task feature vector V i The resource demand vector R in i As a node of a multi-level index structure, each node contains at least one child node, which represents a resource demand vector R i The resource type in the node selects a unique dimension for each node to divide;

[0082] Furthermore, in the selection dimension, the resource type of the current node is used in the resource demand vector R i The data is divided into the dimension with the largest variance;

[0083] Furthermore, in terms of partitioning, the median of the node is selected as the partition point to divide the tasks in the node into left and right child nodes;

[0084] It should be explained that in the data structure, the node is the basic unit of the tree structure; each node contains data and a pointer to its child nodes; the hierarchy and relationship of the nodes determine the structure of the tree; in the scheme of the present invention, the multi-level index structure is a tree, in which the root node is the starting node of the tree structure and stores the task feature vector V i , the node is the branch node storage resource demand vector R i If there is no child node under the node, the node is represented as a leaf node; the resource demand vector R is stored under the child node i The resource type in

[0085] It should be noted that dimension division can ensure that the differences of tasks on this dimension are the greatest. A dimension with a large variance indicates that the resource requirements of tasks on this dimension are significantly different. Division through this dimension can distinguish tasks to the greatest extent and avoid overlapping or proximity of multiple tasks on this dimension. Selecting the median as the node division point can divide the tasks in the current node into two, effectively avoiding too many tasks from gathering in one side of the child node, thereby preventing the tree structure from being too skewed or too deep.

[0086] S3, setting pruning conditions through a comprehensive evaluation function, pruning the multi-level index structure, generating and executing a task scheduling scheme according to the pruned multi-level index structure, and obtaining an efficient task scheduling method;

[0087] Furthermore, a comprehensive evaluation function is defined to calculate the comprehensive score S of the divided task feature vector i ;

[0088] Specifically, the comprehensive evaluation function is expressed as:

[0089] S i =Φ E ×f E (E i )+Φ P ×f P (P i )+Φ R ×f R (R i )+Φ D ×f D (D i )

[0090] Among them, Φ E , Φ P , Φ R , Φ D are respectively represented as the weight coefficients of each task attribute, and satisfy Φ E +Φ P +Φ R +Φ D =1; f E 、f P 、f R 、f D It is expressed as the dimension of each task attribute;

[0091] It should be noted that the weight coefficient can be adjusted according to the total number of nodes in the current multi-level index structure;

[0092] Furthermore, we set the pruning threshold Θ, when the comprehensive score S i When it is less than Θ, it is considered that the task corresponding to the task feature vector after division is N Under current conditions, priority P i Too low, the resource demand vector R in the original task feature vector i The nodes represented by are pruned until the child nodes become leaf nodes;

[0093] It should be noted that the resource demand vector R in the original task feature vector i Pruning the nodes represented is equivalent to removing the related nodes of low-priority tasks from the index structure, which can reduce the system's search for unnecessary nodes in the subsequent scheduling process, thereby reducing the search space for system scheduling and improving scheduling efficiency.

[0094] Furthermore, the pruned multi-level index structure is used to re-assign tasks. NSort (i.e. change the original priority P i Rearrange the task set T so that only high-priority tasks are included in the task set T. Then, sort the tasks in descending order. N Perform scheduling and check task dependencies during scheduling i , if the task dependency D i If there is an unfinished scheduled task, skip the task and the unfinished scheduled task, schedule the next task, and record the actual time of the scheduled task in the current process. Estimated execution time i Compare them. If they are the same, do not update the execution time estimate E. i If they are different, the actual time of the scheduled task in the current process is used. Replace the execution time estimate E i ;

[0095] It should be noted that, since the pruning operation has filtered out low-priority tasks, it can ensure that high-priority tasks are executed first when the system has limited resources.

[0096] Furthermore, this embodiment also provides an efficient task scheduling system based on data enumeration structure optimization, including:

[0097] The task information collection and preprocessing module is configured to collect task information, calculate distribution characteristics in the task information, classify the distribution characteristics, and use different standardization methods for the classified distribution characteristics to obtain standardized task information;

[0098] The task information feature extraction and index structure construction module is configured to obtain a task feature vector according to the standardized task information, create a multi-level index structure, divide the task feature vector, and calculate the corresponding comprehensive evaluation function through the divided task feature vector;

[0099] The task scheduling and scheme generation module is configured to set pruning conditions through the comprehensive evaluation function, prune the multi-level index structure, generate and execute a task scheduling scheme according to the pruned multi-level index structure, and obtain an efficient task scheduling method.

[0100] This embodiment further provides a computer device, which is applicable to the case of an efficient task scheduling method based on data enumeration structure optimization, including:

[0101] A memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the efficient task scheduling method based on data enumeration structure optimization as proposed in the above embodiment.

[0102] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0103] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the efficient task scheduling method based on data enumeration structure optimization proposed in the above embodiment is implemented.

[0104] The storage medium proposed in this embodiment and the data storage method proposed in the above embodiment belong to the same inventive concept. The technical details not fully described in this embodiment can be found in the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0105] Example 2

[0106] Reference Figure 3 and Figure 4 , which is the second embodiment of the present invention, and this embodiment provides an efficient task scheduling method based on data enumeration structure optimization, including: In order to verify the effectiveness of the solution of the present invention, a group of experiments were carried out in a distributed computing environment, and the experimental environment used the following hardware and software configurations:

[0107] Hardware environment: The experimental servers use two Dell PowerEdge R740 servers with the same configuration, equipped with Intel Xeon Silver 4210 processors, 64GB DDR4 memory, and each server has two 1TB SSD storages; Software environment: The operating system is Ubuntu 20.04LTS, and the task scheduling management system is Apache Hadoop YARN. At the same time, two different task scheduling algorithms (the traditional scheduling algorithm and the optimized scheduling method of the present invention) are used in the experiment for comparative testing;

[0108] The task set in the experiment includes three types of computing tasks, namely CPU-intensive, IO-intensive and mixed tasks, with a total of 200 tasks; each task has four attributes: execution time estimate (unit: second), priority, resource requirement vector (CPU core number, memory requirement) and task dependency (a collection of adjacent or non-adjacent tasks); the attributes of 10 tasks are randomly recorded under two task scheduling algorithms, and the selected task attributes are input into the experiment to obtain the best performance. Figure 3 As you can see, the left y-axis (CPU utilization) and the right y-axis (memory utilization) respectively represent the utilization of these two resources by each task during execution; each task corresponds to two lines, one showing CPU utilization and the other showing memory utilization; by comparing the CPU and memory utilization of each task, it is possible to intelligently schedule tasks based on the identification results of task attributes to match them with the allocation strategy of system resources, thereby optimizing system performance; and refer to Figure 4 Each axis of the radar chart represents the estimated execution time and the actual execution time of a task. By comparing whether the two lines (estimated execution time and actual execution time) overlap, the accuracy of task scheduling can be intuitively seen. The figure shows that the actual execution time of almost all tasks overlaps with the estimated execution time line, indicating that the scheduling scheme of the present invention can efficiently control the task execution time and make the actual execution time consistent with the estimate;

[0109] In summary, the solution of the present invention can achieve the expected effect and has a good application prospect.

[0110] Those skilled in the art will appreciate that the embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program codes. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0111] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0112] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0113] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0114] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0115] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. An efficient task scheduling method based on data enumeration structure optimization, characterized in that: include: Collecting task information, calculating distribution features in the task information, classifying the distribution features, and using different standardization methods for the classified distribution features to obtain standardized task information; According to the standardized task information, a task feature vector is obtained, a multi-level index structure is created, the task feature vector is divided, and the corresponding comprehensive evaluation function is calculated through the divided task feature vector; The pruning conditions are set through the comprehensive evaluation function, the multi-level index structure is pruned, and a task scheduling scheme is generated and executed according to the pruned multi-level index structure to obtain an efficient task scheduling method.

2. The efficient task scheduling method based on data enumeration structure optimization according to claim 1 is characterized in that: The collecting task information includes: The collected task information includes a task set T to be scheduled, and the task set T includes multiple tasks, each task N Contains four task attributes, N represents the total number of tasks; The four task attributes are: execution time estimate E i , Priority P i , resource demand vector R i And task dependencies D i .

3. The efficient task scheduling method based on data enumeration structure optimization as claimed in claim 2, characterized in that: Calculating the distribution features in the task information, classifying the distribution features, and using different standardization methods for the classified distribution features to obtain standardized task information, including: Calculate the mean of the task attributes in each task, obtain the standard deviation of each task from the mean of the task attributes, and use the standard deviation as the distribution feature corresponding to each task; A range value of the distribution feature between each task is set, and the distribution feature is divided into multiple intervals according to the range value, and each interval adopts a different standardization rule.

4. The efficient task scheduling method based on data enumeration structure optimization as claimed in claim 3 is characterized in that: Each interval uses different normalization rules, including: If there are no outliers in the current interval range and the current interval range is evenly distributed, linear normalization is used; If there are no outliers in the current interval range value, and the current interval range value belongs to a positively skewed distribution, logarithmic standardization is used; If there are no outliers in the current interval range value, and the current interval range value belongs to a negatively skewed distribution, exponential normalization is used; If there are outliers in the current interval range and they do not meet the above interval range values, quantile normalization is used; If there are outliers in the current interval range, the range value of the distribution feature between each task is reset.

5. The efficient task scheduling method based on data enumeration structure optimization as claimed in claim 4 is characterized in that: According to the standardized task information, a task feature vector is obtained, a multi-level index structure is created, and the task feature vector is divided, including: Extract each task attribute from the standardized task information and combine them into a task feature vector V i , the task feature vector V i Represented as the root node of a multi-level index structure; The task feature vector V i The resource demand vector R in i As a node of a multi-level index structure, each node contains at least one child node, and the child node represents a resource demand vector R i The resource type in the node selects a unique dimension for each node to divide; In the selection dimension, the resource type of the current node is in the resource demand vector R i The data is divided into the dimension with the largest variance; In terms of partitioning, the median of the node is selected as the partition point to divide the tasks in the node into left and right sub-nodes.

6. The efficient task scheduling method based on data enumeration structure optimization according to claim 5 is characterized in that: Calculating the corresponding comprehensive evaluation function through the divided task feature vector, setting the pruning condition through the comprehensive evaluation function, and pruning the multi-level index structure, including: Define a comprehensive evaluation function and calculate the comprehensive score S of the divided task feature vector i ; Set the pruning threshold Θ, when the comprehensive score S i When it is less than Θ, it is considered that the task corresponding to the task feature vector after division is N Under current conditions, priority P i Too low, the resource demand vector R in the original task feature vector i The nodes represented by are pruned until the child nodes become leaf nodes.

7. The efficient task scheduling method based on data enumeration structure optimization according to claim 6 is characterized in that: Generate and execute a task scheduling plan based on the pruned multi-level index structure, including: Re-assign tasks through the pruned multi-level index structure N Sort the tasks in order from high to low. N Perform scheduling and check task dependencies during scheduling i , if the task dependency D i If there is an unfinished scheduled task, skip the task and the unfinished scheduled task, schedule the next task, and record the actual time of the scheduled task in the current process. Estimated execution time i Compare, if the same, do not update the execution time estimate E i If they are different, the actual time of the scheduled task in the current process is used. Replace the execution time estimate E i .

8. An efficient task scheduling system based on data enumeration structure optimization, based on the efficient task scheduling method based on data enumeration structure optimization according to any one of claims 1 to 7, characterized in that: include: The task information collection and preprocessing module is configured to collect task information, calculate distribution characteristics in the task information, classify the distribution characteristics, and use different standardization methods for the classified distribution characteristics to obtain standardized task information; The task information feature extraction and index structure construction module is configured to obtain a task feature vector according to the standardized task information, create a multi-level index structure, divide the task feature vector, and calculate the corresponding comprehensive evaluation function through the divided task feature vector; The task scheduling and scheme generation module is configured to set pruning conditions through the comprehensive evaluation function, prune the multi-level index structure, generate and execute a task scheduling scheme according to the pruned multi-level index structure, and obtain an efficient task scheduling method.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images