A task distribution method and device based on Manage and Worker

Abstract

The present invention provides a task distribution method and device based on Manage and Worker, which relate to the technical field of electronic digital data processing. A genetic algorithm is used to optimize the dynamic allocation of tasks to nodes, with the goal of minimizing the total task execution time and data transmission time. A real-time monitoring and feedback adjustment mechanism is deployed in a Docker Swarm environment, and a scheduling strategy can be dynamically adjusted according to actual feedback from system operation. The scheduling result can be further optimized by fine-tuning an index, so that the system always maintains the best operating efficiency when facing constantly changing workloads and resource states. A graph theory model is introduced to accurately analyze and optimize the dependencies between tasks, so that tasks with close dependencies can be scheduled to nodes with similar physical locations for execution, thereby significantly reducing communication delays and data transmission time.

Description

Technical Field

[0001] The present invention relates to the technical field of electronic digital data processing, and in particular to a task distribution method and device based on Manage and Worker. Background Art

[0002] In modern computing environments, especially when using containerization technologies such as Docker Swarm clusters, task distribution and scheduling optimization are particularly important. In this environment, system scalability, resource utilization, and task execution efficiency are key indicators of technical success. Traditional task distribution methods usually rely on static resource allocation strategies, which often fail to achieve optimal performance under dynamically changing computing requirements and resource states. In addition, existing technologies often show limitations in handling complex task dependencies, dynamic resource configuration, and execution efficiency optimization, especially in high-concurrency and high-load operating environments.

[0003] In the prior art, the publication number CN111506394B discloses a DockerSwarm container scheduling method and scheduling system, which comprehensively measures the performance of the container node load status through five indicators including CPU, memory, I / O load, network bandwidth and disk space, and is closer to the actual container task deployment situation. The threshold-based dual-strategy hybrid frog leaping algorithm is applied to determine the scheduling scheme that makes the node load most balanced, and different local search strategies are selected according to the threshold of the subset, and the learning process between groups is added to expand the search range of the subgroup and the search set of the entire set, so as to improve the local search accuracy and avoid falling into the local optimum; the parallel deployment idea is adopted, which can effectively reduce the resource fragmentation generated when the container task is deployed and maximize the utilization of server resources;

[0004] Existing deficiencies: The existing technology lacks flexibility and adaptability in dealing with complex dependencies between tasks, and cannot effectively capture and use historical execution data to optimize resource allocation. The existing Docker Swarm scheduling algorithm mainly schedules tasks based on the resource status of the node (such as CPU load, memory usage, etc.) without fully considering the dependencies between tasks; and the existing system often fails to make full use of the historical data of task execution to predict resource requirements;

[0005] How to more accurately predict the resource requirements and execution time of tasks by analyzing historical execution efficiency, introduce priority weight calculation based on multiple factors, such as urgency, resource requirements and historical execution efficiency, and balance the influence of these factors by adjusting coefficients to improve resource utilization efficiency and task execution response speed, thereby reducing communication delays and data transmission time is the direction that needs to be improved at present;

[0006] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0007] The purpose of the present invention is to provide a task distribution method and device based on Manage and Worker to solve the problems raised in the above background technology.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] A task distribution method based on Manager and Worker, the specific steps include:

[0010] Step S1: collect historical execution data of containerized tasks, dependencies between tasks, size of shared data, and access frequency information, and analyze and process them. Use these data to construct a task dependency graph, which is represented by a directed graph G = (V, E), where each vertex v∈V represents a task, and each edge e = (u, v)∈E represents that task u depends on task v. The expression formula is as follows:

[0011] G=(V,E),where V=v 1 ,v 2 ,…,vn,E=(vi,vj)|vi depends on vj

[0012] Among them, v 1 ,v 2 ,…,vn are nodes in the graph, each node represents a task, and each edge (vi,vj) indicates that task vi depends on task vj, that is, the completion or output of task vj is necessary for the execution of task vi;

[0013] Step S2: After obtaining the task dependency graph, use topological sorting to determine the execution order of the tasks, and introduce the priority weight w(v) based on task urgency, resource requirements, and historical execution efficiency. The expression formula is as follows:

[0014] w(v)=α×urgency+β×resource demand+γ×historical efficiency

[0015] Among them, α, β, and γ are adjustment coefficients used to balance the influence of different factors. Task urgency, resource demand, and historical efficiency are represented by urgency, resource demand, and historical efficiency, respectively.

[0016] Step S3, obtain the task execution sorting result in step S2, apply the genetic algorithm, dynamically allocate the task to the most suitable node according to the priority weight of the task and the real-time status of the node. The goal of the algorithm is to minimize the total task execution time and data transmission time, while maximizing resource utilization. The expression formula is as follows:

[0017]

[0018] Where texec(v) is the execution time of task v, and ttrans(v,E(v)) is the data transmission time between task v and its dependent task set E(v);

[0019] Step S4, deploying the scheduling strategy in step S3 in the Docker Swarm environment, monitoring its effect, and then collecting the quantifiable parameter set A in the monitoring effect, analyzing and processing the quantifiable parameter set A, generating a calibration index, and generating a first feedback adjustment strategy based on the calibration index, the first feedback adjustment strategy can adjust the α, β, γ parameters according to the system operation feedback, and preliminarily optimize the scheduling result;

[0020] Step 5: Obtain a quantifiable parameter set B of task urgency, resource requirements, and historical execution efficiency, analyze and process the quantifiable parameter set B, generate a fine-tuning index, and generate a second feedback adjustment strategy based on the fine-tuning index. The second feedback adjustment strategy can fine-tune the α, β, and γ parameters based on the first feedback adjustment strategy to form a final optimized scheduling result.

[0021] Further, the first feedback adjustment strategy includes dividing the value range of the calibration index I (0, 1) into three main intervals, each interval corresponding to a different system performance state and a corresponding adjustment strategy;

[0022] Low performance interval I∈(0,0.3), where the system performance is seriously insufficient and requires substantial adjustments;

[0023] The feedback adjustment strategy is to increase α to increase the weight of task urgency by 20%, decrease β by 10% to reduce the impact of resource demand, and increase γ by 10% to take more account of historical execution efficiency;

[0024] Medium performance interval I∈[0.3,0.7), where the system performance basically meets expectations, but there is still room for improvement;

[0025] The feedback adjustment strategy is to slightly increase α by 5%, keep β unchanged, and slightly reduce γ by 5%;

[0026] High performance interval I∈[0.7,1), where the system performance is superior and adjustments are aimed at subtle optimization;

[0027] The feedback adjustment strategy is to reduce α by 5%, keep β unchanged, and increase γ by 5%;

[0028] When I increases, it indicates that system performance is improving. At this time, reducing the response to emergency tasks means reducing the α value, while increasing the consideration of historical efficiency means increasing the γ value.

[0029] When I decreases, it means that the system encounters a bottleneck or efficiency problem. At this time, it is necessary to increase the weight of urgent tasks. At this time, α increases to quickly solve problems that may cause system congestion.

[0030] Furthermore, the acquisition of a quantifiable parameter set B of task urgency, resource requirements, and historical execution efficiency, and analysis and processing of the quantifiable parameter set B to generate a fine-tuning index specifically includes the following contents:

[0031] The calculated data of task urgency Urgency, resource demand ResourceDemand, historical execution efficiency HistoricalEfficiency, node performance change JDXn, and resource application and release frequency ZYSf are uniformly normalized to limit their values ​​to the same range, and the fine-tuning index is defined as Wt. The calculation formula is as follows:

[0032]

[0033] Among them, Urgency(i) represents the urgency of task i. The higher the value, the more urgent the task.

[0034] ResourceDemand(i) represents the resource demand of task i. The larger the value, the more resources are needed.

[0035] Historical Efficiency (i) is the historical execution efficiency. The higher the value, the better the execution efficiency of past tasks. JDXn (n) refers to the performance change of node n at time t. This parameter reflects the real-time change of node performance. ZYSf represents the frequency of resource application and release. The higher the frequency, the more frequent the resource changes.

[0036] The value range of the Wt formula is set to (0,10). When Wt is close to 0, it indicates that the current resource allocation and task urgency are relatively balanced and do not require significant adjustment.

[0037] When Wt is close to 10, it indicates that the task scheduling strategy needs to be significantly adjusted, which may be due to extremely high resource requirements or extremely urgent tasks;

[0038] The value range of Wt (0, 10) is divided into four intervals, each interval corresponds to different adjustment requirements, and the adjustment of each interval is made based on the first feedback adjustment strategy, while ensuring that α+β+γ=1:

[0039] In the low fine-tuning interval Wt∈(0,2.5), the adjustment strategy is to slightly increase α by 2%, reduce β by (1+△h)% of the current value, and increase γ by 1%;

[0040] In the medium fine-tuning interval Wt∈[2.5,5), the adjustment strategy is to increase the current value of α by 5%, keep β unchanged, and reduce the current value of γ by (3+△h)%. At this time, the system performance is basically in line with expectations. The adjustment aims to slightly increase the processing priority of emergency tasks, and the adjustment range is within 15%;

[0041] In the high fine-tuning interval Wt∈[5,7.5), the adjustment strategy is to increase the current value of α by (10+△h)%, reduce β by 5%, and keep γ unchanged; at this time, the system performance needs to be adjusted significantly, and the adjustment range is within 20%, especially in resource allocation and task urgency;

[0042] In the extremely high fine-tuning interval Wt∈[7.5,10), the adjustment strategy is to significantly increase α by 20%, reduce β by 10%, and increase γ by (10+△h)% of the current value;

[0043] At this point, the system performance is seriously insufficient and needs to be adjusted significantly to cope with urgent and high-resource-demand tasks, with the adjustment range within 30%;

[0044] αnew=α·(1+k α ·Wt)

[0045] βnew=β·(1-kβ·Wt)

[0046] γnew=γ·(1+k γ ·Wt)

[0047] Among them, k α ,kβ and k γ It is an adjustment coefficient preset according to different intervals of the Wt value range.

[0048] A task distribution device based on Manage and Worker, the device is used to execute the task distribution method based on Manage and Worker, comprising:

[0049] Task dependency graph construction module: It is used to collect historical execution data of containerized tasks, dependencies between tasks, size of shared data, and access frequency information, and analyze and process them. The task dependency graph is constructed based on these data and represented by a directed graph G = (V, E), where each vertex v∈V represents a task, and each edge e = (u, v)∈E represents that task u depends on task v. The expression formula is as follows:

[0050] G=(V,E),where V=v 1 ,v 2 ,…,vn,E=(vi,vj)|vi depends on vj

[0051] Among them, v 1 ,v 2 ,…,vn are nodes in the graph, each node represents a task, and each edge (vi,vj) indicates that task vi depends on task vj, that is, the completion or output of task vj is necessary for the execution of task vi;

[0052] Priority calculation module: It is used to obtain the task dependency graph and use topological sorting to determine the execution order of tasks. It also introduces the priority weight w(v) based on task urgency, resource requirements, and historical execution efficiency. The expression formula is as follows:

[0053] w(v)=α×urgency+β×resource demand+γ×historical efficiency

[0054] Among them, α, β, and γ are adjustment coefficients used to balance the influence of different factors. Task urgency, resource demand, and historical efficiency are represented by urgency, resource demand, and historical efficiency, respectively.

[0055] Scheduling optimization module: used to obtain the task execution sorting results in step S2, apply genetic algorithm, and dynamically allocate tasks to the most suitable nodes according to the priority weight of the task and the real-time status of the node. The goal of the algorithm is to minimize the total task execution time and data transmission time, while maximizing resource utilization. The expression formula is as follows:

[0056]

[0057] Where texec(v) is the execution time of task v, and ttrans(v,E(v)) is the data transmission time between task v and its dependent task set E(v);

[0058] Feedback adjustment module: used to deploy scheduling strategies in the Docker Swarm environment, monitor their effects, and then collect quantifiable parameter set A in the monitoring effects, analyze and process the quantifiable parameter set A, generate a calibration index, and generate a first feedback adjustment strategy based on the calibration index. The first feedback adjustment strategy can adjust the α, β, and γ parameters according to the system operation feedback to preliminarily optimize the scheduling results;

[0059] Fine-tuning module: used to obtain a quantifiable parameter set B of task urgency, resource requirements and historical execution efficiency, analyze and process the quantifiable parameter set B, generate a fine-tuning index, and generate a second feedback adjustment strategy based on the fine-tuning index. The second feedback adjustment strategy can fine-tune the α, β, and γ parameters based on the first feedback adjustment strategy to form the final optimized scheduling result.

[0060] Compared with the prior art, the present invention has the following beneficial effects: the present invention adopts a genetic algorithm to optimize the dynamic allocation of tasks to nodes, with the goal of minimizing the total task execution time and data transmission time while maximizing resource utilization. By deploying a real-time monitoring and feedback adjustment mechanism in a Docker Swarm environment, the scheduling strategy can be dynamically adjusted according to the actual feedback of the system operation, and the scheduling result can be further optimized by fine-tuning the index, so that the system always maintains the optimal operating efficiency in the face of constantly changing workloads and resource states, thereby solving the scheduling deficiencies of the prior art in a dynamic environment and providing an innovative solution for complex containerized application scenarios.

[0061] A graph theory model is introduced to accurately analyze and optimize the dependencies between tasks, so that tasks with close dependencies can be scheduled to be executed on nodes with similar physical locations, thereby significantly reducing communication delays and data transmission time. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is a schematic diagram of the overall method flow of the present invention;

[0063] Figure 2 It is a system module block diagram of the present invention. DETAILED DESCRIPTION

[0064] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments.

[0065] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0066] Embodiment 1:

[0067] See also Figure 1 , the present invention provides a technical solution:

[0068] A task distribution method based on Manager and Worker, wherein the Manager and Worker are set in a Docker Swarm cluster, and the specific steps include:

[0069] Step S1: collect historical execution data of containerized tasks, dependencies between tasks, size of shared data, and access frequency information, and analyze and process them. Use these data to construct a task dependency graph, which is represented by a directed graph G = (V, E), where each vertex v∈V represents a task, and each edge e = (u, v)∈E represents that task u depends on task v. The expression formula is as follows:

[0070] G=(V,E),where V=v 1 ,v 2 ,…,vn,E=(vi,vj)|vi depends on vj

[0071] Among them, v 1 ,v 2 ,…,vn are nodes in the graph, each node represents a task, and each edge (vi,vj) indicates that task vi depends on task vj, that is, the completion or output of task vj is necessary for the execution of task vi;

[0072] Step S2: After obtaining the task dependency graph, use topological sorting to determine the execution order of the tasks, and introduce the priority weight w(v) based on task urgency, resource requirements, and historical execution efficiency. The expression formula is as follows:

[0073] w(v)=α×urgency+β×resource demand+γ×historical efficiency

[0074] Among them, α, β, and γ are adjustment coefficients used to balance the influence of different factors. Task urgency, resource demand, and historical efficiency are represented by urgency, resource demand, and historical efficiency, respectively.

[0075] Step S3, obtain the task execution sorting result in step S2, apply the genetic algorithm, dynamically allocate the task to the most suitable node according to the priority weight of the task and the real-time status of the node. The goal of the algorithm is to minimize the total task execution time and data transmission time, while maximizing resource utilization. The expression formula is as follows:

[0076]

[0077] Where texec(v) is the execution time of task v, and ttrans(v,E(v)) is the data transmission time between task v and its dependent task set E(v);

[0078] Step S4, deploying the scheduling strategy in step S3 in the Docker Swarm environment, monitoring its effect, and then collecting the quantifiable parameter set A in the monitoring effect, analyzing and processing the quantifiable parameter set A, generating a calibration index, and generating a first feedback adjustment strategy based on the calibration index, the first feedback adjustment strategy can adjust the α, β, and γ parameters of the priority weight w(v) according to the system operation feedback, and preliminarily optimize the scheduling result;

[0079] Step 5: Obtain a quantifiable parameter set B of task urgency, resource requirements, and historical execution efficiency, analyze and process the quantifiable parameter set B, generate a fine-tuning index, and generate a second feedback adjustment strategy based on the fine-tuning index. The second feedback adjustment strategy can fine-tune the α, β, and γ parameters of the priority weight w(v) based on the first feedback adjustment strategy to form a final optimized scheduling result.

[0080] Embodiment 2:

[0081] Further explanation based on the first embodiment, the Manage, Worker and task dependency graph specifically includes the following contents:

[0082] Manager means that in a Docker Swarm cluster, the Manager is responsible for managing the cluster status, scheduling services, and performing task allocation. The Manager is also responsible for handling the cluster's network configuration and service discovery mechanism;

[0083] Workers are nodes that perform actual containerized tasks. They receive instructions from the Manager and run specific services or tasks based on these instructions.

[0084] The task dependency graph is a directed graph used to represent the dependency relationship between tasks. The nodes in the graph represent the tasks, and the edges represent that a task must be executed after another task because it depends on the output or completion status of the former.

[0085] The beneficial effect is that by analyzing historical data, the performance and resource requirements of each task can be predicted, so that more reasonable decisions can be made when scheduling tasks, for example, those key dependent tasks can be scheduled first to avoid delayed delivery and waste of resources;

[0086] The purpose is that the use of task dependency graph enables containerized applications to optimize their execution order, reduce waiting time, and improve overall execution efficiency while ensuring data consistency and task dependency;

[0087] In summary, the combination of traditional task scheduling methods and modern containerization technology, especially in the dynamic and scalable DockerSwarm environment, can dynamically adjust resource allocation and task queues to adapt to system load changes and real-time performance feedback by analyzing task execution data and dependencies in real time. In addition, through detailed data analysis, potential bottlenecks can be foreseen and resolved, thereby maximizing resource utilization without sacrificing service quality.

[0088] After obtaining the task dependency graph, the execution order of the tasks is determined using topological sorting, and a priority weight w(v) based on task urgency, resource requirements, and historical execution efficiency is introduced, specifically including the following:

[0089] The topological sort is an algorithm for sorting directed graphs, so that for any directed edge from vertex u to vertex v, u appears before v in the sort. Using topological sort in task scheduling can ensure that the execution order of tasks complies with the dependency relationship;

[0090] Topological sorting specifically includes:

[0091] Use Kahn algorithm for topological sorting. The core of this algorithm is to continuously remove vertices with in-degree 0, that is, tasks without dependencies, and update the in-degree of related vertices.

[0092] The initialization step is to create a queue Q to store all vertices with in-degree 0;

[0093] The steps for processing vertices in the queue are to take a vertex u from Q each time, output it to the sorted result, remove all edges starting from u, and add new vertices with in-degree 0 to Q;

[0094] Repeat steps until Q is empty;

[0095] Combined with the priority weight to adjust the sorting, based on the sorting output, the task groups with the same in-degree, that is, tasks that can be processed in parallel, are sorted using the priority weight w(v) to ensure that tasks with higher weights are executed first;

[0096] Execution scheduling: Execute tasks in order according to the adjusted topological sorting results. In actual scheduling, real-time adjustments are considered. If a task cannot be started on time due to insufficient resources, the execution order of the remaining tasks is readjusted.

[0097] Optimization and feedback: After the task is executed, the execution data is collected, including the actual start time, end time, and resource usage, which is used to update the historical execution efficiency of the task. These data will be fed back into the priority weight w(v) to optimize the subsequent weight calculation;

[0098] The priority weight w(v) is a calculation model for evaluating the priority of task execution, where v represents a specific task node. The weight calculation takes into account the urgency, resource requirements and historical execution efficiency of the task, and is a comprehensive evaluation indicator.

[0099] The adjustment coefficients α, β, and γ are used to adjust the influence of different factors in the priority weight formula, so that task scheduling can be flexibly adjusted according to actual conditions;

[0100] The beneficial effects are that topological sorting ensures the logical order of task execution and avoids execution failures caused by unsatisfied dependencies. The introduction of priority weight w(v) makes task scheduling more intelligent and accurate, and can reasonably sort tasks according to their urgency, resource occupancy, and historical performance, effectively optimizing resource allocation and reducing waiting time.

[0101] The purpose is that this step is applicable to resource-intensive and time-sensitive application scenarios, such as cloud computing and data center management, and can significantly improve the efficiency of task processing and the overall performance of the system;

[0102] The complex task dependencies and resource management requirements are converted into practical and operational task execution strategies through advanced mathematical models and algorithms. In particular, the introduction of priority weight w(v) reflects the innovative application in real-world complexity management. By adjusting the parameters α, β, and γ, it can not only adapt to different operating environments and business needs, but also reflect environmental changes and task status updates in real time to achieve dynamic adjustment.

[0103] Embodiment three:

[0104] Further explanation is given on the basis of the second embodiment, the task execution sorting result in step S2 is obtained, and a genetic algorithm is applied to dynamically allocate tasks to the most suitable nodes according to the priority weight of the tasks and the real-time status of the nodes. The goal of the algorithm is to minimize the total task execution time and data transmission time, while maximizing resource utilization, which specifically includes the following contents:

[0105] The genetic algorithm is a heuristic search algorithm used to solve optimization and search problems, simulating the process of natural selection, that is, the principle of "survival of the fittest", and evolving the optimal solution through iteration, which is used to solve complex optimization problems;

[0106] The genetic algorithm includes, in sequence, initializing population, evaluating fitness, selection, crossover and mutation, new generation population, repetition and result extraction;

[0107] Initializing the population is to randomly generate an initial population as a starting point;

[0108] Fitness assessment is to calculate the fitness of each individual and determine its probability of survival and reproduction;

[0109] Selection is based on individual fitness, using a roulette wheel method to select individuals for reproduction;

[0110] Crossover and mutation are to generate new individuals through crossover and mutation to increase the diversity of the population;

[0111] The new generation of population is formed by combining some old individuals with newly generated individuals;

[0112] Repeat steps 2-5 until the termination condition is met;

[0113] The result is extracted by selecting the individual with the highest fitness as the final solution;

[0114] The genetic algorithm specifically comprises the following steps:

[0115] Initialize the population: In the genetic algorithm, you first need to generate an initial population, which consists of multiple individuals. Each individual represents a possible task distribution scheme. Here, an individual is regarded as a vector, in which each element represents the distribution status of a task on a specific node.

[0116] The formula is Individual = [n 1 ,n 2 ,...,ni,...,nk], where ni represents the node to which the i-th task is assigned;

[0117] By initializing the population, selecting and optimizing among multiple possible solutions, this is the basis of the genetic algorithm's global search capability and helps find the best task distribution strategy;

[0118] Evaluate fitness: The fitness of each individual determines its survival probability in future reproduction. The fitness function is calculated based on the total execution time and data transmission time of the task. The goal is to minimize the sum of these two parameters; the formula is as follows:

[0119]

[0120] Where texec(v) is the execution time of task v, and ttrans(v,E(v)) is the data transmission time between task v and its dependent task set E(v);

[0121] Choosing the fitness function that minimizes the sum of execution and transmission time can ensure fast task processing while optimizing network resource utilization, which is crucial in real-time computing environments;

[0122] Selection: Roulette wheel selection is used to select individuals from the current population for mating. This method depends on the fitness of the individual. Individuals with high fitness are more likely to be selected.

[0123] The roulette wheel method prefers to select individuals with high fitness while maintaining population diversity, which helps to optimize the quality of the solution;

[0124] Crossover and mutation:

[0125] For crossover, the selected individuals produce offspring through the single-point crossover method, where a random point is selected and the gene sequences of the two parent individuals after that point are exchanged;

[0126] For mutation, some genes of offspring individuals are modified with a small probability to introduce new genetic information. The formula is:

[0127] Crossover(P1,P2)→C1,C2

[0128] Mutation(C1)→C1′

[0129] Crossover and mutation are key mechanisms for introducing new genetic variations in genetic algorithms, which help the algorithm to jump out of the local optimal solution and explore a wider solution space;

[0130] New generation population: Use the best individuals of the previous generation and the newly generated offspring individuals to form a new generation population. This step includes a certain proportion of elite retention strategy, that is, directly retaining the best individuals in the previous generation to the next generation;

[0131] Doing so ensures that the excellent genes in the population can be retained, while introducing new features through new generations of crossover and mutation, balancing the needs of exploration and development, which is the key to achieving fast convergence of genetic algorithms;

[0132] Algorithm termination condition: Set the termination condition of the algorithm, which is when the preset maximum number of iterations is reached, or the fitness of the best individual in the population has not improved in several consecutive generations; the setting of the termination condition ensures that the algorithm can stop after reaching a certain optimization level to avoid ineffective consumption of computing resources. In addition, monitoring the change of fitness can help determine whether the algorithm has converged, that is, a sufficiently good solution has been found;

[0133] Result extraction and application,Once the genetic algorithm terminates, the individual with the highest fitness,is selected from the final population as the optimal solution, which,represents the best task allocation strategy under given conditions;

[0134] This step is to convert the results of algorithm processing into an actual executable task allocation plan, which ensures that the computing resources used can be converted into actual benefits, such as reducing costs and improving efficiency;

[0135] The priority weight is a weight assigned to each task in task scheduling based on urgency, resource requirements and historical execution efficiency to determine its execution priority;

[0136] The real-time status refers to the current status of each execution node when the task is distributed, including available resources, network bandwidth, and CPU usage;

[0137] The total task execution time and data transmission time is one of the optimization goals of the genetic algorithm, that is, to minimize the execution time of each task and the data transmission time between each task and its dependent tasks by reasonably allocating tasks;

[0138] Beneficial effects: The use of genetic algorithms can effectively find the optimal task allocation scheme, which can balance execution efficiency and resource consumption, thereby improving the performance of the overall system. In addition, the algorithm's adaptability enables it to adjust according to real-time changes in network and node status to maintain efficient operation;

[0139] Application: Applicable to environments that require highly dynamic resource management, such as cloud data centers and large-scale parallel computing systems. In these environments, task loads and network conditions change frequently, and traditional static allocation methods are difficult to cope with;

[0140] Introducing genetic algorithms into the task distribution process makes task allocation not only rely on static priority determination, but also continuously optimizes the solution by simulating the dynamic evolution process of natural selection. Genetic algorithms can effectively explore and quickly converge to the approximate optimal solution in the global search space through population, selection, crossover and mutation operations, which significantly improves the flexibility and response speed of task distribution.

[0141] Embodiment 4:

[0142] Further explained on the basis of Example 3, the quantifiable parameter set A includes task completion time, data transmission time, resource utilization, task failure rate and task queuing time, and the task completion time is marked as texec, the data transmission time is marked as ttrans, the resource utilization is marked as ZYl, the task failure rate is marked as RWSb, and the task queuing time is marked as PDt;

[0143] The task completion time texec represents the time required for each task from start to completion;

[0144] texec(i)=tend(i)-tstart(i)

[0145] Where tstart(i) represents the start time of task i; tend(i) represents the end time of task i;

[0146] The data transmission time ttrans represents the time required for data transmission between tasks due to data dependency;

[0147] ttrans(i,j)=tdatasent(j)-tdatareceived(i)

[0148] Where tdatareceived(i) indicates the time when task i receives the required data; tdatasent(j) indicates the time when task j sends the data;

[0149] The resource utilization rate ZY1 includes CPU utilization rate, memory occupancy rate, and network bandwidth usage, and the calculation formula is as follows:

[0150]

[0151] Among them, utilizationm(t) represents the utilization of the mth resource at time t, including CPU utilization, memory occupancy, and network bandwidth usage;

[0152] The task failure rate RWSb represents the frequency of failure of each task execution;

[0153]

[0154] Where number(i) indicates the number of times task i fails to execute; total executions(i) indicates the total number of times task i is executed;

[0155] The task queuing time PDt represents the time a task spends waiting for resource allocation;

[0156] PDt(i)=tstart(i)-tsubmit(i)

[0157] Where tsubmit(i) indicates the time when task i is submitted to the system; tstart(i) indicates the time when task i starts to execute;

[0158] The quantifiable parameter set B includes task urgency, resource demand, historical execution efficiency, node performance change, resource application and release frequency, and the task urgency is marked as Urgency, the resource demand is marked as ResourceDemand, the historical execution efficiency is marked as HistoricalEfficiency, the node performance change is marked as JDXn, and the resource application and release frequency is marked as ZYSf;

[0159] The task urgency Urgency represents the urgency of the task combined with the deadline and the importance level;

[0160]

[0161] Where tdeadline(i) is the deadline of task i, tcurrent is the current time, importance(i) is the importance level of task i, a pre-set value; a1 and a2 are non-negative weight coefficients used to adjust the impact of deadline urgency and importance level, and a1+a2=1, which is determined by the expert group through data analysis;

[0162] The resource demand ResourceDemand represents the CPU time, memory size, and I / O bandwidth required for the task;

[0163] Resource Demand(i)=b1·cputime(i)+b2·memorysize(i)+b3·iobandwidth(i)

[0164] Among them, cputime(i), memorysize(i), iobandwidth(i) represent the CPU time, memory size and I / O bandwidth required by task i, respectively, and b1, b2 and b3 are the positive weight coefficients corresponding to CPU time, memory size and I / O bandwidth, and b1+b2+b3=1, which is determined by the expert group through data analysis;

[0165] The historical execution efficiency indicates the completion efficiency of similar tasks in history.

[0166]

[0167] Where pasttasks(i) is the set of historical tasks similar to task i; successrate(j) is the success rate of historical task j;

[0168] The node performance change JDXn represents the performance fluctuation data of each processing node, and in this embodiment, it is the change of processing speed;

[0169]

[0170] Among them, performancet(n) is the performance index of node n at time t; T is the time set considered;

[0171] The resource application and release frequency ZYSf represents the frequency at which resources are applied and released, so as to evaluate the efficiency of resource management;

[0172]

[0173] Among them, numrequests+numreleased is the total number of times the resource is requested and released within time Δt.

[0174] Embodiment five:

[0175] Further explanation based on the fourth embodiment, the analysis and processing of the quantifiable parameter set A to generate a calibration index, and the generation of a first feedback adjustment strategy based on the calibration index specifically includes the following contents:

[0176] After obtaining the task completion time texec, data transmission time ttrans, resource utilization ZYl, task failure rate RWSb and task queuing time PDt, normalized dimension operations are performed and analyzed to generate a calibration index. The calibration index is defined as I, and the calculation formula is as follows:

[0177]

[0178] Texec(i) represents the execution time of task i. Shorter execution time has a positive impact on system performance. Ttrans(i,j) measures the efficiency of data transmission between tasks i and j. Shorter transmission time improves system efficiency. ZYl represents the efficiency of system resource utilization. High utilization reflects efficient resource management. RWSb(i) represents the failure rate of task i, which directly affects the stability and reliability of the system. A lower failure rate enhances system performance. PDt(i) represents the time spent by task i waiting for resource allocation. Shorter queuing time improves system response speed. k, d1, d2, d3, d4 and d5 are adjustment coefficients used to adjust the relative importance and influence of each parameter in the formula. η is a positive adjustment constant, which is determined by the expert group experiment, 0.02≤η≤0.32.

[0179] The output value range of formula I is set to (0,1). The closer the value is to 1, the better the system performance is. The closer the value is to 0, the potential efficiency problem or bottleneck exists in the system. By adjusting the values ​​of parameters d1, d2, d3, d4 and d5, the fine control of system performance is achieved, thereby optimizing the task scheduling strategy, as follows;

[0180] In the following content, those with an impact degree of less than half are considered to have a small impact, and those with an impact degree of more than half are considered to have a large impact;

[0181] For the allocation ratio of d1:

[0182] The scenario with a greater impact means that when the task execution time has a greater impact on system performance and has a smaller impact than other factors, the proportion of d1 ranges from 30% to 50%;

[0183] Assume that the average execution time Texec(i) of the task in the current system is 100 seconds, and the target of the system calibration index I is 0.8. If the proportion of d1 is set to 40%, that is, d1 = 0.4, then when the execution time Texec(i) of task i increases from the average value to 120 seconds, the value of the calibration index I will decrease to 0.6, which shows that the increase in task execution time has a negative impact on system performance, and the change of d1 has a significant impact on the calibration index I.

[0184] The scenario with less impact means that when the task execution time has less impact on system performance, or when other factors have a greater impact, the proportion of d1 ranges from 10% to 30%;

[0185] If the execution time Texec(i) of the tasks in the system is evenly distributed and is within 50 seconds, it means that the average execution time is short, while other factors such as data transmission time, resource utilization, failure rate and queuing time have a more significant impact on system performance; if the proportion of d1 is set to 20%, that is, d1 = 0.2, even if the task execution time Texec(i) increases to 70 seconds, the change in the calibration index I is relatively small, only around 0.1, which shows that in this case, the change in task execution time has little impact on system performance, and the weight of d1 is relatively low;

[0186] For the allocation ratio of d2:

[0187] The scenario with a greater impact means that when data transmission time has a greater impact on system performance and has a smaller impact than other factors, the proportion of d2 ranges from 20% to 40%;

[0188] Assuming that the average data transmission time Ttrans(i,j) between tasks in the current system is 20 seconds, and the target of the system calibration index I is 0.8, if the proportion of d2 is set to 30%, that is, d2 = 0.3, then when the transmission time Ttrans(i,j) of a task increases from the average value to 30 seconds, the value of the calibration index I will decrease to the range of 0.45 to 0.58, which shows that the increase in data transmission time has a negative impact on system performance, and the change of d2 has a significant impact on the calibration index I;

[0189] The scenario with less impact means that when the data transmission time has less impact on the system performance, or when the impact of other factors is greater, the proportion of d2 ranges from 10% to 20%;

[0190] If the data transmission time Ttrans(i,j) between tasks in the system changes little and the average transmission time is short, setting it within 10 seconds means that the data transmission time Ttrans(i,j) changes little, and other factors have a more significant impact on system performance. Set the proportion of d2 to 15%, that is, d2 = 0.15. Even if the transmission time Ttrans(i,j) of a task increases to 15 seconds, the change of the calibration index I is relatively small, only around 0.05. This shows that in this case, the change of data transmission time has little impact on system performance, and the weight of d2 is relatively low.

[0191] For the distribution ratio of d3:

[0192] The scenario with a greater impact indicates that when the system resource utilization has a greater impact on system performance and has a smaller impact relative to other factors, the proportion of d3 ranges from 15% to 35%;

[0193] Assuming that the resource utilization ZYl in the current system is 80% on average, and the target of the system calibration index I is 0.8, the ratio of d3 is set to 25%, that is, d3 = 0.25, then when the system resource utilization ZYl increases from the average value to 90%, the value of the calibration index I will decrease to the range of 0.5 to 0.6, which shows that the increase in resource utilization has a negative impact on system performance, and the change of d3 has a significant impact on the calibration index I;

[0194] The scenario with less impact means that when the system resource utilization has less impact on system performance, or when other factors have a greater impact, the proportion of d3 ranges from 5% to 15%; if the resource utilization ZYl in the system has a smaller range of variation and the average utilization is low, such as 60%, and other factors have a more significant impact on system performance, the proportion of d3 is set to 10%, that is, d3 = 0.1. Even if the system resource utilization ZYl increases to 70%, the change in the calibration index I is relatively small, only around 0.05, which shows that in this case, the change in resource utilization has a smaller impact on system performance, and the weight of d3 is relatively low;

[0195] For the allocation ratio of d4:

[0196] The scenario with a greater impact indicates that when the task failure rate has a greater impact on system performance and has a smaller impact relative to other factors, the proportion of d4 ranges from 10% to 30%;

[0197] Assuming that the failure rate RWSb(i) of the tasks in the current system is 5% on average, and the target of the system calibration index I is 0.8, the ratio of d4 is set to 20%, that is, d4 = 0.2. When the failure rate RWSb(i) of task i increases from the average value to 10%, the value of the calibration index I will decrease to the range of 0.6 to 0.67, which shows that the increase in the task failure rate has a negative impact on the system performance, and the change of d4 has a significant impact on the calibration index I.

[0198] The scenario with less impact means that when the task failure rate has less impact on system performance, or when the impact of other factors is greater, the proportion of d4 ranges from 5% to 15%;

[0199] If the failure rate RWSb(i) of the tasks in the system has a small variation range and the average failure rate is low, setting it within 2% means that the variation range of RWSb(i) is small, and other factors have a more significant impact on system performance. The proportion of d4 is set to 10%, that is, d4 = 0.1. Even if the failure rate RWSb(i) of a certain task increases to 3%, the change of the calibration index I is relatively small, only around 0.05. This shows that in this case, the change of the task failure rate has little impact on the system performance, and the weight of d4 is relatively low.

[0200] For the allocation ratio of d5:

[0201] The scenario with a greater impact indicates that when the task queuing time has a greater impact on system performance and has a smaller impact relative to other factors, the proportion of d5 ranges from 5% to 20%;

[0202] Assuming that the average queuing time PDt(i) of tasks in the current system is 10 seconds, and the target of the system calibration index I is 0.7, the proportion of d5 is set to 15%, that is, d5 = 0.15. When the queuing time PDt(i) of task i increases to 15 seconds, the value of the calibration index I will decrease to the range of 0.63 to 0.72, which shows that the increase in task queuing time has a negative impact on system performance, and the change of d5 has a significant impact on the calibration index I.

[0203] The scenario with less impact means that when the task queuing time has less impact on system performance, or when other factors have a greater impact, the proportion of d5 ranges from 2% to 10%;

[0204] The queuing time PDt(i) of the tasks in the system has a small variation range, and the average queuing time is short. For example, within 5 seconds, it means that the variation range of PDt(i) is small, and other factors have a more significant impact on system performance. The proportion of d5 is set to 5%, that is, d5 = 0.05. Even if the queuing time PDt(i) of task i increases to 7 seconds, the change of calibration index I is relatively small, only around 0.02; this shows that in this case, the change of task queuing time has little impact on system performance, and the weight of d5 is relatively low;

[0205] The first feedback adjustment strategy includes dividing the value range of the calibration index I (0, 1) into three main intervals, each interval corresponding to a different system performance state and a corresponding adjustment strategy;

[0206] Low performance interval I∈(0,0.3), where the system performance is seriously insufficient and requires substantial adjustments;

[0207] The feedback adjustment strategy is to increase α to increase the weight of task urgency by 20%, decrease β by 10% to reduce the impact of resource demand, and increase γ by 10% to take more account of historical execution efficiency;

[0208] Increasing α can enhance the impact of task urgency, speed up the response and processing of urgent tasks, and reduce system waiting time and potential congestion;

[0209] Reducing β can reduce the impact of resource requirements on task priorities, contribute to a more balanced resource allocation, and reduce the risk of over-concentration of resources;

[0210] Increasing γ can strengthen the consideration of historical execution efficiency, optimize resource allocation and scheduling decisions, and predict and improve future task execution by using historical data;

[0211] Medium performance interval I∈[0.3,0.7), where the system performance basically meets expectations, but there is still room for improvement;

[0212] The feedback adjustment strategy is to slightly increase α by 5%, keep β unchanged, and slightly reduce γ by 5%;

[0213] The expected effect is to optimize the handling of urgent tasks while maintaining a balance in resource allocation;

[0214] Increasing α can slightly improve the responsiveness to emergency tasks, ensuring that the system can respond quickly to emergencies while maintaining stability;

[0215] Keeping β can maintain the weight of current resource demand unchanged and ensure the continuity and stability of resource allocation;

[0216] Reducing γ can reduce the impact of historical efficiency, encourage the system to explore new task scheduling strategies, and help find a better resource allocation model;

[0217] High performance interval I∈[0.7,1), where the system performance is superior and adjustments are aimed at subtle optimization;

[0218] The feedback adjustment strategy is to reduce α by 5%, keep β unchanged, and increase γ by 5%;

[0219] The expected effect is to reduce over-reaction to urgent tasks and make greater use of historical data to guide resource allocation;

[0220] Reducing α can reduce sensitivity to emergency tasks and prevent uneven resource scheduling caused by over-response;

[0221] Maintaining β can continue to maintain a stable weight of resource demand and ensure the efficiency and fairness of resource allocation;

[0222] Increasing γ can enhance the application of historical execution data and use past successful experiences to optimize current and future task execution strategies;

[0223] The feedback adjustment strategy of each interval directly affects the values ​​of α, β, and γ, thereby adjusting the calculation of the task priority weight w(v). The following is a detailed description of the interactive changes of parameters:

[0224] When I increases, it indicates that system performance is improving. At this time, reducing the response to emergency tasks means reducing the α value, while increasing the consideration of historical efficiency means increasing the γ value.

[0225] When I decreases, it means that the system encounters a bottleneck or efficiency problem. At this time, it is necessary to increase the weight of urgent tasks. At this time, α increases to quickly solve the problem that causes system congestion.

[0226] Embodiment six:

[0227] Further explanation based on the fifth embodiment, the acquisition of the quantifiable parameter set B of the task urgency, resource requirements and historical execution efficiency, the analysis and processing of the quantifiable parameter set B, and the generation of the fine-tuning index specifically include the following contents:

[0228] The calculated data of task urgency, resource demand, historical execution efficiency, node performance change, resource application and release frequency are uniformly normalized to limit their values ​​to the same range. The fine-tuning index is defined as Wt. The calculation formula is as follows:

[0229]

[0230] Among them, Urgency(i) represents the urgency of task i. The higher the value, the more urgent the task.

[0231] ResourceDemand(i) represents the resource demand of task i. The larger the value, the more resources are needed.

[0232] HistoricalEfficiency(i) is the historical execution efficiency. The higher the value, the better the execution efficiency of past tasks. JDXn(n) refers to the performance change of node n at time t. This parameter reflects the real-time change of node performance. ZYSf represents the frequency of resource application and release. The higher the frequency, the more frequent the resource changes.

[0233] The value range of the Wt formula is set to (0,10), and it is controlled by adjusting the constant factor and the denominator to ensure that the output fine-tuning index can be directly applied to the parameter adjustment strategy. Different values ​​of the value range will directly affect the fine-tuning effect of the scheduling strategy:

[0234] When Wt is close to 0, it indicates that the current resource allocation and task urgency are relatively balanced and do not require significant adjustment;

[0235] When Wt is close to 10, it indicates that the task scheduling strategy needs to be significantly adjusted, because the resource demand is extremely large or the task is extremely urgent;

[0236] The value range of Wt (0, 10) is divided into four intervals, each interval corresponds to different adjustment requirements, and the adjustment of each interval is made based on the first feedback adjustment strategy, while ensuring that α+β+γ=1:

[0237] In the low fine-tuning interval Wt∈(0,2.5), the adjustment strategy is to slightly increase α2%, reduce β by (1+△h)% of the current value, and increase γ by 1%, where (1+△h)% is used to ensure that α+β+γ=1;

[0238] Make minor adjustments when the system performance is basically balanced to slightly increase the processing priority of urgent tasks while slightly reducing the impact of resource requirements. Increasing α will increase the response speed to urgent tasks; reducing β will reduce the intensity of resource requirements; increasing γ will take more account of historical execution efficiency;

[0239] In the medium fine-tuning interval Wt∈[2.5,5), the adjustment strategy is to increase the current value of α by 5%, keep β unchanged, and reduce the current value of γ by (3+△h)%, (3+△h)% is used to ensure that α+β+γ=1; at this time, the system performance is basically in line with expectations, and the adjustment is aimed at slightly increasing the processing priority of emergency tasks, and the adjustment range is within 15%;

[0240] Increasing α more significantly improves the impact of task urgency, and decreasing γ means slightly reducing the reliance on historical execution efficiency;

[0241] In the high fine-tuning interval Wt∈[5,7.5), the adjustment strategy is to increase the current value of α by (10+△h)%, (10+△h)% is used to ensure that α+β+γ=1, reduce β by 5%, and keep γ unchanged; at this time, the system performance needs to be adjusted significantly, and the adjustment range is within 20%, especially in resource allocation and task urgency;

[0242] The substantial increase in α significantly improves the response to emergency tasks;

[0243] The reduction of β more obviously reduces the dependence on resources, which is conducive to optimizing allocation when resources are limited;

[0244] In the extremely high fine-tuning interval Wt∈[7.5,10), the adjustment strategy is to increase α by 20%, reduce β by 10%, and increase the current value of γ by (10+△h)%, and (10+△h)% is used to ensure that α+β+γ=1. At this time, the system performance is seriously insufficient and needs to be adjusted significantly to cope with urgent and high resource demand tasks. The adjustment range is within 30%.

[0245] αnew=α·(1+k α ·Wt)

[0246] βnew=β·(1-kβ·Wt)

[0247] γnew=γ·(1+k γ ·Wt)

[0248] Among them, k α ,kβ and k γ It is the adjustment coefficient preset according to different intervals of Wt value range;

[0249] k α ∈(0.02,0.05,0.10,0.20), kβ∈(0.01,0.00,0.05,0.10); k γ ∈(0.01,-0.03,0.00,0.10); k α ,kβ and k γ Corresponding to four intervals respectively;

[0250] For k α ,kβ and k γ The quantitative analysis steps are as follows:

[0251] Data collection: Based on historical data and actual operation experience, collect data on the relationship between Wt and system performance indicators;

[0252] Statistical analysis: Use regression analysis and variance analysis statistical methods to determine the strength and characteristics of the relationship between Wt and key performance indicators;

[0253] Simulation experiment: By simulating different Wt scenarios, observe the impact of adjusting α, β, and γ on system performance and optimize the selection of adjustment coefficients;

[0254] Effect-based feedback loop: adjust k based on feedback collected from simulation and actual operation α ,kβ and k γ This is a dynamic process that is adjusted as the system environment and requirements change;

[0255] Interval division:

[0256] When the Wt value in the low fine-tuning range is low, the system does not need to be adjusted significantly. At this time, the adjustment coefficient is within 0.03, and fine adjustments are mainly used;

[0257] The Wt value in the medium fine-tuning interval is at a medium level, the system needs to be adjusted to adapt to the changes, and the adjustment coefficient can be increased moderately. At this time, the adjustment coefficient is within 0.05;

[0258] When the Wt value in the high fine-tuning range is high, the system faces extreme conditions or high loads and requires significant adjustments. At this time, the adjustment coefficient should be large, within 0.1 and above 0.03, to quickly adapt to system needs;

[0259] When the Wt value in the extremely high fine-tuning interval is extremely high, the adjustment coefficient is above 0.1 and within 0.3;

[0260] The adjustment strategy dynamically adjusts the values ​​of α, β, and γ according to the real-time calculation results of Wt;

[0261] Continuously monitor system performance and external conditions, and adjust coefficients and interval settings based on actual performance.

[0262] Embodiment seven:

[0263] Further explanation is given on the basis of Example 6. The actual adjustment effect of the fine-tuning index Wt under different task scheduling conditions is verified through scientific experiments, and the optimization of the overall system performance by the dynamic adjustment of the α, β, and γ parameters is evaluated. The experiment uses a virtual task scheduling system to simulate the actual operating environment, which includes multiple tasks with different resource requirements, urgency, and historical execution efficiency.

[0264] The experimental equipment and environment configuration are as follows:

[0265] Hardware equipment: including multiple server nodes, each of which is equipped with standard CPU, memory and storage devices;

[0266] Software environment: Use relevant task scheduling simulation software, which can simulate task generation, execution and resource allocation;

[0267] The task definition and parameter settings are as follows:

[0268] Task urgency (i) indicates that different weights are assigned according to the time sensitivity of the task, ranging from 1 to 10;

[0269] Resource demand ResourceDemand(i) indicates that the amount of computing resources required for task processing is divided into three levels: low (1-3), medium (4-6), and high (7-10);

[0270] Historical Efficiency (i) indicates the efficiency of previous task completion, which is divided into high efficiency (>80%), medium (50%-80%), and low efficiency (<50%).

[0271] Node performance change JDXn(n) indicates that the node performance changes dynamically based on the performance after the most recent maintenance, and the value fluctuates between 0.8 and 1.2;

[0272] Resource application and release frequency ZYSf represents the frequency of resource application and release operations in the system, from 1 (very rare) to 10 (very frequent);

[0273] The calculation and adjustment strategy of the fine-tuning index Wt are as follows:

[0274] Based on the above parameters, calculate the Wt value using the formula provided;

[0275] The system is divided into four adjustment intervals according to the value of Wt, and each interval corresponds to a specific α, β, and γ adjustment strategy;

[0276] The experimental process is as follows:

[0277] For the initial setup, all parameters were set to medium levels to ensure that the system was in a balanced state at the beginning;

[0278] Simulate task generation, automatically and randomly generate 50 tasks, and randomly assign each task's urgency, resource requirements, and historical execution efficiency;

[0279] Task execution and scheduling, execute tasks according to the initial α, β, and γ configurations, and dynamically adjust these three parameters based on the Wt value after each task is completed;

[0280] Data collection, recording the Wt value of each task execution, the parameter values ​​before and after adjustment, and the time and resource consumption of task completion;

[0281] Efficiency evaluation,At the end of the experiment, the system efficiency before and after the adjustment was compared,,using the average time to complete the task and the resource utilization as the main,evaluation indicators;

[0282] Below is a detailed experimental data table showing the initial and adjusted parameter values ​​for the six groups of test subjects, as well as the corresponding system performance indicators;

[0283] Table 1

[0284]

[0285] For the above table data, the beneficial effects and innovative features of the present invention are demonstrated by comparing and analyzing the changes between each group of data in detail. By adjusting the parameters α, β, and γ, the task completion time and resource utilization are optimized, and obvious numerical interaction changes and technical effects are demonstrated. The following is a specific analysis of these data and changes:

[0286] 1. Relationship between parameter adjustment and task performance:

[0287] The parameters α, β, and γ represent different system control factors, such as processing speed, resource allocation priority, and task completion efficiency;

[0288] The Wt index can generate a second feedback adjustment strategy, which can fine-tune the α, β, and γ parameters based on the first feedback adjustment strategy to form the final optimization scheduling result;

[0289] 2. Analyze specific data changes:

[0290] The initial values ​​of α, β, and γ for Tasks 1 to 6 are 0.33, 0.34, and 0.33, respectively, which indicates that before adjustment, the system adopted a balanced strategy for these factors;

[0291] After adjustment, it was observed that the increase or decrease of α and γ was closely related to the improvement of resource utilization and completion time. After α was significantly improved, the completion time of Task 4 and Task 5 was reduced to 75 seconds and 70 seconds respectively, and the resource utilization was improved to 90% and 95% respectively.

[0292] 3. Numerical changes and technical effects:

[0293] Increasing α is accompanied by an increase in task processing speed. For example, after α is increased to 0.53 in Task 4 and Task 5, the completion time is significantly shortened, indicating that adjusting the processing speed parameter can effectively improve efficiency;

[0294] Adjusting β is related to the error tolerance rate and resource allocation priority. Lowering the β value, such as 0.24 in Task 5, means that a higher priority is assigned to the current task, thereby optimizing resource utilization;

[0295] Adjusting γ is related to the stability or error handling capability of the system. The increase of γ to 0.43 in Task 5 corresponds to its high resource utilization of 95%, indicating that the system has enhanced its error handling capability while maintaining high efficiency;

[0296] 4. Quantitative representation of specific technical effects:

[0297] Completion time,By comparing the data before and after the adjustment, it can be seen that the completion time of Task 4 and Task 5 was reduced by 6.67% (from 80 seconds to 75 seconds) and 12.5% ​​(from 80 seconds to 70 seconds) after the adjustment;

[0298] Resource utilization,Task 5 shows the most significant improvement, from 90% to 95%, an increase of 5.56%;

[0299] These data analyses show that by precisely adjusting α, β, and γ, the task processing efficiency and resource utilization can be significantly improved while ensuring system stability. This not only proves the innovativeness of the invention, but also provides an efficient task scheduling method with practical application value and technical prospects. The numerical interactive changes achieved by adjusting different parameters provide a quantifiable and specific execution method, making the processing of each task more efficient and customized.

[0300] Embodiment eight:

[0301] See also Figure 2: A task distribution device based on Manage and Worker, the device is used to execute the task distribution method based on Manage and Worker, comprising:

[0302] Task dependency graph construction module: It is used to collect historical execution data of containerized tasks, dependencies between tasks, size of shared data, and access frequency information, and analyze and process them. The task dependency graph is constructed based on these data and represented by a directed graph G = (V, E), where each vertex v∈V represents a task, and each edge e = (u, v)∈E represents that task u depends on task v. The expression formula is as follows:

[0303] G=(V,E),where V=v 1 ,v 2 ,…,vn,E=(vi,vj)|vi depends on vj

[0304] Among them, v 1 ,v 2 ,…,vn are nodes in the graph, each node represents a task, and each edge (vi,vj) indicates that task vi depends on task vj, that is, the completion or output of task vj is necessary for the execution of task vi;

[0305] Priority calculation module: It is used to obtain the task dependency graph and use topological sorting to determine the execution order of tasks. It also introduces the priority weight w(v) based on task urgency, resource requirements, and historical execution efficiency. The expression formula is as follows:

[0306] w(v)=α×urgency+β×resource demand+γ×historical efficiency

[0307] Among them, α, β, and γ are adjustment coefficients used to balance the influence of different factors. Task urgency, resource demand, and historical efficiency are represented by urgency, resource demand, and historical efficiency, respectively.

[0308] Scheduling optimization module: used to obtain the task execution sorting results in step S2, apply genetic algorithm, and dynamically allocate tasks to the most suitable nodes according to the priority weight of the task and the real-time status of the node. The goal of the algorithm is to minimize the total task execution time and data transmission time, while maximizing resource utilization. The expression formula is as follows:

[0309]

[0310] Where texec(v) is the execution time of task v, and ttrans(v,E(v)) is the data transmission time between task v and its dependent task set E(v);

[0311] Feedback adjustment module: used to deploy scheduling strategies in the Docker Swarm environment, monitor their effects, and then collect quantifiable parameter set A in the monitoring effects, analyze and process the quantifiable parameter set A, generate a calibration index, and generate a first feedback adjustment strategy based on the calibration index. The first feedback adjustment strategy can adjust the α, β, and γ parameters according to the system operation feedback to preliminarily optimize the scheduling results;

[0312] Fine-tuning module: used to obtain a quantifiable parameter set B of task urgency, resource requirements and historical execution efficiency, analyze and process the quantifiable parameter set B, generate a fine-tuning index, and generate a second feedback adjustment strategy based on the fine-tuning index. The second feedback adjustment strategy can fine-tune the α, β, and γ parameters based on the first feedback adjustment strategy to form the final optimized scheduling result.

[0313] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0314] The above embodiments may be implemented in whole or in part by software, hardware, firmware or any other combination thereof. When implemented by software, the above embodiments may be implemented in whole or in part in the form of a computer program product. Those skilled in the art may appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein may be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed by hardware or software methods depends on the specific application and design constraints of the technical solution.

[0315] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0316] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A task distribution method based on Manage and Worker, characterized in that: The specific steps include: Step S1: collect historical execution data of containerized tasks, dependencies between tasks, size of shared data, and access frequency information, and analyze and process them. Use these data to construct a task dependency graph, which is represented by a directed graph G = (V, E), where each vertex v∈V represents a task, and each edge e = (u, v)∈E represents that task u depends on task v. The expression formula is as follows: G=(V,E),where V=v1,v2,…,v n ,E=(v i ,v j )|v i depends on v j Among them, v1, v2, …, v n is a node in the graph, each node represents a task, and each edge (v i, v j ) represents task v i Depends on task v j ; Step S2: After obtaining the task dependency graph, use topological sorting to determine the execution order of the tasks, and introduce the priority weight w(v) based on task urgency, resource requirements, and historical execution efficiency. The expression formula is as follows: w(v)=α×urgency+β×resource demand+γ×historical efficiency Among them, α, β, and γ are adjustment coefficients used to balance the influence of different factors. Task urgency, resource demand, and historical efficiency are represented by urgency, resource demand, and historical efficiency, respectively. Step S3, obtain the task execution sorting result in step S2, apply the genetic algorithm, dynamically allocate the task to the most suitable node according to the priority weight of the task and the real-time status of the node. The goal of the algorithm is to minimize the total task execution time and data transmission time, while maximizing resource utilization. The expression formula is as follows: Among them, t exec (v) is the execution time of task v, t trans (v, E(v)) is the data transfer time between task v and its dependent task set E(v); Step S4, deploying the scheduling strategy in step S3 in the Docker Swarm environment, monitoring its effect, and then collecting the quantifiable parameter set A in the monitoring effect, analyzing and processing the quantifiable parameter set A, generating a calibration index, and generating a first feedback adjustment strategy based on the calibration index, the first feedback adjustment strategy can adjust the parameters of the priority weight w(v) according to the system operation feedback, and preliminarily optimize the scheduling result; Get the task completion time t exec , data transmission time t trans , resource utilization rate ZYl, task failure rate RWSb and task queuing time PDt are normalized and analyzed to generate a calibration index, which is defined as I. The calculation formula is as follows: T exec (i) represents the execution time of task i, T trans (i, j) measures the efficiency of data transmission between tasks i and j, ZYl represents the efficiency of system resource utilization, RWSb(i) represents the failure rate of task i, PDt(i) represents the time spent by task i waiting for resource allocation, k, d1, d2, d3, d4 and d5 are adjustment coefficients used to adjust the relative importance and influence of each parameter in the formula, and η is a positive adjustment constant; The output value range of the I formula is set to (0,1). The closer the value is to 1, the better the system performance is. The closer the value is to 0, the potential efficiency problem or bottleneck exists in the system. The first feedback adjustment strategy includes dividing the value range of the calibration index I (0, 1) into three main intervals, each interval corresponding to a different system performance state and a corresponding adjustment strategy; Low performance interval I∈(0,0.3), where the system performance is seriously insufficient and requires substantial adjustments; The feedback adjustment strategy is to increase α to increase the weight of task urgency by 20%, decrease β by 10% to reduce the impact of resource demand, and increase γ by 10% to take more account of historical execution efficiency; Medium performance interval I∈[0.3,0.7), where the system performance basically meets expectations, but there is still room for improvement; The feedback adjustment strategy is to slightly increase α by 5%, keep β unchanged, and slightly reduce γ by 5%; High performance interval I∈[0.7,1), where the system performance is superior and adjustments are aimed at subtle optimization; The feedback adjustment strategy is to reduce α by 5%, keep β unchanged, and increase γ by 5%; When I increases, it indicates that system performance is improving. At this time, reducing the response to emergency tasks means reducing the α value, while increasing the consideration of historical efficiency means increasing the γ value. When I decreases, it means that the system encounters bottlenecks or efficiency problems. At this time, the weight of urgent tasks needs to be increased. At this time, α increases to quickly solve the problem that causes system congestion; Step 5: Obtain a quantifiable parameter set B of task urgency, resource requirements, and historical execution efficiency, analyze and process the quantifiable parameter set B, generate a fine-tuning index, and generate a second feedback adjustment strategy based on the fine-tuning index. The second feedback adjustment strategy can fine-tune the parameters of the priority weight w(v) on the basis of the first feedback adjustment strategy to form a final optimized scheduling result; The calculated data of task urgency Urgency, resource demand ResourceDemand, historical execution efficiency HistoricalEfficiency, node performance change JDXn, and resource application and release frequency ZYSf are uniformly normalized to limit their values ​​to the same range, and the fine-tuning index is defined as Wt. The calculation formula is as follows: Among them, Urgency(i) represents the urgency of task i. The higher the value, the more urgent the task. ResourceDemand(i) represents the resource demand of task i. The larger the value, the more resources are needed. HistoricalEfficiency(i) is the historical execution efficiency. The higher the value, the better the execution efficiency of past tasks. JDXn(n) refers to the performance change of node n at time t. This parameter reflects the real-time change of node performance. ZYSf represents the frequency of resource application and release. The higher the frequency, the more frequent the resource changes. The value range of the Wt formula is set to (0,10). When Wt is close to 0, it indicates that the current resource allocation and task urgency are relatively balanced and do not require significant adjustment. When Wt is close to 10, it indicates that the task scheduling strategy needs to be significantly adjusted, because the resource demand is extremely large or the task is extremely urgent; The value range of Wt (0, 10) is divided into four intervals, each interval corresponds to different adjustment requirements, and the adjustment of each interval is made based on the first feedback adjustment strategy, while ensuring that α+β+γ=1: In the low fine-tuning interval Wt∈(0,2.5), the adjustment strategy is to slightly increase α by 2%, reduce β by (1+△h)% of the current value, and increase γ by 1%; In the medium fine-tuning interval Wt∈[2.5,5), the adjustment strategy is to increase the current value of α by 5%, keep β unchanged, and reduce the current value of γ by (3+△h)%. At this time, the system performance is basically in line with expectations. The adjustment aims to slightly increase the processing priority of emergency tasks, and the adjustment range is within 15%; In the high fine-tuning interval Wt∈[5,7.5), the adjustment strategy is to increase the current value of α by (10+△h)%, reduce β by 5%, and keep γ unchanged; at this time, the system performance needs to be adjusted significantly, and the adjustment range is within 20%, in terms of resource allocation and task urgency; In the extremely high fine-tuning interval Wt∈[7.5,10), the adjustment strategy is to significantly increase α by 20%, reduce β by 10%, and increase γ by (10+△h)% of the current value; at this time, the system performance is seriously insufficient and needs to be significantly adjusted to cope with urgent and high-resource-demand tasks, and the adjustment range is within 30%; and the following adjustment formulas for α, β, and γ are set in turn; α new =α·(1+k α ·Wt) β new =β·(1-k β ·Wt) γ new =γ·(1+k γ ·Wt) Among them, k α ,k β and k γ It is an adjustment coefficient preset according to different intervals of the Wt value range.

2. According to claim 1, a task distribution method based on Manage and Worker is characterized in that: The Manage, Worker and task dependency graph specifically includes the following contents: Manager means that in a Docker Swarm cluster, the Manager is responsible for managing the cluster status, scheduling services, and performing task allocation. The Manager is also responsible for handling the cluster's network configuration and service discovery mechanism; Workers are nodes that perform actual containerized tasks. They receive instructions from the Manager and run specific tasks based on these instructions. The task dependency graph is a directed graph used to represent the dependency relationship between tasks. The nodes in the graph represent each task, and the edges represent that a task must be executed after another task. After obtaining the task dependency graph, the execution order of the tasks is determined using topological sorting, and a priority weight w(v) based on task urgency, resource requirements, and historical execution efficiency is introduced, specifically including the following: The algorithm for sorting directed graphs by topological sorting makes u appear before v in the sorting for any directed edge from vertex u to vertex v. Using topological sorting in task scheduling can ensure that the execution order of tasks complies with the dependency relationship. The priority weight w(v) is used in a computational model for evaluating the task execution priority, where v represents a specific task node, and the weight calculation takes into account the urgency, resource requirements, and historical execution efficiency of the task.

3. According to claim 2, a task distribution method based on Manage and Worker is characterized in that: The task execution sorting result in step S2 is obtained, and a genetic algorithm is applied to dynamically allocate tasks to the most suitable nodes according to the priority weight of the tasks and the real-time status of the nodes. The goal of the algorithm is to minimize the total task execution time and data transmission time while maximizing resource utilization, which specifically includes the following contents: The genetic algorithm includes, in sequence, initializing population, evaluating fitness, selection, crossover and mutation, new generation population, repetition and result extraction; Initializing the population is to randomly generate an initial population as a starting point; Fitness assessment is to calculate the fitness of each individual and determine its probability of survival and reproduction; Selection is based on individual fitness, using a roulette wheel method to select individuals for reproduction; Crossover and mutation are to generate new individuals through crossover and mutation to increase the diversity of the population; The new generation of population is formed by combining some old individuals with newly generated individuals; Repeat the above steps until the termination condition is met; The result is extracted by selecting the individual with the highest fitness as the final solution; The priority weight is a weight assigned to each task in task scheduling based on urgency, resource requirements and historical execution efficiency to determine its execution priority; The real-time status refers to the current status of each execution node when the task is distributed, including available resources, network bandwidth, and CPU usage; The total task execution time and data transmission time are one of the objectives of genetic algorithm optimization.

4. According to claim 3, a task distribution method based on Manage and Worker is characterized in that: The quantifiable parameter set A includes task completion time, data transmission time, resource utilization, task failure rate and task queuing time, and the task completion time is marked as t exec , mark the data transmission time as t trans , the resource utilization is marked as ZYl, the task failure rate is marked as RWSb, and the task queuing time is marked as PDt; The task completion time t exec Indicates the time required for each task from start to completion; The data transmission time t trans It represents the time required for data transfer between tasks due to data dependency; The resource utilization rate ZY1 includes CPU utilization rate, memory occupancy rate, and network bandwidth usage. The task failure rate RWSb represents the frequency of failure of each task execution; The task queuing time PDt represents the time a task spends waiting for resource allocation; The quantifiable parameter set B includes task urgency, resource demand, historical execution efficiency, node performance change, resource application and release frequency, and the task urgency is marked as Urgency, the resource demand is marked as ResourceDemand, the historical execution efficiency is marked as HistoricalEfficiency, the node performance change is marked as JDXn, and the resource application and release frequency is marked as ZYSf; The task urgency Urgency represents the urgency of the task combined with the deadline and the importance level; The resource demand ResourceDemand represents the CPU time, memory size, and I / O bandwidth required for the task; The historical execution efficiency HistoricalEfficiency represents the completion efficiency of similar tasks in history; The node performance change JDXn represents the performance fluctuation data of each processing node. The resource application and release frequency ZYSf represents the frequency at which resources are applied for and released, so as to evaluate the efficiency of resource management.

5. A task distribution device based on Manage and Worker, characterized in that: The device is used to execute the task distribution method based on Manage and Worker according to any one of claims 1 to 4, including: Task dependency graph construction module: It is used to collect historical execution data of containerized tasks, dependencies between tasks, size of shared data, and access frequency information, and analyze and process them. The task dependency graph is constructed based on these data and represented by a directed graph G = (V, E), where each vertex v∈V represents a task, and each edge e = (u, v)∈E represents that task u depends on task v. The expression formula is as follows: G=(V,E),where V=v1,v2,…,v n ,E=(v i, v j )|v i depends on v j Among them, v1, v2, …, v n is a node in the graph, each node represents a task, and each edge (v i ,v j ) represents task v i Depends on task v j , that is, task v j The completion or output of task v i necessary for the execution of Priority calculation module: It is used to obtain the task dependency graph and use topological sorting to determine the execution order of tasks. It also introduces the priority weight w(v) based on task urgency, resource requirements, and historical execution efficiency. The expression formula is as follows: w(v)=α×urgency+β×resource demand+γ×historical efficiency Among them, α, β, and γ are adjustment coefficients used to balance the influence of different factors. Task urgency, resource demand, and historical efficiency are represented by urgency, resource demand, and historical efficiency, respectively. Scheduling optimization module: used to obtain the task execution sorting results in step S2, apply genetic algorithm, and dynamically allocate tasks to the most suitable nodes according to the priority weight of the task and the real-time status of the node. The goal of the algorithm is to minimize the total task execution time and data transmission time, while maximizing resource utilization. The expression formula is as follows: Among them, t exec (v) is the execution time of task v, t trans (v, E(v)) is the data transfer time between task v and its dependent task set E(v); Feedback adjustment module: used to deploy scheduling strategies in the Docker Swarm environment, monitor their effects, and then collect quantifiable parameter set A in the monitoring effects, analyze and process the quantifiable parameter set A, generate a calibration index, and generate a first feedback adjustment strategy based on the calibration index. The first feedback adjustment strategy can adjust the parameters of the priority weight w(v) according to the system operation feedback to preliminarily optimize the scheduling results; Fine-tuning module: used to obtain a quantifiable parameter set B of task urgency, resource requirements and historical execution efficiency, analyze and process the quantifiable parameter set B, generate a fine-tuning index, and generate a second feedback adjustment strategy based on the fine-tuning index. The second feedback adjustment strategy can fine-tune the parameters of the priority weight w(v) based on the first feedback adjustment strategy to form the final optimized scheduling result.

Images