A dynamic and static combined parallel computing method and system and a storage medium

Abstract

The application provides a kind of dynamic and static combined parallel computing method, system and storage medium, method includes: the computing task to be processed is decomposed into multiple subtasks and constructs directed acyclic graph, after multiple subtasks are assigned to multiple CPU cores, the encoding value of each subtask is calculated and random exchange is carried out, and then the obtained multiple initial task execution sequences are updated by genetic algorithm iteration to obtain the optimal task execution sequence, after the fitness of multiple initial task execution sequences is calculated, part is reserved and is crossed and mutated, according to the optimal task execution sequence, the subtask is added to the low priority task queue;Subtask is selected in high priority task queue first, and it is judged whether to execute subtask, when resource is enough to process, then according to the preset multi-thread programming interface, subtask is distributed to idle CPU core and parallel execution is carried out to the subtask with the same level value, when it is not enough to process, it is added to high priority queue, and the parallel computing result is obtained after all subtasks are executed completely.

Description

Technical Field

[0001] This invention relates to the field of parallel computing technology, and in particular to a dynamic-static combined parallel computing method, system, and storage medium. Background Technology

[0002] In recent years, with the continuous improvement of computer performance and the in-depth research in various industries, a large number of computing tasks with high computational load, timeliness requirements, multi-scale and multi-variable coupling, repetitiveness and iterativeness have emerged. It is difficult to meet the requirements of these applications using traditional methods on general computers.

[0003] Existing technologies often use single-machine multi-core processor computers for computational tasks, but serial computing struggles to process large grid data within a reasonable timeframe. Current parallel computing methods cannot fully and effectively utilize the performance of multi-core CPUs while ensuring the shortest total execution time for all tasks. Furthermore, they cannot guarantee that the dependencies between tasks will not be disrupted during parallel task scheduling. Moreover, besides the limitation imposed by the number of CPU cores on parallel capabilities, the size of the computer's memory also fails to consider the limitations on the number of computational tasks that can be executed simultaneously. Therefore, how to efficiently utilize CPU computing power and shorten the execution time of computational tasks is a pressing issue that needs to be addressed. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a dynamic-static combined parallel computing method, system and storage medium to eliminate or improve one or more defects existing in the prior art, and solve the problems of low CPU computing efficiency and long computing task execution time in the prior art.

[0005] One aspect of the present invention provides a dynamic-static combined parallel computing method, the method comprising the following steps:

[0006] Obtain the computation task to be processed and decompose the computation task into multiple sub-tasks, and construct a directed acyclic graph based on the multiple sub-tasks and the processing order relationship between the sub-tasks;

[0007] Multiple subtasks in the directed acyclic graph are allocated to multiple CPU cores according to a preset allocation method. The encoding value of each subtask is calculated according to the gene value encoding method, and the encoding values ​​of each subtask are randomly swapped to obtain multiple initial task execution sequences. Genetic algorithm is used to iteratively update the sequence with the shortest task completion time. After fitness calculation of multiple initial task execution sequences based on task completion time and penalty term, a roulette wheel selection method is used to retain some initial execution sequences. The encoding values ​​of any two initial execution sequences in the retained initial execution sequences are swapped, and the positions of the encoding values ​​of the generated task execution sequences are adjusted. The iteration is updated until a preset termination condition is met to obtain the optimal task execution sequence. Multiple subtasks are added to a low-priority task queue according to the optimal task execution sequence.

[0008] A subtask is selected from the high-priority task queue. If there is no subtask in the high-priority task queue, a subtask is selected from the low-priority task queue. The current CPU utilization and memory usage determine whether to execute the subtask. If the CPU system resources are sufficient to process the selected subtask, multiple subtasks are allocated to idle CPU cores for parallel execution according to the number of threads in the parallel zone and the thread allocation relationship set by the preset multi-threaded programming interface. If the CPU system resources are insufficient to process the selected subtask, the subtask is added to the high-priority queue. This process is repeated until all subtasks are completed and the parallel computing result is obtained.

[0009] In some embodiments, the process of allocating multiple subtasks in the directed acyclic graph to multiple CPU cores according to a preset allocation method, calculating the encoding value of each subtask according to the gene value encoding method, and randomly exchanging the encoding values ​​of each subtask to obtain multiple initial task execution sequences includes:

[0010] Multiple subtasks are assigned to multiple CPU cores to obtain an initial allocation relationship. The encoding value of each subtask is calculated based on the CPU core number, subtask number, and total number of CPU cores, and the task execution sequence corresponding to the initial allocation relationship is constructed. When the number of subtasks with the same level value can be evenly distributed among the CPU cores, they are evenly distributed. When the number of subtasks with the same level value cannot be evenly distributed among the CPU cores, the remainder is obtained by dividing the number of subtasks by the number of CPU cores. The subtasks with the remainder are selected and randomly assigned to CPU cores. Then, the remaining subtasks are evenly distributed among the CPU cores.

[0011] Multiple initial task execution sequences are obtained by randomly swapping the encoding values ​​of subtasks on each CPU core in the task execution sequence corresponding to the initial allocation relationship.

[0012] In some embodiments, the expression for calculating the encoded value of each subtask based on the CPU core number, subtask number, and total number of CPU cores is as follows:

[0013] v i =p i +i×P;

[0014] Where i represents the subtask number, p i This indicates the CPU core number of subtask number i; P indicates the total number of CPU cores.

[0015] In some embodiments, the expression for calculating the fitness of multiple initial task execution sequences using task completion time and penalty terms is as follows:

[0016]

[0017] Where λ represents the penalty weight coefficient; Makespan represents the total task completion time; and Penalty represents the penalty term.

[0018] In some embodiments, the process of using a roulette wheel selection method to retain a portion of the initial execution sequence, swapping the partial encoded values ​​of any two initial execution sequences in the retained portion, and adjusting the positions of the partial encoded values ​​of the generated task execution sequences after the swap includes:

[0019] A roulette table is obtained based on the proportion of the fitness of each initial task execution sequence to the total fitness value. The roulette table is used to randomly select and retain the selected initial task execution sequence as the task execution sequence to be processed.

[0020] Randomly select any two tasks from the sequence of tasks to be processed, and set an initial crossover point and a cutoff crossover point at the same position in the two task sequences. Then, according to a preset crossover probability, swap the two segments of encoded values ​​intercepted through the initial crossover point and the cutoff crossover point.

[0021] In the task execution sequences generated after the exchange, two mutation points are randomly selected and their encoded values ​​are exchanged according to a preset mutation probability.

[0022] In some embodiments, determining whether to execute the subtask based on current CPU utilization and memory usage further includes:

[0023] The utilization of each CPU core is calculated using a preset performance counter, and the total CPU utilization is calculated as well.

[0024] Use the Global Memory Status Ex function to obtain memory usage information.

[0025] In some embodiments, multiple subtasks having the same level value are executed in parallel, wherein the expression for the level value is:

[0026]

[0027] Where pre(i) represents all direct predecessor nodes of task number i; maxh(j) represents the maximum level value among all predecessor nodes of task number i; This represents the empty set.

[0028] On the other hand, the present invention also provides a dynamic-static combined parallel computing system, the system being used to execute the dynamic-static combined parallel computing method described in any of the above claims, the system comprising:

[0029] The task dispatcher module is used to acquire the computing tasks to be processed and decompose the computing tasks into multiple subtasks. The multiple subtasks in the directed acyclic graph are allocated to multiple CPU cores according to a preset allocation method. The encoding value of each subtask is calculated according to the gene value encoding method, and the encoding values ​​of each subtask are randomly swapped to obtain multiple initial task execution sequences. The genetic algorithm is used to iteratively update the task execution sequence with the goal of generating the task with the shortest completion time. After calculating the fitness of the multiple initial task execution sequences through task completion time and penalty terms, a roulette wheel selection method is used to retain some initial execution sequences. The encoding values ​​of any two initial execution sequences in the retained initial execution sequences are swapped, and the positions of the encoding values ​​of the generated task execution sequences are adjusted. The iterative update continues until a preset termination condition is met to obtain the optimal task execution sequence. The multiple subtasks are then added to the low-priority task queue according to the optimal task execution sequence.

[0030] The queue manager module is used to add multiple subtasks to a low-priority task queue according to the optimal task execution sequence and to determine whether there are subtasks in the high-priority queue. The queue manager uses a single task queue and adopts a multi-level priority queue. After receiving the subtasks added by the task dispatcher, it inputs them into the task executor for execution.

[0031] The task executor module is used to select a subtask from the high-priority task queue after the queue manager module determines that there is a subtask in the high-priority task queue. When the queue manager module determines that there is no subtask in the high-priority task queue, it selects a subtask from the low-priority task queue. It determines whether to execute the subtask based on the current CPU utilization and memory usage. When the CPU system resources are sufficient to process the selected subtask, it allocates multiple subtasks to idle CPU cores for parallel execution based on the number of threads in the parallel area and the thread allocation relationship set by the preset multi-threaded programming interface. When the CPU system resources are insufficient to process the selected subtask, it adds the subtask to the high-priority queue. The process is repeated until all subtasks are completed and the parallel computing result is obtained.

[0032] The system monitor module is used to monitor CPU utilization and memory usage in real time.

[0033] In some embodiments, the static-dynamic parallel computing system further includes:

[0034] The data management module is used to manage and store various types of data used and generated during the calculation process.

[0035] On the other hand, the present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of any of the methods described above.

[0036] In the static-dynamic parallel computing method and system described in this invention, the computational task is divided into multiple subtasks. A directed acyclic graph is constructed based on these subtasks and their processing order to ensure that the dependencies between subtasks are not disrupted, thus guaranteeing the accuracy of the computation results. A high-quality initial task execution sequence is selected through genetic computation, while low-quality sequences are eliminated. Crossover and mutation are performed among the selected high-quality initial task execution sequences, iterating continuously until the optimal task execution sequence with the desired fitness is obtained. Multiple subtasks are then added to a low-priority task queue, and subtasks are preferentially selected from the high-priority task queue. The execution of the current subtask is determined based on the current CPU utilization and memory usage. If the CPU system resources are insufficient to process the selected subtask, it is added to the high-priority queue. When system resources are abundant, more CPU cores or memory can be utilized. The shared memory space enhances the parallelism of the overall task. When system resources are insufficient, it can degenerate into a serial program, ensuring the smooth execution of computational tasks. The pre-defined multi-threaded programming interface sets the number of threads in the parallel zone and the thread allocation relationship, distributing multiple subtasks to idle CPU cores for parallel execution of multiple subtasks with the same level value. This achieves efficient parallel programming in a shared memory system, reducing the difficulty and complexity of parallel programming, while also possessing good portability and flexibility. Distributing multiple subtasks across multiple CPU cores for parallel computation effectively utilizes the performance of multi-core CPUs, improves the efficiency of parallel computing, ensures the shortest time to complete all subtasks, and enhances the timeliness of the results data. This dynamic-static combined approach can efficiently utilize the CPU's computing power, shorten the execution time of computational tasks, and realize a dynamic-static combined parallel computing system equipped with this dynamic-static combined parallel computing method.

[0037] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0038] Those skilled in the art will understand that the objectives and advantages achievable with this invention are not limited to those specifically described above, and that the above and other objectives achievable with this invention will become clearer from the following detailed description. Attached Figure Description

[0039] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, are not intended to limit the scope of the invention. In the drawings:

[0040] Figure 1 This is a flowchart illustrating a static-dynamic parallel computing method according to an embodiment of the present invention.

[0041] Figure 2 This is a flowchart illustrating the genetic algorithm according to an embodiment of the present invention.

[0042] Figure 3 This is a schematic diagram of the structure of the dynamic-static combined parallel computing method according to an embodiment of the present invention.

[0043] Figure 4 This is a schematic diagram of the structure of a directed acyclic graph according to an embodiment of the present invention. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0045] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0046] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0047] It should also be noted that, unless otherwise specified, the term "connection" in this article can refer not only to a direct connection, but also to an indirect connection involving an intermediary.

[0048] In the following description, embodiments of the invention will be illustrated with reference to the accompanying drawings. In the drawings, the same reference numerals represent the same or similar parts, or the same or similar steps.

[0049] In existing technologies, serial computing struggles to process large grid data within a reasonable timeframe, while parallel computing methods cannot fully and effectively utilize the performance of multi-core CPUs and guarantee the shortest total execution time for all tasks. Furthermore, during parallel task scheduling, the dependencies between tasks cannot be guaranteed to remain intact. Moreover, besides the limitation imposed by the number of CPU cores on parallel capabilities, the size of the computer's memory also limits the number of simultaneous computational tasks. This invention proposes a dynamic-static combined parallel computing method, system, and storage medium. It acquires the computational task to be processed and decomposes it into multiple subtasks. A directed acyclic graph is constructed based on the multiple subtasks and their processing order relationships. The subtasks are allocated to multiple CPU cores according to a preset allocation method. The encoded values ​​of each subtask are calculated using a gene value encoding method, and these values ​​are randomly swapped to obtain multiple initial task execution sequences. A genetic algorithm iteratively updates the sequence with the shortest completion time, and the initial task execution sequences are evaluated based on task completion time and penalties. After fitness calculation, a roulette wheel selection method is used to retain a portion of the initial execution sequence. Partial encoding values ​​from any two initial execution sequences in the retained portion are swapped, and the positions of partial encoding values ​​in the generated task execution sequences are adjusted. This process is iterated until a preset termination condition is met to obtain the optimal task execution sequence. Multiple subtasks are then added to a low-priority task queue according to this optimal sequence. Subtasks are selected from the high-priority task queue. If there are no subtasks in the high-priority task queue, a subtask is selected from the low-priority task queue. The current CPU utilization and memory usage are used to determine whether to execute the subtask. If the CPU system resources are sufficient to process the selected subtask, multiple subtasks are allocated to idle CPU cores for parallel execution based on the number of threads in the parallel zone and the thread allocation relationship set by the preset multi-threaded programming interface. If the CPU system resources are insufficient to process the selected subtask, the subtask is added to the high-priority queue. This process is repeated until all subtasks are executed and the parallel computation result is obtained.

[0050] Figure 1 This is a flowchart illustrating a static-dynamic combined parallel computing method according to an embodiment of the present invention. Specifically, this application provides a static-dynamic combined parallel computing method, which includes the following steps S101 to S103:

[0051] Step S101: Obtain the computation task to be processed and decompose the computation task into multiple subtasks. Construct a directed acyclic graph based on the multiple subtasks and the processing order relationship between the subtasks.

[0052] Step S102: Assign multiple subtasks in the directed acyclic graph to multiple CPU cores according to a preset allocation method. Calculate the encoding value of each subtask according to the gene value encoding method and randomly swap the encoding values ​​of each subtask to obtain multiple initial task execution sequences. Perform genetic algorithm iterative updates with the goal of generating the task execution sequence with the shortest completion time. Calculate the fitness of multiple initial task execution sequences based on task completion time and penalty terms, and then use a roulette wheel selection method to retain some initial execution sequences. Swap some encoding values ​​between any two initial execution sequences in the retained initial execution sequences and adjust the positions of some encoding values ​​in the generated task execution sequences. Iterate and update until a preset termination condition is met to obtain the optimal task execution sequence. Add multiple subtasks to the low-priority task queue according to the optimal task execution sequence.

[0053] Step S103: Select a subtask from the high-priority task queue. If there is no subtask in the high-priority task queue, select a subtask from the low-priority task queue. Determine whether to execute the subtask based on the current CPU utilization and memory usage. If the CPU system resources are sufficient to process the selected subtask, allocate multiple subtasks to idle CPU cores for parallel execution based on the number of threads in the parallel zone and the thread allocation relationship set by the preset multi-threaded programming interface. If the CPU system resources are insufficient to process the selected subtask, add the subtask to the high-priority queue. Repeat this process until all subtasks are completed and the parallel computing result is obtained.

[0054] In steps S101 and S102, after obtaining the computational task to be processed, it is first determined whether parallel computation should be performed. If parallel computation is not performed, data is obtained from the input file and serial computation is performed. If parallel computation is performed, the computational task is decomposed into multiple subtasks and static task scheduling is performed using a genetic algorithm. Subtasks are used as nodes, and the processing order relationship between subtasks is used as edges to construct a directed acyclic graph (DAG). Multiple subtasks form a directed acyclic graph G = <T, E>, where T is the set of subtask nodes, and E = {e...} ij Let} be the set of directed edges between subtasks, and let edge e be the set of edges between subtasks. ij This means that the subsequent subtask j can only be executed after the preceding subtask i has been executed, m = {M} ij} represents the size of the data that needs to be loaded during task computation, in M. ijThis indicates that subtasks i and j share a data relationship, meaning they require the same data for computation. A connection matrix stores the in-degree and out-degree of each task and ensures that subsequent tasks begin upon the completion of the preceding task. After obtaining the directed acyclic graph, the level value is calculated based on the dependencies between subtasks. The level value represents the level of the subtask. Tasks at the same level indicate no dependency between them, therefore subtasks at the same level can be executed in parallel. In some embodiments, multiple subtasks with the same level value are executed in parallel. The expression for the level value is:

[0055]

[0056] Where pre(i) represents all direct predecessor nodes of task number i; maxh(j) represents the maximum level value among all predecessor nodes of task number i; This represents the empty set.

[0057] In some embodiments, the process of allocating multiple subtasks in a directed acyclic graph to multiple CPU cores according to a preset allocation method, calculating the encoding value of each subtask according to the gene value encoding method, and randomly exchanging the encoding values ​​of each subtask to obtain multiple initial task execution sequences includes steps S1011 to S1012:

[0058] Step S1011: Assign multiple subtasks to multiple CPU cores to obtain an initial allocation relationship. Calculate the encoding value of each subtask based on the CPU core number, subtask number, and total number of CPU cores, and construct the task execution sequence corresponding to the initial allocation relationship. When the number of subtasks with the same level value can be evenly distributed to each CPU core, they are evenly distributed. When the number of subtasks with the same level value cannot be evenly distributed to each CPU core, obtain the remainder by dividing the number of subtasks by the number of CPU cores. Select the subtasks with the remainder and randomly assign them to CPU cores. Then, evenly distribute the remaining subtasks to each CPU core.

[0059] Step S1012: Randomly swap the encoding values ​​of the subtasks on each CPU core in the task execution sequence corresponding to the initial allocation relationship to obtain multiple initial task execution sequences.

[0060] Specifically, when allocating CPU cores to subtasks at each level, if the number of subtasks is n and the number of CPU cores is P, and n / P has no remainder, the tasks are evenly distributed among the CPU cores. If there is a remainder, the remaining number of subtasks are randomly selected and randomly allocated, and the remaining tasks are evenly distributed among the CPU cores. After obtaining the initial allocation relationship, the encoding value is calculated according to the gene value encoding method to obtain an initial task execution sequence. The encoding values ​​in this initial execution sequence are randomly swapped to obtain multiple initial task execution sequences. The random swapping rule is as follows: if there are m subtasks on a CPU core, the encoding values ​​of two subtasks are randomly swapped and this is repeated m / 2 times. The encoding values ​​on multiple CPU cores are randomly swapped. In some embodiments, the expression for calculating the encoding value of each subtask based on the CPU core number, subtask number, and total number of CPU cores is:

[0061] v i =p i +i×P;

[0062] Where i represents the subtask number; p i This indicates the CPU core number of subtask number i; P indicates the total number of CPU cores.

[0063] In some embodiments, the expression for calculating the fitness of multiple initial task execution sequences using task completion time and penalty terms is as follows:

[0064]

[0065] Where λ represents the penalty weight coefficient; Makespan represents the total task completion time; and Penalty represents the penalty term.

[0066] In some embodiments, the process of using a roulette wheel selection method to retain a portion of the initial execution sequence, swapping the partial encoded values ​​of any two initial execution sequences in the retained portion, and adjusting the positions of the partial encoded values ​​of the generated task execution sequences after the swap includes:

[0067] A roulette table is obtained based on the proportion of the fitness of each initial task execution sequence to the total fitness value. The roulette table is used to randomly select and retain the selected initial task execution sequence as the task execution sequence to be processed.

[0068] Randomly select any two tasks from the sequence of tasks to be processed, and set the initial crossover point and the cutoff crossover point at the same position in the two task sequences. Then, according to the preset crossover probability, swap the two segments of encoded values ​​intercepted through the initial crossover point and the cutoff crossover point.

[0069] In the task execution sequences generated after the exchange, two mutation points are randomly selected and their encoded values ​​are exchanged according to a preset mutation probability.

[0070] Figure 2 This is a flowchart illustrating the genetic algorithm according to an embodiment of the present invention. Specifically, in the genetic algorithm, multiple initial task execution sequences obtained after encoding are used as the initial population, and each initial task execution sequence is used as an individual in the population. The fitness calculation, selection, crossover, and mutation are performed on multiple initial execution sequences to obtain the optimal task execution sequence with the highest fitness. The sequence with the highest fitness has the shortest task completion time. A penalty term is introduced to check the data relationship between sub-tasks. When it is found that the data sharing relationship is not satisfied, a positive penalty term is added during fitness calculation to reduce the fitness, thereby reducing the probability of being selected in the subsequent selection process. This is to eliminate inferior individuals and retain superior individuals. When performing the roulette wheel selection method, the higher the fitness of the initial task execution sequence, the higher the probability of it being selected and retained. The process of the roulette wheel selection method includes: generating a uniformly distributed random number r in the interval [0,1]; if r≤q(1), then the first initial task execution sequence V1 is selected; if q(k-1)<r≤q(k) (2≤k≤SN, SN is the population size), then the initial task execution sequence V1 is selected. i Selected; q(i) is the cumulative probability of chromosome selection, expressed as: fit(V i ) represents the fitness of the initial task execution sequence V1.

[0071] Furthermore, when performing crossover and mutation operations, two random numbers are generated respectively. Crossover and mutation operations are performed when the random numbers are lower than the preset crossover probability and preset mutation probability. The preset termination condition is that the task execution sequence generated in the past few iterations has not improved significantly. The maximum value of the current fitness is compared with the maximum value of the usage obtained in the previous few iterations. The process terminates when the difference is less than the threshold.

[0072] In step S103, this application relies on a preset multi-threaded programming interface to implement the dynamic task scheduling stage. The preset multi-threaded programming interface adopts OpenMP (Open Multiprocessing Interface). After adding multiple subtasks to the low-priority task queue according to the optimal task execution sequence, it prioritizes selecting subtasks from the high-priority queue. Then, it determines whether to execute the selected subtask. If yes, it is assigned to the currently idle CPU core for parallel execution; otherwise, the selected subtask is added to the high-priority sequence and waits for the next round of priority execution. In some embodiments, when determining whether to execute the subtask based on the current CPU utilization and memory usage, it also includes: using a preset performance counter to calculate the utilization of each CPU core and calculating the total CPU utilization. The Global Memory Status Ex function is used to obtain the memory usage. The OpenMP framework, as a framework for performing parallel computing, sets the number of threads in the parallel area and sets the number of CPU cores used for computing through the omp set num threads function. It sets the affinity between threads and CPU cores to realize the correspondence between threads and CPU cores and obtain the allocation relationship of subtasks to specific CPU cores. The Windows performance counter is used, called through the pdh library, to calculate the utilization of each core and simultaneously calculate the total CPU utilization.

[0073] On the other hand, the present invention also provides a dynamic-static combined parallel computing system, the system being used to execute any of the above-mentioned dynamic-static combined parallel computing methods, the system comprising:

[0074] The task dispatcher module is used to acquire the computational tasks to be processed and decompose them into multiple subtasks. These subtasks in the directed acyclic graph are then allocated to multiple CPU cores according to a preset allocation method. The module calculates the encoding value of each subtask based on a gene value encoding method and randomly swaps these values ​​to obtain multiple initial task execution sequences. A genetic algorithm is used to iteratively update the sequence with the shortest completion time. After calculating the fitness of the multiple initial task execution sequences based on their completion time and penalty term, a roulette wheel selection method is used to retain a portion of the initial execution sequences. The module then swaps some of the encoding values ​​between any two of the retained initial execution sequences and adjusts the positions of some of the encoding values ​​in the resulting task execution sequences. This iterative update continues until a preset termination condition is met, at which point the optimal task execution sequence is obtained. Finally, multiple subtasks are added to a low-priority task queue according to the optimal task execution sequence.

[0075] The queue manager module is used to add multiple subtasks to the low-priority task queue according to the optimal task execution sequence and to determine whether there are subtasks in the high-priority queue. The queue manager uses a single task queue with multiple priority levels. After receiving the subtasks added by the task dispatcher, it inputs them into the task executor for execution.

[0076] The task executor module is used by the queue manager module to select a subtask from the high-priority queue after determining that a subtask exists in the high-priority queue. When the queue manager module determines that there is no subtask in the high-priority queue, it selects a subtask from the low-priority queue. It then determines whether to execute the subtask based on the current CPU utilization and memory usage. If the CPU system resources are sufficient to process the selected subtask, it allocates multiple subtasks to idle CPU cores for parallel execution based on the number of threads in the parallel zone and the thread allocation relationship set by the preset multi-threaded programming interface. If the CPU system resources are insufficient to process the selected subtask, it adds the subtask to the high-priority queue. This process is repeated until all subtasks are completed and the parallel computing result is obtained.

[0077] The system monitor module is used to monitor CPU utilization and memory usage in real time.

[0078] In some embodiments, the static-dynamic parallel computing system further includes:

[0079] The data management module is used to manage and store various types of data used and generated during the calculation process.

[0080] On the other hand, the present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0081] The present invention will now be described with reference to a specific embodiment:

[0082] Figure 3This is a schematic diagram of the static-dynamic combined parallel computing method according to an embodiment of the present invention. The present invention proposes a static-dynamic combined parallel computing method, system, and storage medium, implemented based on a genetic algorithm and the OpenMP framework. The method includes a static task scheduling phase and a dynamic task scheduling phase. In the static task scheduling phase, the computational task is divided into several sub-tasks. Before actual execution, a genetic algorithm is used to find the optimal task execution sequence with the shortest total execution time. This sequence arranges the execution order of each sub-task, and all sub-tasks are added to a low-priority task queue according to this sequence. In the dynamic task scheduling phase, when actually executing a computational task, tasks are preferentially retrieved from the high-priority task queue. If there are no tasks in the high-priority queue, tasks are retrieved from the low-priority queue. After retrieving a task, the current actual computational task is determined based on information such as current CPU utilization and memory usage. If the current system resources are insufficient to execute the task, it is added to the high-priority queue so that it can be executed first when system resources are sufficient. If the current system resources are sufficient, the task is executed directly. This process is repeated until all tasks in all queues are completed. Static scheduling is the primary method, and dynamic scheduling is secondary. The static-dynamic parallel computing system comprises a task dispatcher module, a queue manager module, a task executor module, and a system monitor module. The task dispatcher module uses a genetic algorithm to generate a static scheduling sequence, while the task executor module uses a dynamic scheduling strategy to temporarily adjust the task execution order and execute tasks in parallel using OpenMP. Additionally, the system has a data management module responsible for managing various types of data involved in the computation process. After creating a computation task, the system calculates the task information and determines whether to enter the parallel computing submodule for parallel computation. If so, it enters the task dispatcher; otherwise, it performs serial computation using the input file of the data storage module. The parallel computation results and the serial computation results are stored as output files.

[0083] 1. The task dispatcher module is responsible for handling dependencies between tasks, managing task states, determining the initial task execution sequence using a genetic algorithm, and adding executable tasks to the task queue. Under limited CPU cores and memory constraints, it finds the optimal task execution sequence with the shortest execution time. Based on its understanding of the problem model, the task dispatcher module designs a static task decomposition table and then decomposes the computational tasks. For each operator involved, it further decomposes them according to its internal computational block logic to obtain a set of subtasks. After obtaining a set of subtasks to be scheduled, the genetic algorithm is used as the core method of static task scheduling to find the optimal task execution sequence. For each subtask, iterative genetic operations of chromosome encoding, initial population generation, selection, and crossover / kernel mutation are performed, retaining the task execution sequence with high fitness, ultimately obtaining the optimal task execution sequence. A directed acyclic graph is constructed to describe the dependencies between subtasks. In the actual implementation, an adjacency matrix is ​​used to store the in-degree and out-degree of each task to check whether its predecessor task is completed and whether subsequent tasks can begin.

[0084] 2. The task executor module retrieves tasks from the queue manager module and determines whether a task can be executed based on its requirements and the current system status obtained from the system monitoring module. This enables dynamic task scheduling: if a task cannot be executed, it is placed in a high-priority queue for priority execution next time; if a task can be executed, it is allocated to currently idle CPU cores and the core load is balanced as much as possible; after a task is completed, its execution status is sent to the task dispatcher module, which uses this information to ensure the dependencies between tasks and adds executable tasks to the task queue.

[0085] The task executor module relies on the OpenMP framework to achieve parallel task execution, fully utilizing the computing power of multi-core CPUs. OpenMP provides a high-level abstract description of parallel algorithms. By setting parallel regions in the source code to specify the intent, the compiler can automatically parallelize the program and add synchronization, mutual exclusion, and communication where necessary. When these pragmas are ignored, or the compiler does not support OpenMP, the program degenerates into a normal serial program. The code can still operate normally, but multithreading cannot be used to accelerate program execution. In OpenMP, users can set the number of threads in the parallel region using the `omp_set_num_threads` function to obtain the parallelism of the entire system representing the number of CPU cores used for computation. Setting thread-CPU core affinity establishes the mapping between threads and CPU cores, specifying the specific CPU core on which a thread runs and assigning tasks to specific CPU cores.

[0086] 3. The queue manager module uses a single queue overall. The task dispatcher module adds subtasks sequentially to the queue manager module and then retrieves tasks from the queue for execution. To adapt to machines with different numbers of cores and computing tasks, the entire system uses a single queue. To meet the needs of dynamic scheduling, a multi-level priority queue is adopted: when the task executor module retrieves tasks, it checks each queue in descending order of priority, finds the first non-empty queue, and retrieves the task from it for execution. The queue is divided into a low-priority queue and a high-priority sequence. All tasks initially enter the low-priority queue. If a task cannot be executed, it is added to the high-priority queue, and then the next task is executed. In the multi-level priority queue, a task can only be placed in the multi-level priority queue for scheduling if its dependencies are satisfied.

[0087] 4. The system monitor module uses a monitor to monitor the system's resource usage in real time and provides necessary resource status information to the task dispatcher and task executor modules to assist them in task scheduling and execution decisions; CPU utilization uses the Windows performance counter, called through the pdh library, to calculate the utilization of each core and simultaneously calculate the total CPU utilization; memory usage can be obtained using the Global Memory Status Ex function, the expression of which is:

[0088]

[0089] Used physical memory = Total physical memory - Available physical memory;

[0090]

[0091] 5. The task model abstracted in this invention is as follows: tasks need to be executed on a single CPU at the same time and do not involve preemption mechanisms. In addition to being constrained by the CPU cores, they are also constrained by the size of the memory space. Each task needs to load its required data into memory when performing calculations. The memory space occupied by calculation tasks performed at the same time cannot exceed the available memory space. At the same time, tasks also have data sharing relationships, that is, two tasks will use the same data when performing calculations. In order to improve memory space utilization and reduce the time lost in loading and removing data in memory, tasks using the same data should be allocated to run simultaneously as much as possible during task scheduling.

[0092] The task model is represented by a directed acyclic graph, G = <T, E>. Here, T is the set of task nodes, T = {i | 1 ≤ i ≤ N} represents the set of N tasks; E = {e...} ij Let} be the set of directed edges between tasks, and let edge e be the set of directed edges between tasks. ij This means that task i can only be executed after task j is executed. Also, assume t = {t1, t2, ..., t...} N} represents the execution time of the task; m = {M ij} represents the size of the data that needs to be loaded during task computation, in M. ij This indicates that tasks i and j have a data-sharing relationship, and the data they need to load for computation is the same. In task scheduling, besides T = {T1, T2, ..., T...},... n In addition to the task set}, there is also P = {P1, P2, ..., P} m A set of CPU cores, where each P i Represents a core, for each computational task T i The task execution time is estimated based on the time complexity of its internal operator operation logic and the scale of data used, and the memory space occupied is estimated based on the data that needs to be loaded into memory.

[0093] Therefore, the overall task scheduling condition of this system can be defined as follows: In a system with P available CPU cores and a memory size of M, for N tasks with an execution time of t... i M needs to be loaded i Task T stores large and small data in memory and finds an optimal task execution sequence that satisfies the dependencies and data sharing relationships between tasks. The dependencies between tasks are maintained using a directed acyclic graph (DAG). Considering that performing dependency checks on the chromosome sequence every time in scenarios with a large task load would increase the overall time complexity, for a known DAG, the task level value h can be defined to represent the level of a task node based on the dependencies between tasks, as follows:

[0094]

[0095] Where pre(i) represents all direct predecessor nodes of task number i; maxh(j) represents the maximum level value among all predecessor nodes of task number i; This represents the empty set.

[0096] Figure 4 This is a schematic diagram of the structure of a directed acyclic graph according to an embodiment of the present invention. The task level values ​​are {1; 2, 2; 3, 3; 4, 4, 4; 5, 5; 6}, and tasks with different level values ​​are separated by semicolons. All tasks are layered according to their level values, with tasks of the same level value placed in the same layer. The task set of the j-th layer is T. j =T{h=j}, therefore divided into 6 levels: T1={1}, T2={2,3}, T3={4,5}, T4={6,7,8}, T5={9,10}, T6={11}. Tasks with the same level value can be executed in parallel.

[0097] 6. This invention employs a Genetic Algorithm (GA) to optimize task allocation and scheduling on multi-core processors, achieving optimal task allocation and minimizing the total task completion time. The GA starts with a population representing the potential solutions to the problem. A population consists of a certain number of individuals encoded with genes; each individual is essentially a chromosome-based entity. First, the encoding process from phenotype to genotype is implemented, including but not limited to binary encoding. After the initial population is generated, it evolves generation by generation according to the principles of survival of the fittest and natural selection, producing increasingly better approximate solutions. In each iteration, individuals are selected based on their fitness in the problem domain, and crossover and mutation are performed to generate a new population representing the solution set. This process leads to a population where subsequent generations are more adapted to the environment than previous generations, much like natural evolution. The optimal individual in the final generation is decoded as the approximate optimal solution to the problem. The process of a genetic algorithm includes: randomly generating a certain number of chromosomes to obtain an initial population; repeatedly calculating the fitness function value for each chromosome in the population and selecting chromosomes with high fitness to enter the next generation; performing crossover on the selected chromosomes to generate new individuals; randomly changing some genes in the chromosomes to increase diversity; iterating until a preset termination condition is met, which includes, but is not limited to, the number of iterations and the fitness threshold; and finally selecting the chromosome with the highest fitness as the final task scheduling scheme.

[0098] 6.1 Chromosome Encoding. After expressing a specific problem using gene sequences, the gene sequences are compared and processed to obtain the corresponding gene sequence results. These results are then decoded to become the actual solution. The encoding method directly affects subsequent operations of the genetic algorithm, and the chosen encoding method influences the algorithm's convergence speed and optimal value. This invention considers that task scheduling involves tasks T = {T1, T2, ..., T...}. n} and the corresponding CPU core P = {P1, P2, ..., P m For each task, its gene value is calculated and its gene value sequence is used to construct a chromosome. The gene value encoding method is as follows:

[0099] v i =p i +i×P;

[0100] Where i represents the subtask number; p i This indicates the CPU core number of subtask number i; P indicates the total number of CPU cores.

[0101] The number of processors allocated to a task can be deduced by taking the remainder between the task's gene value and the total number of processors. The expression is as follows:

[0102] p i =mod(vi , P);

[0103] 6.2 Population Initialization. Common initialization strategies for genetic algorithms include random initialization and uniform initialization. This invention adopts hierarchical random initialization. Considering the need to reduce the load on individual cores and shorten task completion time, tasks at each level should be evenly distributed across processors to avoid over-allocation of tasks to a single core. Assuming the total number of CPU cores is P, and the total number of subtasks at each level is n, then n / P = a…b (where b is the remainder). b subtasks (b < P) are randomly selected from the total tasks and randomly assigned to different CPU cores; for the remaining a × P tasks, each CPU core randomly selects a tasks from them.

[0104] for Figure 4 Given a directed acyclic graph (DAG) with 2 processor cores (P), for T4 = {6, 7, 8}, n / P = 3 / 2 = 1…1. First, randomly assign one task to any core, then evenly distribute the remaining tasks between P1 and P2; thus, one allocation for T4 is {P1, P2, P1}. Based on this allocation principle, an initial allocation relationship for the 11 tasks is P = {P1; P1, P2; P1, P2; P1, P2, P1; P2, P1; P1}. After generating the initial allocation of processor cores, the genome sequence of each individual can be calculated according to the gene value encoding method, such as the individual V0 = {3; 5, 8; 9, 12; 13, 16, 17; 20, 21; 23} corresponding to the initial allocation relationship P. Next, gene values ​​are randomly swapped according to the following rules: Assuming a processor has m tasks, two tasks on that processor are randomly selected, and their gene values ​​are swapped. This swapping is repeated m / 2 times. This operation is performed on each processor to obtain a new chromosome. Individual V0 = {3; 5, 8; 9, 12; 13, 16, 17; 20, 21; 23}; After the random swapping, a possible individual is V = {23; 5, 20; 13, 16; 9, 12, 21; 8, 3; 17}. At this point, the gene values ​​corresponding to tasks 1 to 11 are 23, 5, 20, 13, 16, 9, 12, 21, 8, 3, 17 respectively. For tasks with the same level value, they are scheduled according to their gene values ​​from largest to smallest. The task execution order for individual V is: 1→3→2→5→4→8→7→6→9→10→11. Based on the above formula, the tasks assigned to each core are deduced as follows: P1: 1→2→4→8→6→10→11; P2: 3→5→7→9.

[0105] 6.3 The fitness function represents an individual's adaptability to its environment. The higher an individual's fitness, the greater the probability that the individual will be passed on to the next generation. The overall task completion time, Makespan, is used to evaluate fitness. For each individual, its task scheduling sequence is obtained by decoding its chromosome sequence. The time Makespan required to complete all tasks according to the task scheduling sequence is calculated. A smaller Makespan indicates a better task scheduling sequence, while a larger Makespan indicates a worse one. To align with the principle that individuals with higher fitness should be retained, and considering data sharing among tasks, tasks using the same data should be scheduled for execution at the same time. To satisfy this soft constraint, a penalty term is added to the fitness function to account for data sharing during the genetic process. λ represents the penalty weight coefficient used to control the severity of the penalty. When decoding an individual's chromosomes, the data sharing relationship is simultaneously checked. If a data sharing relationship is not satisfied, a positive penalty term is applied to the individual during fitness function calculation, reducing its fitness value. This lowers the probability of the individual entering the next generation during subsequent selection, thus eliminating inferior individuals and preserving the genotypes of superior individuals. The expression is:

[0106]

[0107] 6.4 Selection operations are used to select superior individuals and reproduce the next generation. First, the fitness of each individual is calculated and selected according to certain rules, and then the population is updated. Common genetic algorithm selection operations include Roulette Wheel Selection and Tournament Selection. Individuals are selected according to their fitness ratio, and individuals with higher fitness have a higher probability of being selected. Roulette Wheel Selection is a random sampling method in which the probability of selecting an individual is proportional to its fitness. Tournament Selection randomly selects a certain number of individuals, and then selects the individual with the highest fitness as the parent individual. The advantage of this method is that it can ensure that some individuals with lower fitness also have a chance to be selected. The specific process of the Roulette Wheel Selection method used in this algorithm includes: generating a uniformly distributed random number r in the interval [0,1]; if r≤q(1), then the first initial task execution sequence V1 is selected; if q(k-1)<r≤q(k) (2≤k≤SN, SN is the population size), then the initial task execution sequence V1 is selected. i Selected; q(i) is the cumulative probability of chromosome, P(V i ) for individual V i The probability of being selected, fit(V) i ) represents the initial task execution sequence V i The fitness is expressed as:

[0108]

[0109] 6.5 Crossover is an operation in which two paired chromosomes exchange parts of their genes to form two new individuals. Crossover operations include, but are not limited to, single-point crossover, multi-point crossover, consistent crossover, and sequential crossover. This invention uses two-point crossover based on hierarchy. Using crossover operations within the same level avoids disrupting the dependencies between tasks, thereby avoiding the need to check whether the chromosomes satisfy the dependencies after crossover and improving the speed of the algorithm itself. Two-point crossover within a hierarchy involves selecting two adjacent parent chromosomes based on the crossover rate and performing a crossover operation based on random numbering to generate two offspring. For the two individuals V1 and V2 undergoing crossover, an initial crossover point and a cutoff crossover point are randomly selected in V1, with a total crossover task count of l. This initial crossover point and cutoff crossover point are also the initial crossover point and cutoff crossover point for V2. A small segment of individuals from V1 and a small segment of individuals from V2 are swapped as a whole, thus obtaining new individuals S1 and S2.

[0110] 6.6 Mutation operations replace the gene values ​​at certain loci in an individual's chromosome coding string with other alleles at those loci, thereby forming a new individual. Common mutation methods include single-point mutation and uniform mutation. Mutations that result in decreased fitness will gradually eliminate individuals after multiple iterations. This invention uses single-point mutation, randomly selecting two different mutation points and exchanging their gene values ​​to obtain a new individual. A mutation probability is set to determine whether mutation occurs. A random number is generated for each offspring; if the random number is less than the mutation probability, the offspring is mutated to generate a new individual.

[0111] 6.7 Evolutionary Iteration. Each round of crossover, mutation, and selection operations is called an iterative evolution. The algorithm stops iterating and outputs the final optimal individual when certain conditions are met. Common exit conditions include reaching the maximum number of iterations and the individual not showing significant improvement in the past few generations. This application uses the second method to determine whether the optimal individual has been found. By storing the fitness value of the optimal individual in each generation, it is determined whether the fitness value of the optimal individual has not changed for multiple generations. If it is still changing, the iteration continues; otherwise, the current optimal individual is output and decoded to obtain the optimal task scheduling sequence.

[0112] In summary, this invention provides a dynamic-static combined parallel computing method, system, and storage medium. The computing task is decomposed into multiple subtasks. A directed acyclic graph (DAG) is constructed based on the multiple subtasks and their processing order relationships. The subtasks in the DAG are allocated to multiple CPU cores according to a preset allocation method. The encoding value of each subtask is calculated based on a gene value encoding method, and the encoding values ​​of each subtask are randomly swapped to obtain multiple initial task execution sequences. A genetic algorithm is used for iterative updates with the goal of generating the task execution sequence with the shortest completion time. After fitness calculations based on task completion time and penalty terms, a roulette wheel selection method is used to retain a portion of the initial execution sequences. For any two initial execution sequences in the retained portion, the initial execution sequence is selected from the remaining sequences. Some encoded values ​​are swapped, and the positions of some encoded values ​​in the generated task execution sequences are adjusted. The process is iteratively updated until a preset termination condition is met to obtain the optimal task execution sequence. Multiple subtasks are then added to the low-priority task queue according to the optimal task execution sequence. Subtasks are selected from the high-priority task queue first. The current CPU utilization and memory usage are used to determine whether to execute the subtask. If there is enough CPU to process the selected subtask, multiple subtasks are allocated to idle CPU cores for parallel execution according to the number of threads in the parallel area and the thread allocation relationship set by the preset multi-threaded programming interface. If there is not enough CPU to process the selected subtask, the subtask is added to the high-priority queue. The process is repeated until all subtasks are completed and the parallel computing result is obtained.

[0113] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned edge computing server deployment method. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.

[0114] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0115] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0116] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0117] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A dynamic-static combined parallel computing method, characterized in that, The method includes the following steps: Obtain the computation task to be processed and decompose the computation task into multiple sub-tasks, and construct a directed acyclic graph based on the multiple sub-tasks and the processing order relationship between the sub-tasks; Multiple subtasks in the directed acyclic graph are allocated to multiple CPU cores according to a preset allocation method. The encoding value of each subtask is calculated according to the gene value encoding method, and the encoding values ​​of each subtask are randomly swapped to obtain multiple initial task execution sequences. Genetic algorithm is used to iteratively update the sequence with the shortest task completion time. After fitness calculation of multiple initial task execution sequences based on task completion time and penalty term, a roulette wheel selection method is used to retain some initial execution sequences. The encoding values ​​of any two initial execution sequences in the retained initial execution sequences are swapped, and the positions of the encoding values ​​of the generated task execution sequences are adjusted. The iteration is updated until a preset termination condition is met to obtain the optimal task execution sequence. Multiple subtasks are added to a low-priority task queue according to the optimal task execution sequence. A subtask is selected from the high-priority task queue. If there is no subtask in the high-priority task queue, a subtask is selected from the low-priority task queue. The current CPU utilization and memory usage determine whether to execute the subtask. If the CPU system resources are sufficient to process the selected subtask, multiple subtasks are allocated to idle CPU cores for parallel execution according to the number of threads in the parallel zone and the thread allocation relationship set by the preset multi-threaded programming interface. If the CPU system resources are insufficient to process the selected subtask, the subtask is added to the high-priority queue. This process is repeated until all subtasks are completed and the parallel computing result is obtained.

2. The dynamic-static combined parallel computing method according to claim 1, characterized in that, The process of allocating multiple subtasks in the directed acyclic graph to multiple CPU cores according to a preset allocation method, calculating the encoding value of each subtask according to the gene value encoding method, and randomly swapping the encoding values ​​of each subtask to obtain multiple initial task execution sequences includes: Multiple subtasks are assigned to multiple CPU cores to obtain an initial allocation relationship. The encoding value of each subtask is calculated based on the CPU core number, subtask number, and total number of CPU cores, and the task execution sequence corresponding to the initial allocation relationship is constructed. When the number of subtasks with the same level value can be evenly distributed among the CPU cores, they are evenly distributed. When the number of subtasks with the same level value cannot be evenly distributed among the CPU cores, the remainder is obtained by dividing the number of subtasks by the number of CPU cores. The subtasks with the remainder are selected and randomly assigned to CPU cores. Then, the remaining subtasks are evenly distributed among the CPU cores. Multiple initial task execution sequences are obtained by randomly swapping the encoding values ​​of subtasks on each CPU core in the task execution sequence corresponding to the initial allocation relationship.

3. The dynamic-static combined parallel computing method according to claim 2, characterized in that, The expression for calculating the encoded value of each subtask based on the CPU core number, subtask number, and total number of CPU cores is as follows: in i =p i +i×P; Where i represents the subtask number; p i This indicates the CPU core number of subtask number i; P indicates the total number of CPU cores.

4. The dynamic-static combined parallel computing method according to claim 1, characterized in that, The expression for calculating the fitness of multiple initial task execution sequences using task completion time and penalty terms is as follows: Where λ represents the penalty weight coefficient; Makespan represents the total task completion time; and Penalty represents the penalty term.

5. The dynamic-static combined parallel computing method according to claim 4, characterized in that, The process of using the roulette wheel selection method to retain a portion of the initial execution sequence, swapping partial encoded values ​​between any two initial execution sequences within the retained portion, and then adjusting the positions of partial encoded values ​​in the resulting task execution sequences includes: A roulette table is obtained based on the proportion of the fitness of each initial task execution sequence to the total fitness value. The roulette table is used to randomly select and retain the selected initial task execution sequence as the task execution sequence to be processed. Randomly select any two tasks from the sequence of tasks to be processed, and set an initial crossover point and a cutoff crossover point at the same position in the two task sequences. Then, according to a preset crossover probability, swap the two segments of encoded values ​​intercepted through the initial crossover point and the cutoff crossover point. In the task execution sequences generated after the exchange, two mutation points are randomly selected and their encoded values ​​are exchanged according to a preset mutation probability.

6. The dynamic-static combined parallel computing method according to claim 1, characterized in that, Determining whether to execute a subtask based on current CPU utilization and memory usage also includes: The utilization of each CPU core is calculated using a preset performance counter, and the total CPU utilization is calculated as well. Use the Global Memory Status Ex function to obtain memory usage information.

7. The dynamic-static combined parallel computing method according to claim 1, characterized in that, Multiple subtasks with the same level value are executed in parallel, where the level value is expressed as: Where pre(i) represents all direct predecessor nodes of task number i; maxh(j) represents the maximum level value among all predecessor nodes of task number i; This represents the empty set.

8. A static-dynamic parallel computing system, characterized in that, The system is used to execute the dynamic-static combined parallel computing method as described in any one of claims 1 to 7, and the system includes: The task dispatcher module is used to acquire the computing tasks to be processed and decompose the computing tasks into multiple subtasks. The multiple subtasks in the directed acyclic graph are allocated to multiple CPU cores according to a preset allocation method. The encoding value of each subtask is calculated according to the gene value encoding method, and the encoding values ​​of each subtask are randomly swapped to obtain multiple initial task execution sequences. The genetic algorithm is used to iteratively update the task execution sequence with the goal of generating the task with the shortest completion time. After calculating the fitness of the multiple initial task execution sequences through task completion time and penalty terms, a roulette wheel selection method is used to retain some initial execution sequences. The encoding values ​​of any two initial execution sequences in the retained initial execution sequences are swapped, and the positions of the encoding values ​​of the generated task execution sequences are adjusted. The iterative update continues until a preset termination condition is met to obtain the optimal task execution sequence. The multiple subtasks are then added to the low-priority task queue according to the optimal task execution sequence. The queue manager module is used to add multiple subtasks to a low-priority task queue according to the optimal task execution sequence and to determine whether there are subtasks in the high-priority queue. The queue manager uses a single task queue and adopts a multi-level priority queue. After receiving the subtasks added by the task dispatcher, it inputs them into the task executor for execution. The task executor module is used to select a subtask from the high-priority task queue after the queue manager module determines that there is a subtask in the high-priority task queue. When the queue manager module determines that there is no subtask in the high-priority task queue, it selects a subtask from the low-priority task queue. It determines whether to execute the subtask based on the current CPU utilization and memory usage. When the CPU system resources are sufficient to process the selected subtask, it allocates multiple subtasks to idle CPU cores for parallel execution based on the number of threads in the parallel area and the thread allocation relationship set by the preset multi-threaded programming interface. When the CPU system resources are insufficient to process the selected subtask, it adds the subtask to the high-priority queue. The process is repeated until all subtasks are completed and the parallel computing result is obtained. The system monitor module is used to monitor CPU utilization and memory usage in real time.

9. The static-dynamic combined parallel computing system according to claim 8, characterized in that, The static-dynamic combined parallel computing system also includes: The data management module is used to manage and store various types of data used and generated during the calculation process.

10. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method as described in any one of claims 1 to 7.

Images