Process mapping optimization method and device based on genetic algorithm and storage medium

Abstract

The application relates to a process mapping optimization method and device based on a genetic algorithm and a storage medium. The method comprises the following steps: obtaining communication data corresponding to a plurality of sub-processes of a target computing program, wherein the communication data comprises process communication volume and process communication time delay; performing real number coding on the process communication volume and the process communication time delay to generate a first process mapping coding array, wherein the first process mapping coding array comprises a plurality of first codes, and the first codes are used for representing the corresponding relationship between a first process core for executing a sub-process and the sub-process; performing genetic evolution operation on the first codes based on the genetic algorithm, performing redundancy scheduling processing on the first codes after the genetic evolution operation, and generating a second process mapping coding array with a plurality of second codes; determining the fitness of the second codes, and performing genetic evolution operation updating on the second codes in the second process mapping coding array according to the fitness to generate a target process mapping coding array, so that a process mapping optimization result is obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and particularly to a process mapping optimization method and device based on a genetic algorithm and a storage medium. BACKGROUND

[0002] High Performance Computing (HPC) refers to using resources beyond general standard computers to complete large-scale computing tasks, and is an application discipline combining computer networks, computer architecture and distributed computing and other disciplines. HPC is widely used in fields such as molecular simulation, weather forecasting, artificial intelligence model training and the like.

[0003] In related technologies, an HPC cluster exhibits the characteristics of multiple layers, multiple nodes and many cores. At the same time, with the maturity of application systems such as cloud computing, edge computing and federated learning, computing resources located on remote nodes can also be used by the same multi-process program, thus resulting in higher inter-subprocess communication overhead. For an HPC cluster, when there is a computing demand of a large-scale parallel program, hundreds or thousands of parallel subprocesses need to be created. Since the number of processes and the number of computing resources involved in the task are large, the solution space of the process mapping problem is also very large, which in turn causes the process mapping scheme of the HPC cluster to have the problems of uneven process allocation and uneven number of processes carried by cores, large fluctuations in the number of processes carried by different cores, low running efficiency of high-performance tasks and large communication overhead.

[0004] At present, there is no effective solution to the problem of low running efficiency and large communication overhead of high-performance tasks in the process mapping scheme of the HPC cluster in related technologies. SUMMARY

[0005] Embodiments of the present application provide a process mapping optimization method and device based on a genetic algorithm and a storage medium, to at least solve the problem of low running efficiency and large communication overhead of high-performance tasks in the process mapping scheme of the HPC cluster in related technologies.

[0006] In a first aspect, an embodiment of the present application provides a process mapping optimization method based on a genetic algorithm, comprising: obtaining communication data corresponding to a plurality of sub-processes of a target computing program, wherein the communication data comprises process communication volume and process communication latency; performing real number coding on the process communication volume and the process communication latency to generate a first process mapping coding array, wherein the first process mapping coding array comprises a plurality of first codes corresponding to the sub-processes, and the first codes are used to represent a corresponding relationship between a first process core executing the sub-processes and the sub-processes; performing genetic evolution operations on the first codes based on a preset genetic algorithm, and performing redundancy scheduling processing on the first codes that have completed the genetic evolution operations to generate a second process mapping coding array with a plurality of second codes, wherein the genetic evolution operations at least include one of tournament selection, uniform crossover, and mutation, and the second codes are used to represent a corresponding relationship between a second process core executing the sub-processes and the sub-processes; determining an adaptability corresponding to the second codes, and performing genetic evolution operation updates on the second codes in the second process mapping coding array according to the adaptability to generate a target process mapping coding array, and obtaining a process mapping optimization result, wherein the adaptability is determined according to the process communication volume and the process communication latency corresponding to the second codes, and the adaptability is used to represent a communication overhead of the corresponding process core executing the sub-processes.

[0007] In a second aspect, an embodiment of the present application provides a process mapping optimization device based on a genetic algorithm, comprising:

[0008] An obtaining module is configured to obtain communication data corresponding to a plurality of sub-processes of a target computing program, wherein the communication data comprises process communication volume and process communication latency.

[0009] A coding module is configured to perform real number coding on the process communication volume and the process communication latency to generate a first process mapping coding array, wherein the first process mapping coding array comprises a plurality of first codes corresponding to the sub-processes, and the first codes are used to represent a corresponding relationship between a first process core executing the sub-processes and the sub-processes.

[0010] A processing module is configured to perform genetic evolution operations on the first codes based on a preset genetic algorithm, and perform redundancy scheduling processing on the first codes that have completed the genetic evolution operations to generate a second process mapping coding array with a plurality of second codes, wherein the genetic evolution operations at least include one of tournament selection, uniform crossover, and mutation, and the second codes are used to represent a corresponding relationship between a second process core executing the sub-processes and the sub-processes.

[0011] The generating module is configured to determine an adaptability corresponding to the second encoding, and perform a genetic evolution operation update on the second encoding in the second process mapping encoding array according to the adaptability, to generate a target process mapping encoding array, and obtain a process mapping optimization result, wherein the adaptability is determined according to the process communication traffic and the process communication time delay corresponding to the second encoding, and the adaptability is used to represent a communication overhead of a corresponding process core in executing the sub-process.

[0012] In a third aspect, an electronic device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the process mapping optimization method based on a genetic algorithm according to the first aspect when executing the computer program.

[0013] In a fourth aspect, a storage medium is provided, which stores a computer program executable by a processor, and the computer program implements the steps of the process mapping optimization method based on a genetic algorithm according to the first aspect when executed by the processor.

[0014] Compared with the related art, the process mapping optimization method, device and storage medium based on a genetic algorithm provided by the embodiments of the present application, by acquiring communication data corresponding to a plurality of sub-processes of a target computing program, wherein the communication data includes process communication volume and process communication latency; performing real number coding on the process communication volume and process communication latency to generate a first process mapping coding array, wherein the first process mapping coding array includes a plurality of first codes corresponding to the sub-processes, and the first code is used to represent the correspondence between a first process core executing the sub-process and the sub-process; performing genetic evolution operation on the first code based on a preset genetic algorithm, and performing redundancy scheduling processing on the first code after the genetic evolution operation to generate a second process mapping coding array with a plurality of second codes, wherein the genetic evolution operation at least includes one of tournament selection, uniform crossover and mutation, and the second code is used to represent the correspondence between a second process core executing the sub-process and the sub-process; determining the fitness of the second code, and updating the second code in the second process mapping coding array according to the fitness to generate a target process mapping coding array, and obtaining a process mapping optimization result, wherein the fitness is determined according to the process communication volume and the process communication latency corresponding to the second code, and the fitness is used to represent the communication overhead of the corresponding process core executing the sub-process; by mapping the process group with larger communication volume to the core group with smaller communication cost, and averaging the number of processes carried by each core, the communication overhead in the task is reduced, the running speed of the task is improved, the running efficiency of the high-performance task is improved, and the problem of low running efficiency and large communication overhead of the process mapping scheme of the HPC cluster in the related art is solved.

[0015] The details of one or more embodiments of the present application are presented in the following drawings and description to make other features, objects and advantages of the present application more apparent. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings illustrated herein are used to provide further understanding of the present application, constitute a part of the present application, the schematic embodiments of the present application and the description thereof are used to explain the present application, and do not constitute improper limitation on the present application. In the drawings:

[0017] Figure 1 It is a hardware structure block diagram of the terminal of the process mapping optimization method based on a genetic algorithm of the embodiments of the present application;

[0018] Figure 2 It is a flow chart of the process mapping optimization method based on a genetic algorithm according to the embodiments of the present application;

[0019] Figure 3is a flowchart of a process optimization method according to a preferred embodiment of the present application;

[0020] Figure 4 is a structural block diagram of a process mapping optimization apparatus based on a genetic algorithm according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the objects, technical solutions and advantages of the present application clearer, the present application is described and explained below in connection with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of the present application. In addition, it can be understood that although the efforts made in this development process can be complex and lengthy, some designs, manufacturing or production changes made on the basis of the technical content disclosed in the present application by those of ordinary skill in the art related to the content disclosed in the present application are only routine technical means and should not be understood as insufficient disclosure of the content disclosed in the present application.

[0022] In the present application, "embodiments" means that the specific features, structures or characteristics described in connection with the embodiments can be included in at least one embodiment of the present application. The phrase appears at various places in the specification does not necessarily mean the same embodiment, nor is it an independent or alternative embodiment to other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in the present application can be combined with other embodiments without conflict.

[0023] Unless otherwise defined, technical terms and scientific terms used in the present application shall have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terms "a", "an", "one", "this", and the like, as used in the present application, do not denote a quantity of particular aforementioned material or element. The terms "include", "comprise", "have", and any variations thereof, as used in the present application, are intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or device that comprises a list of steps or units (elements) is not limited to the listed steps or units, but can also include additional steps or units not listed, or can also include other steps or units inherent to the process, method, system, product, or device. The term "multiple stages" refers to more than or equal to two stages. The term "and / or" describes the association relationship of associated objects, which means that there can be three relationships, for example, "A and / or B" can mean that A exists alone, A and B exist together, and B exists alone. The terms "first", "second", "third", and the like in the present application are only to distinguish similar objects, and do not represent a specific order of the objects.

[0024] Before the embodiments of the present application are described, the related technologies involved in the present application are described as follows:

[0025] High-performance computing, also known as HPC, is the use of resources beyond general standard computers to complete large-scale computing tasks, and is an application discipline combining computer networks, computer architecture, and distributed computing and other disciplines.

[0026] MPI communication interface

[0027] MPI is a cross-language communication protocol used to write parallel computers, supporting point-to-point and broadcast, and the goal of MPI is high performance, large scale, and portability; MPI is an industry standard for message passing.

[0028] Pingpong test

[0029] Pingpong test is a program for testing the time required for sending Ping and receiving message Pong back and forth between any two computing units.

[0030] The method embodiments provided in the embodiments can be executed in a terminal, a computer, or a similar computing device. Taking the case of running on a terminal, Figure 1 is a hardware structure block diagram of a terminal based on the process mapping optimization method of the genetic algorithm of the embodiments of the present application. As Figure 1 shown, the terminal can include one or more Figure 1The terminal shown in FIG. 1 includes only one processor 102 (the processor 102 can include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Optionally, the terminal can further include a transmission device 106 for communication function and an input / output device 108. Those skilled in the art can understand that Figure 1 The structure shown in FIG. 1 is only schematic and does not limit the structure of the terminal. For example, the terminal can include more or fewer components than those shown in FIG. 1, or have a different configuration from that shown in FIG. 1. Figure 1 Figure 1 The structure shown in FIG. 1 is only schematic and does not limit the structure of the terminal. For example, the terminal can include more or fewer components than those shown in FIG. 1, or have a different configuration from that shown in FIG. 1.

[0031] The memory 104 can be used to store computer programs, such as software programs of application software and modules, such as a computer program corresponding to the process mapping optimization method based on genetic algorithm in the embodiments of the present application. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, that is, implements the above-mentioned method. The memory 104 can include a high-speed random access memory, and can further include a non-volatile memory, such as one or more magnetic storage devices, a flash memory, or other non-volatile solid-state memories. In some examples, the memory 104 can further include a memory remotely arranged with respect to the processor 102, and these remote memories can be connected to the terminal 10 through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0032] The transmission device 106 is used to receive or send data via a network. Specific examples of the network can include a wireless network provided by a communication provider of the terminal 10. In one example, the transmission device 106 includes a network adapter (NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet in a wireless manner.

[0033] The embodiments of the present application provide a process mapping optimization method based on genetic algorithm running on the terminal, Figure 2 The process mapping optimization method based on genetic algorithm according to the embodiments of the present application is shown in FIG. 2, which includes the following steps: Figure 2

[0034] In step S201, communication data corresponding to a plurality of sub-processes of a target computing program is obtained, wherein the communication data includes process communication volume and process communication time delay.

[0035] ​​In the embodiment, by collecting the process communication amount transmitted by each sub-process of a high-performance computing program (corresponding target computing program) through the MPI interface of the cross-language communication protocol, by normalizing the collected process communication amount, and establishing a process communication amount matrix M based on the data of the normalized process communication amount, M ∈ R n×n , n represents the number of sub-processes, and the element M ij in the process communication amount matrix M represents the communication amount of the sub-process i to the sub-process j; in the embodiment, the obtained communication data further includes the communication delay between each process core allocated for use by the target computing program, and a Pingpong test is performed to collect the communication delay between each process core under the same process communication amount, and the collected corresponding communication delay is normalized, and a process communication cost matrix P is established based on the normalized communication delay, P ∈ R q×q , q represents the number of process cores, and the element P ij in P represents the communication cost of the process core i to the process core j.

[0036] In step S202, the process communication amount and the process communication delay are real-coded to generate a first process mapping coding array, wherein the first process mapping coding array includes a plurality of first codes corresponding to the sub-processes, and the first code is used to represent the corresponding relationship between the first process core executing the sub-process and the sub-process.

[0037] In the embodiment, after obtaining the communication data corresponding to the plurality of sub-processes of the target computing program, the hyperparameters of the genetic algorithm are first set and the data initialization is set, specifically, the initial values of the population size PS of genetic evolution, the mutation rate MR of genetic evolution, the crossover rate CR of genetic evolution, the number of participating individuals TS of each round of tournament selection of genetic evolution, the mutation surge step size ratio SR of genetic evolution, and the initial value of the redundancy scheduling observation ratio TR are set.

[0038] In the embodiment, after completing the hyperparameter setting and initialization setting of the genetic algorithm, the initial mapping scheme generated by the initialization is coded, and the initial mapping scheme is generated by mapping the plurality of sub-processes to the plurality of process cores allocated for use by the target computing program according to the obtained process communication amount and process communication delay, that is, the corresponding sub-process is configured to the corresponding process core, and the corresponding sub-process is executed by the process core, the initial mapping scheme represents the configuration information of the original sub-process execution, and then the initial mapping scheme is real-coded to form a first process mapping coding array including a plurality of first codes, that is, a plurality of chromosomes with sub-processes as individuals are obtained, specifically, the corresponding chromosome is set as x = (x0, x1, …, x n-1 ), each element x i of the chromosome satisfies xi ∈ [0, q-1], and x i ∈ N + wherein, x i The value of x i When x i Correspondingly, the first code represents the corresponding relationship between the first process core (determined according to the value of x i ) executing the sub-process i and the sub-process i, and the first code in the first process mapping code array is also associated with the process communication matrix and the process communication cost matrix, so as to determine the communication delay required by the process pair composed of the two first codes under the set process communication, that is, the communication cost.

[0039] In step S203, the first code is subjected to genetic evolution operation based on the preset genetic algorithm, and the first code subjected to the genetic evolution operation is subjected to redundancy scheduling processing to generate a second process mapping code array with multiple second codes, wherein the genetic evolution operation at least includes one of tournament selection, uniform crossover and mutation, and the second code is used to represent the corresponding relationship between the second process core executing the sub-process and the sub-process.

[0040] In this embodiment, after the initial code generates the first process mapping code array, the individuals (first codes) in the first process mapping code array are subjected to genetic evolution operation based on the preset genetic algorithm, that is, subjected to tournament selection, uniform crossover and mutation operation, and then the first code after the genetic evolution operation is subjected to process redundancy scheduling, that is, the process core executing the corresponding sub-process is scheduled, for example, the sub-process carried by the process core m in the first process mapping code array is changed from n to k (k

[0041] In some optional embodiments, the population size PS of genetic evolution is set to 50, the mutation rate MR of genetic evolution is set to 0.005, the crossover rate CR of genetic evolution is set to 0.6, and the redundancy scheduling observation ratio TR is set to 0.1.

[0042] In this embodiment, the first code after the genetic evolution operation is subjected to redundancy scheduling, so that the sub-process carried by each second process core in the second process mapping code array tends to reach the sub-process mean.

[0043] In step S204, the fitness of the second encoding is determined, and the second encoding in the second process mapping encoding array is updated by a genetic evolution operation according to the fitness, to generate a target process mapping encoding array, and obtain a process mapping optimization result, wherein the fitness is determined according to the process communication traffic and the process communication time delay corresponding to the second encoding, and the fitness is used to represent the communication overhead of the process core executing the sub-process corresponding to the second encoding.

[0044] In this embodiment, after the whole genetic evolution operation and redundancy scheduling are completed, the fitness of the individual corresponding to the second encoding in the generated second process mapping encoding array is calculated, to determine whether the fitness of the second encoding generated by the genetic evolution operation and the redundancy scheduling is optimal, that is, whether the communication overhead of the process core executing the sub-process corresponding to each individual is minimal, and then the second encoding in the second process mapping encoding array is iteratively updated through multiple iterations, that is, through multiple genetic evolution operations and redundancy scheduling, to generate a target process mapping encoding array; in this embodiment, the mapping scheme corresponding to the encoding individual in the target process mapping encoding array satisfies that the process group with large communication traffic is mapped to the process core group with small communication cost, and the number of sub-processes carried by each process core is relatively average, the communication overhead of running the task corresponding to the target computing program is small, the running speed is fast, and the running efficiency is high; it can be understood that after multiple iterations, the optimal solution cannot be obtained, that is, the mapping scheme corresponding to the target process mapping encoding data is not the scheme with the smallest communication overhead and the best running efficiency, therefore, in some optional embodiments, the number of iterations is set in advance, for example, the number of iterations is set to 1e^6, and when the number of iterations reaches the set number of iterations, the iteration is directly stopped.

[0045] In this embodiment, after the target process mapping encoding array is determined, the corresponding process mapping optimization result is also determined, specifically, the process core executing the sub-process and the communication overhead required under the set process communication traffic are determined based on the corresponding relationship between the process core executing the sub-process and the sub-process represented by the encoding individual in the target process mapping encoding array.

[0046] By the steps S201 to S204, the communication data corresponding to the plurality of sub-processes of the target computing program is obtained, wherein the communication data includes process communication volume and process communication delay; the process communication volume and the process communication delay are real number coded to generate a first process mapping coding array, the first process mapping coding array includes a plurality of first codes corresponding to the sub-processes, and the first code is used to represent the correspondence between the first process core executing the sub-process and the sub-process; the first code is subjected to genetic evolution operation based on a preset genetic algorithm, and the first code subjected to the genetic evolution operation is subjected to redundancy scheduling processing to generate a second process mapping coding array having a plurality of second codes, the genetic evolution operation at least includes one of tournament selection, uniform crossover and mutation, and the second code is used to represent the correspondence between the second process core executing the sub-process and the sub-process; the fitness corresponding to the second code is determined, and the second code in the second process mapping coding array is updated by genetic evolution operation according to the fitness to generate a target process mapping coding array, and a process mapping optimization result is obtained, the fitness is determined according to the process communication volume and the process communication delay corresponding to the second code, and the fitness is used to represent the communication overhead of the corresponding process core executing the sub-process; by mapping the process group with larger communication volume to the core group with smaller communication cost and averaging the number of processes carried by each core, the communication overhead in the task is reduced, the running speed of the task is improved, the running efficiency of the high-performance task is improved, and the problem of low running efficiency and large communication overhead of the process mapping scheme of the HPC cluster in the related art is solved.

[0047] It should be noted that the process mapping optimization method of the embodiments of the present application is used for the purposeful scheduling of the sub-processes created by the high-performance computing task, the sub-processes with larger communication volume are mapped to the process core groups with smaller communication cost as much as possible, and the number of processes carried by each core is averaged to reduce the communication overhead in the task, improve the running speed of the task, and improve the running efficiency of the high-performance task. It can be understood that in the process mapping optimization method of the embodiments of the present application, the influence of uneven distribution of sub-processes on the task running efficiency and the difference in inter-core communication cost caused by the allocation of sub-processes to different process cores are comprehensively considered in the fitness, which can guide the genetic algorithm to find a process mapping scheme that better meets the needs of the real scene. At the same time, the redundancy scheduling strategy proposed in the embodiments of the present application can perform local search with the individual code solution as the center, and schedule the sub-processes on each process core until the number of processes carried by each process core is averaged, under the premise of as little as possible affecting the individual fitness.

[0048] In some embodiments, the first code includes a sub-process code and a process core code, the redundancy scheduling processing of the first code subjected to the genetic evolution operation generates a second process mapping coding array having a plurality of second codes, and the redundancy scheduling processing is implemented by the following steps:

[0049] Step 21, according to the first encoding, determining all process cores corresponding to the first process core encoding carried by the sub-process to obtain the first sub-process set.

[0050] In this embodiment, the first sub-process set includes a plurality of first process cores and a sub-process set initially configured for each first process core, and all sub-process sets form the first sub-process set.

[0051] Step 22, based on the process communication amount corresponding to each sub-process in the first sub-process set, a process communication amount matrix is constructed, and in the process communication amount matrix, the row number value is the process communication amount of the target sub-process corresponding to the sub-process code, to obtain the first process communication set.

[0052] In this embodiment, the first sub-process set includes all sub-processes of the target computing program, and therefore, based on the process communication amount corresponding to each sub-process in the first sub-process set, the process communication amount matrix is constructed, which is a matrix of the communication amount between the two sub-processes in the plurality of sub-processes of the target computing program. One element in the process communication amount matrix represents the communication amount between one sub-process and another sub-process that forms a process communication pair, that is, through one element in the process communication amount matrix, a process communication pair composed of two sub-processes and the communication amount between the corresponding two sub-processes can be determined.

[0053] Step 23, from the first process communication set, a preset number of process communication amounts are obtained in the order of communication amount value from large to small, and the sub-process corresponding to each process communication amount obtained is determined to obtain a candidate communication process set corresponding to the target sub-process, wherein the target sub-process and all sub-processes of the candidate communication process set form a plurality of first process communication pairs.

[0054] In this embodiment, after determining all sub-processes that form a communication process pair with the target sub-process according to the sub-process code of the target sub-process, the communication process pairs with process communication amount within a certain value range are selected according to the process communication amount of the corresponding communication process pair (for example: T process communication amounts with the largest communication amount value are obtained), that is, a preset number of process communication amounts are obtained from the first process communication set in the order of communication amount value from large to small. After a preset number of process communication amounts are obtained, the sub-processes that form process communication pairs with the target sub-process are also determined, that is, the candidate communication process set is obtained, for example: it is assumed that the target sub-process i communicates with processes 1 to n, and the first T sub-processes with the largest communication amount are: process t, process t+1, …, process t+T-1, then the candidate communication process set is process t, process t+1, …, process t+T-1.

[0055] Step 24, detecting a second process communication pair carried by the first process core carrying the target sub-process from the plurality of first process communication pairs, and updating the first sub-process set according to the number of the second process communication pair and a preset process scheduling rule, to generate a second sub-process set, wherein the second sub-process set comprises corresponding relationship information between the second process core and the sub-process carried by the second process core, and the second process core is used to execute the target sub-process.

[0056] In the embodiment, after obtaining the candidate communication process set, a communication process pair bound to the same first process core is detected, that is, the process communication amount between the sub-processes in communication with the target sub-process belongs to the top T sub-processes, and the sub-process carried by the first process core carrying the target sub-process is detected. When the sub-process bound to the same first process core is detected, a corresponding second process communication pair is obtained, and when the number of the second process communication pair associated with the target sub-process is detected, the more the target sub-process contributes to the intra-core process communication amount of the corresponding first process core; thereafter, the size of the communication amount contributed by the corresponding target sub-process to the corresponding first process core is determined to determine the scheduling level of the target sub-process, that is, the priority of the corresponding scheduling. Specifically, the larger the communication amount contributed by the corresponding target sub-process to the corresponding first process core, the higher the scheduling level of the target sub-process, that is, the target sub-process is most likely to be scheduled to the outside of the core. Then, based on the determined scheduling priority and the preset process scheduling rule, the scheduling of the sub-process is implemented, that is, the first sub-process set is updated. It should be understood that the candidate communication process set is a set of sub-processes that form a communication process pair with a selected target sub-process, and the sub-process that forms a communication process pair with the target sub-process is allocated to different first process cores. When the sub-process that forms a communication process pair with the target sub-process is carried by the same first process core as the target sub-process, the priority of the sub-process being scheduled to other first process cores is lower.

[0057] In some embodiments, the updating of the first sub-process set according to the number of the second process communication pair and the preset process scheduling rule in step 24 to generate the second sub-process set comprises the following steps:

[0058] Step 241, determining the scheduling level corresponding to each target sub-process according to the number of the second process communication pair corresponding to the target sub-process, wherein the scheduling level is used to represent the contribution degree of the intra-core process communication amount of the target sub-process.

[0059] In the embodiment, when detecting a communication process pair bound to the same first process core, that is, when the process communication between the target sub-process and the sub-processes belongs to the top T sub-processes, the sub-process carried by the first process core carrying the target sub-process is detected, when detecting the sub-process bound to the same first process core, a corresponding second process communication pair is obtained, and when detecting the number of the target sub-process associated second process communication pairs, the more the target sub-process contributes to the intra-core process communication of the corresponding first process core; the size of the communication amount contributed by the target sub-process to the corresponding first process core determines the scheduling level of the target sub-process, that is, the priority of the corresponding scheduling, specifically, the larger the communication amount contributed by the corresponding target sub-process to the corresponding first process core, the higher the scheduling level of the target sub-process, that is, the minimum possible scheduling to the out-core; in some optional ways, assuming that there are sub-processes 1, 3, 4, 6, 10, 15 and 17 on the first process core 1, and the sub-processes communicating with the target sub-process 1 and the communication amount are in the top 5, the determined scheduling level value of the target sub-process 1 is 3.

[0060] In step 242, the number of processes in the sub-process subset carried by each first process core is determined, and at least one process core group including the first target core and the second target core is determined in all first process cores according to the number of processes, wherein the first target core includes the first process core corresponding to the number of processes greater than the preset average number of processes, and the second target core includes the first process core corresponding to the number of processes less than the average number of processes.

[0061] In the embodiment, before implementing the scheduling of the sub-process, it is necessary to determine which corresponding first process core carries the sub-process to be scheduled according to the number of processes of all sub-processes carried by the corresponding first process core; in the embodiment, the sub-process carried by the first process core with more number of processes than the average number of processes is selected as the sub-process to be scheduled, and is scheduled to the first process core with less number of processes than the average number of processes, that is, the sub-process selected from the sub-process carried by the first target core according to the corresponding scheduling rule is scheduled to the second target core.

[0062] In step 243, the target sub-process of the preset process scheduling number is selected in the sub-process subset corresponding to the first target core in the order from low to high according to the scheduling level, the scheduling sub-process is obtained, and the scheduling sub-process is scheduled to the second target core to generate the second sub-process set, wherein the process scheduling number includes the minimum deviation in the deviation of the number of processes and the average number of processes, the second process core includes the first target core and the second target core, and the sub-process carried by the second process core includes the sub-process carried by the first target core after the scheduling and the sub-process carried by the second target core.

[0063] In the embodiment, when scheduling the sub-processes, the number of scheduled processes is to make the sub-processes carried by the first target core and the second target core tend to the process core average value, and the process scheduling number is the minimum deviation of the process number of the sub-processes carried by the first target core and the second target core respectively from the process number average value, for example, the process number of the sub-processes carried by the first process core 1 is 56, the process number of the sub-processes carried by the first process core 2 is 45, and the process number average value is 50, then the scheduled process scheduling number is 5, and the process number of the sub-processes carried by the first process core 1 is 53, the process number of the sub-processes carried by the first process core 2 is 45, and the process number average value is 50, then the scheduled process scheduling number is 3; in the embodiment, after determining the process scheduling number, the target sub-processes of the process scheduling number are selected from the first target core in the order of low to high scheduling level, and the selected target sub-processes are scheduled to be carried by the second target core, for example, assuming that the first process core 1 carries 56 sub-processes, the first process core 2 carries 45 processes, and the process number average value is set to 50, among the sub-processes carried by the first process core 1, the scheduling level values of target sub-process 1, target sub-process 2, target sub-process 3, target sub-process 4, target sub-process 5, target sub-process 6 and target sub-process 7 are 3, 3, 3, 2, 5, 8 and 1 respectively, and the sub-processes scheduled to the first process core 2 are target sub-process 6, target sub-process 4, target sub-process 1, target sub-process 2 and target sub-process 3 in turn.

[0064] Step 25, generating a second process mapping code group according to the second sub-process set.

[0065] In some embodiments, the step 25 of generating a second process mapping code group according to the second sub-process set includes the following steps:

[0066] Step 251, respectively determining the sub-processes carried by the first target core and the second target core after completing the scheduling, to obtain a first sub-process group and a second sub-process group.

[0067] Step 252, performing process mapping on the first target core and the first sub-process group, the second target core and the second sub-process group to generate a second process mapping code group.

[0068] In the embodiment, the first target core and the first sub-process group, the second target core and the second sub-process group are mapped to generate the second process mapping code group, which realizes the generation of the process mapping scheme after the genetic evolution operation and the redundant scheduling, so as to approach the optimal process mapping scheme, and each sub-process is bound to the corresponding process core before the task execution, and the task is completed with the minimum communication cost before the task execution, thereby further reducing the communication cost in the task, improving the running speed of the task, and improving the running efficiency of the high-performance task.

[0069] Through the steps 21 to 25, the predetermined redundant scheduling strategy is used to balance the scheduling of the sub-processes carried by all the first process cores corresponding to the first process mapping code array, so that the number of processes carried by all the process cores tends to be the average value, thereby further reducing the communication cost in the task, improving the running speed of the task, and greatly improving the running efficiency of the high-performance task.

[0070] It should be noted that in the embodiment, when the number of sub-processes carried by the two process cores (the first target core and the second target core) of each found process core group deviates from the average value of the number of processes, the part of the target sub-process with the minimum level value of the scheduling level on the process core with more processes (i.e., the first target core) is scheduled to the process core with less processes (i.e., the second target core), until the number of processes carried by all the process cores reaches the average value of the number of processes, so as to modify the chromosome (i.e., the corresponding process mapping code array) of the next generation population individual to a better state through the redundant scheduling method. In the embodiment, the configuration relationship between the sub-processes and the process cores at the initialization is determined according to the first code, that is, the sub-processes carried by the corresponding process cores and the process cores executing the corresponding sub-processes are determined. In the embodiment, each process core carries at least one sub-process, and therefore each first process core corresponds to a first sub-process set.

[0071] It should be further noted that in the embodiment, the value of each dimension element x i of the chromosome x corresponding to the first process mapping code array is traversed, and it is assumed that the value of x i is 1, which indicates that the sub-process i is bound to the process core 1. Each process core carrying a process is recorded according to this rule, and is represented in the form of a set (corresponding to the first sub-process set). It is assumed that the first sub-process set H i ={h0,h1,...,h k-1} is set. idenotes the number of all sub-processes carried by the process core i, k is the number of processes carried by the process core, a parameter T = TR*n is introduced, and a count variable V (corresponding to the scheduling level) with an initial value of 0 is created for each sub-process, and each sub-process h carried by each process core is checked i , i∈[0,k-1], the h i th row of the process communication matrix M is queried, the column values of the T (preset number) elements with the largest values in the row are found and recorded as Y = (y1, y2,..., y t ), Y denotes a corresponding candidate process communication set, each element y of Y (corresponding to a process communication) is traversed, and each time x hi = x y = i, that is, it is determined that the sub-process h i and the sub-process y in the second process communication pair corresponding to the target sub-process are bound to the same process core i, the V count of the sending sub-process h i is increased by one, and the larger the V count is, the more the sending sub-process can contribute to the intranode process communication under the current mapping scheme, and the sub-process should not be scheduled to other process cores.

[0072] In the redundant scheduling of the embodiment, the redundant scheduling observation ratio TR is set, and it can be understood that TR is used to specify the number of observed process communication pairs when performing redundant scheduling. If the number of observations is large, the calculation amount will be increased, and if the number of observations is small, the determination accuracy when selecting the sub-process to be transferred will be reduced, thereby reducing the effect of the redundant scheduling strategy.

[0073] In some embodiments, in the process of updating the second encoding in the second process mapping encoding array according to the fitness, the following steps are further implemented:

[0074] Step 31, according to the fitness, selecting a plurality of target encodings with the minimum fitness from the second encoding after each iteration update, wherein the number of target encodings is equal to the number of iteration updates.

[0075] Step 32, judging whether the fitness corresponding to the plurality of target encodings changes in sequence, and adjusting the mutation rate corresponding to the mutation operation according to the judgment result according to a preset mutation explosion rule.

[0076] In some embodiments, the adjusting the mutation rate corresponding to the mutation operation according to the judgment result according to a preset mutation explosion rule in step S32 comprises:

[0077] Step 41, in the case where the judgment result is that the fitness does not change, increasing the mutation rate by a preset step size;

[0078] Step 42, in the case of the judgment result is the existence of fitness changes, the current mutation rate back to the initial value of the mutation rate.

[0079] In the present embodiment, the genetic evolution operation update based on the mutation surge strategy is implemented, so as to temporarily expand the search range when the genetic algorithm falls into local optimum, so that the genetic algorithm has better global search ability, and the search range is called back after a new solution is found, which will not affect the efficiency in the normal search process; it can be understood that the mutation surge is achieved by temporarily increasing the mutation rate of the algorithm to improve the global search ability of the genetic algorithm and avoid the dilemma of falling into local optimum but unable to jump out.

[0080] In some optional embodiments, during the genetic evolution operation update process based on the second process mapping coding array according to the fitness, the last generation best coding individual (the coding corresponding to the minimum value of the fitness) and its fitness in each genetic evolution operation iteration and redundant scheduling iteration are recorded, a local optimal observation window W is set, and when the fitness of the best coding individual of the last W generations does not change, it is determined that the local optimal state is entered, and the mutation surge is triggered. The corresponding mutation surge strategy is: temporarily increasing the mutation rate by a certain step S, and the mutation rate cannot exceed the upper limit MRT. Once the fitness of the best coding individual of a generation changes from the last generation, the mutation rate is immediately called back to the initial value MR, and the local optimum is re-counted. In the present embodiment, the mutation surge step ratio SR is preferably 1.0. At the same time, it should be understood that in the mutation surge, the parameter W is used to specify the number of generations for observing the local optimum when the mutation surge is performed. If the number of observation generations is large, the sensitivity of the mutation surge will be reduced. If the number of observation generations is small, the mutation rate will be at a high level, which will make the solution volatility too large. Furthermore, the parameter S is used to specify the step of the mutation surge. If the step is large, the controllability of the solution search range will be reduced. If the step is small, the speed of introducing randomness to the solution will be too slow, which will further reduce the search efficiency.

[0081] In some embodiments, the fitness F is calculated according to the following formula:

[0082]

[0083]

[0084] Wherein, X represents the corresponding process mapping coding group, X i The number of process cores carrying the sub-process i; M represents the process communication matrix constructed based on the process communication of each sub-process, M ∈ R n×n , n represents the number of sub-processes, M ij The communication amount of process i to process j; P represents the communication cost matrix, P ∈ Rq×q q represents the number of process cores, P ij represents the process communication latency between process core i and process core j; γ(i,j,X) represents the process communication latency of the process pair composed of process i and process j, and r(i) represents the number of processes carried by the process i on the corresponding process core; represents the minimum process communication latency in the process communication matrix; exp is the exponential function with the natural constant e as the base; r(X i ) represents the number of processes carried by the process core carrying process i; n / q represents the average number of processes.

[0085] Figure 3 is a flowchart of a process optimization method according to a preferred embodiment of the present application, referring to Figure 3 The genetic algorithm-based process mapping optimization method of the embodiments of the present application is further described as follows, which includes the following steps:

[0086] Step S301, input algorithm hyperparameters, and then perform step S302.

[0087] Step S302, initialize the population, and then perform step S303.

[0088] Step S303, calculate the fitness, and then perform step S304.

[0089] Step S304, tournament selection, uniform crossover, and gene mutation, and then perform step S305.

[0090] Step S305, redundant scheduling, and then perform step S306.

[0091] Step S306, determine whether the mutation explosion strategy is triggered, if yes, perform step S307, otherwise perform step S308.

[0092] Step S307, mutation explosion, and then perform step S308.

[0093] Step S308, determine whether the maximum number of iterations is reached, if yes, perform step S309, otherwise perform step S303.

[0094] Step S309, output the target process mapping encoding array.

[0095] In some optional embodiments, the following steps are also implemented to realize the process mapping optimization of the embodiments of the present application:

[0096] Step 1, collect the process communication traffic of each sub-process of the target computing program transmitted through the MPI interface of the cross-language communication protocol, normalize the collected process communication traffic, and establish a process communication traffic matrix M based on the data of the normalized process communication traffic, M ∈ R n×n , n represents the number of sub-processes, and M ij in the process communication traffic matrix M represents the communication traffic of the sub-process i to the sub-process j.

[0097] Step 2, test the communication latency between each process core allocated for use by the target computing program through a Pingpong test, collect the communication latency under the same communication traffic, and normalize the collected data, thereby establishing a process communication cost matrix P, P ∈ R q×q , q represents the number of process cores, and P ij in the process communication cost matrix P represents the communication cost of the process core i to the process core j.

[0098] Step 3, algorithm hyperparameter setting and initialization, set the initial values of the population size PS, the mutation rate MR of genetic evolution, the crossover rate CR of genetic evolution, the number of participating individuals TS of each round of tournament selection of genetic evolution, the mutation surge step size ratio SR of genetic evolution, and the initial value of the redundancy scheduling observation ratio TR, encode the process mapping scheme using real number coding, that is, the chromosome of an individual x = (x0, x1, …, x n-1 ), and each element x i satisfies x i ∈ [0, q-1], and x i ∈ N + , wherein the value of x i represents which execution core executes the sub-process, for example: when x i takes the value 1, it means that the sub-process i is executed by the process core 1, and the input communication traffic matrix M and the communication cost matrix P are used as the data for genetic algorithm optimization.

[0099] In this embodiment, the population size PS is set to 50, the mutation rate MR of genetic evolution is 0.005, the crossover rate CR of genetic evolution is 0.6, the mutation surge step size ratio SR of genetic evolution is 1.0, and the redundancy scheduling observation ratio TR is 0.1.

[0100] Step 4, initialize the population of individuals with a number of PS, and each individual is randomly generated under the individual limitation condition mentioned in step S1.

[0101] Step 5, calculate the fitness of each individual in the population, and then perform tournament selection, uniform crossover, and mutation operation on the individuals in the population through the genetic algorithm to obtain the chromosome of the next generation of population individuals.

[0102] In this embodiment, the calculation formula of the fitness of the individual is the calculation formula of the fitness F in the above embodiment, which will not be described here.

[0103] Step 6, redundancy scheduling.

[0104] Step 7, mutation surge.

[0105] In this embodiment, during the genetic evolution operation updating process of the second encoding in the second process mapping encoding array based on the fitness, the last generation best encoding individual (the encoding corresponding to the minimum value of the fitness) and the fitness thereof in each genetic evolution operation iteration and redundancy scheduling iteration are recorded, a local optimal observation window W is set, and when the fitness of the best encoding individual of the last W generations does not change, it is determined that the local optimal state is entered, and the mutation surge is triggered. Specifically, the mutation rate is temporarily increased by a certain step S, and the mutation rate cannot exceed the upper limit MRT. Once the fitness of the best encoding individual of a certain generation changes compared with that of the last generation, the mutation rate is immediately adjusted back to the initial value MR, and the local optimum is re-counted.

[0106] Step 8, if the maximum iteration number is not reached, steps 3-5 are continued to be iteratively executed, otherwise the algorithm exits the optimization, and the chromosome x of the best individual of the population is output as the process mapping scheme finally obtained by the algorithm.

[0107] After steps 1-8 are executed, the target process mapping encoding array is output, that is, the corresponding process mapping scheme is output. It should be noted that in the case that the task has a large number of sub-processes, a better process mapping scheme can be found through steps 1-8. For example, for an initial random population (corresponding to the first process mapping encoding array), after the first population iteration, the solution carried by the best individual of the population (corresponding to the encoding) is close to the optimal solution, and the individuals in the population all introduce a certain randomness through the operations of crossover, mutation and redundancy scheduling. Therefore, each individual retains part of the randomness in the optimization process, thereby giving the algorithm stronger global search capability. Similarly, in the subsequent population iterations, the population continues to be optimized until the fitness converges. After a better process mapping scheme is obtained, the sub-processes of the task can be scheduled according to the corresponding scheme, and each sub-process is bound to the corresponding process core before the task is executed, so that the task can be completed in the way with the minimum communication overhead.

[0108] The embodiments further provide a process mapping optimization apparatus based on a genetic algorithm, which is used to implement the above-described embodiments and preferred embodiments, and will not be described again. As used below, the terms "module", "unit", "sub-unit", and the like can be a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.

[0109] Figure 4 is a structural block diagram of process mapping optimization based on a genetic algorithm according to the embodiments of the present application, as Figure 4 shown, the apparatus includes an acquisition module 41, an encoding module 42, a processing module 43, and a generation module 44, wherein,

[0110] The acquisition module 41 is configured to acquire communication data corresponding to a plurality of sub-processes of a target computing program, wherein the communication data includes process communication volume and process communication latency.

[0111] The encoding module 42 is coupled to the acquisition module 41 and is configured to perform real number encoding on the process communication volume and the process communication latency to generate a first process mapping encoding array, wherein the first process mapping encoding array includes a plurality of first encodings corresponding to the sub-processes, and each first encoding is used to represent a corresponding relationship between a first process core executing a sub-process and the sub-process.

[0112] The processing module 43 is coupled to the encoding module 42 and is configured to perform genetic evolution operations on the first encodings based on a preset genetic algorithm, and perform redundancy scheduling processing on the first encodings after the genetic evolution operations to generate a second process mapping encoding array including a plurality of second encodings, wherein the genetic evolution operations include at least one of tournament selection, uniform crossover, and mutation, and each second encoding is used to represent a corresponding relationship between a second process core executing a sub-process and the sub-process.

[0113] The generation module 44 is coupled to the processing module 43 and is configured to determine a fitness of each second encoding, and perform genetic evolution operation updating on the second encodings in the second process mapping encoding array according to the fitness to generate a target process mapping encoding array, thereby obtaining a process mapping optimization result, wherein the fitness is determined according to the process communication volume and the process communication latency corresponding to each second encoding, and the fitness is used to represent a communication overhead of a corresponding process core executing a sub-process.

[0114] The process mapping optimization device based on the genetic algorithm comprises the following steps: obtaining communication data corresponding to a plurality of sub-processes of a target computing program, wherein the communication data comprises process communication volume and process communication time delay; performing real number coding on the process communication volume and the process communication time delay to generate a first process mapping coding array, the first process mapping coding array comprising a plurality of first codes corresponding to the sub-processes, the first code being used to represent the correspondence between a first process core executing the sub-process and the sub-process; performing genetic evolution operation on the first code based on a preset genetic algorithm, and performing redundancy scheduling processing on the first code after the genetic evolution operation to generate a second process mapping coding array with a plurality of second codes, the genetic evolution operation comprising at least one of tournament selection, uniform crossover and mutation, the second code being used to represent the correspondence between a second process core executing the sub-process and the sub-process; determining the fitness of the second code, and updating the second code in the second process mapping coding array based on the fitness to generate a target process mapping coding array, thereby obtaining a process mapping optimization result, the fitness being determined according to the process communication volume and the process communication time delay corresponding to the second code, and the fitness being used to represent the communication overhead of the corresponding process core executing the sub-process; and mapping the process group with larger communication volume to the core group with smaller communication cost, and averaging the number of processes borne by each core, thereby reducing the communication overhead in the task, improving the running speed of the task, improving the running efficiency of the high-performance task, and solving the problem of low running efficiency and large communication overhead of the process mapping scheme of the HPC cluster in the related art.

[0115] In some embodiments, the first code comprises a sub-process code and a process core code, and the processing module 43 further comprises:

[0116] A first determining unit is configured to determine the sub-process borne by the first process core corresponding to all the process core codes according to the first code, and obtain a first sub-process set.

[0117] A first selecting unit is coupled to the first determining unit and configured to construct a process communication volume matrix based on the process communication volume corresponding to each sub-process in the first sub-process set, select the process communication volume with a row number value being the sub-process code corresponding to the target sub-process in the process communication volume matrix, and obtain a first process communication volume set.

[0118] A first obtaining unit is coupled to the first selecting unit and configured to obtain a preset number of process communication volumes in the first process communication volume set in descending order of the communication volume value, determine the sub-process corresponding to each obtained process communication volume, and obtain a candidate communication process set corresponding to the target sub-process, wherein the target sub-process and all the sub-processes in the candidate communication process set form a plurality of first process communication pairs.

[0119] The first scheduling unit is coupled to the first obtaining unit and is configured to detect, in the plurality of first process communication pairs, a second process communication pair carried by a first process core carrying a target sub-process, and update the first sub-process set according to a pair number of the second process communication pair and a preset process scheduling rule, to generate a second sub-process set, wherein the second sub-process set includes correspondence information between a second process core and a sub-process carried by the second process core, and the second process core is configured to execute the target sub-process.

[0120] The first generating unit is coupled to the first scheduling unit and is configured to generate a second process mapping code group according to the second sub-process set.

[0121] In some embodiments, the first scheduling unit is further configured to determine a scheduling level corresponding to each target sub-process according to a pair number of a second process communication pair corresponding to the target sub-process, wherein the scheduling level is used to represent a contribution degree of an intra-core process communication amount of the target sub-process; determine a process number of a sub-process subset composed of sub-processes carried by each first process core, and in all first process cores, determine at least a process core group including a first target core and a second target core according to the process number, wherein the first target core includes a first process core corresponding to a process number greater than a preset process number average, and the second target core includes a first process core corresponding to a process number less than the process number average; in the sub-process subset corresponding to the first target core, select target sub-processes of a preset process scheduling number in order from low to high according to the scheduling level, to obtain scheduling sub-processes, and schedule the scheduling sub-processes to the second target core, to generate the second sub-process set, wherein the process scheduling number includes a minimum deviation in a deviation of the process number from the process number average, the second process core includes the first target core and the second target core, and the sub-processes carried by the second process core include sub-processes carried by the first target core after scheduling and sub-processes carried by the second target core.

[0122] In some embodiments, the first generating unit is further configured to respectively determine the sub-processes carried by the first target core after scheduling and the sub-processes carried by the second target core, to obtain a first sub-process group and a second sub-process group; perform process mapping between the first target core and the first sub-process group, the second target core and the second sub-process group, to generate the second process mapping code group.

[0123] In some embodiments, the apparatus is further configured to, in the process of performing the genetic evolution operation on the second codes in the second process mapping code group according to the fitness, select a plurality of target codes with minimum fitness from the second codes after each iteration update, wherein the number of the target codes is equal to the number of iteration updates; in turn, judge whether the fitness corresponding to the plurality of target codes changes, and according to the judgment result, adjust a mutation rate corresponding to a mutation operation according to a preset mutation explosion rule.

[0124] In some embodiments, the device is further configured to increase the mutation rate by a preset step size if the result of the judgment is that the fitness has not changed; and to decrease the current mutation rate to the initial value of the mutation rate if the result of the judgment is that the fitness has changed.

[0125] In some embodiments, the device is further configured to calculate the fitness F according to the following formula:

[0126]

[0127]

[0128] wherein X represents a corresponding process mapping code group, X i represents the number of process cores carrying the sub-process i; M represents a process communication matrix constructed based on the process communication of each sub-process, M n×n , n represents the number of sub-processes, M ij represents the process communication of process i to process j; P represents a communication cost matrix, P q×q , q represents the number of process cores, P ij represents the process communication delay of process core i to process core j; γ(i, j, X) represents the process communication delay of the process pair composed of sub-process i and sub-process j, r(i) represents the number of processes carried by the sub-process on the corresponding process core i; represents the minimum process communication delay in the process communication matrix; exp is the exponential function with the natural constant e as the base; r(X i ) represents the number of sub-processes carried by the process core carrying sub-process i; n / q represents the average number of processes.

[0129] The embodiment also provides an electronic device including a memory and a processor, the memory storing a computer program, and the processor being configured to execute the computer program to perform the steps in any of the method embodiments.

[0130] Optionally, the electronic device can further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0131] Optionally, in the embodiment, the processor can be configured to execute the following steps through the computer program:

[0132] S1, obtaining communication data corresponding to a plurality of sub-processes of a target computing program, wherein the communication data includes process communication and process communication delay.

[0133] S2, real number coding is performed on the process communication traffic and the process communication delay to generate a first process mapping coding array, wherein the first process mapping coding array comprises a plurality of first codes corresponding to the sub-processes, and the first code is used to represent the corresponding relationship between the first process core executing the sub-process and the sub-process.

[0134] S3, based on the preset genetic algorithm, the first code is subjected to genetic evolution operation, and the first code after the genetic evolution operation is subjected to redundancy scheduling processing to generate a second process mapping coding array with a plurality of second codes, wherein the genetic evolution operation at least includes one of the following: tournament selection, uniform crossover, mutation, and the second code is used to represent the corresponding relationship between the second process core executing the sub-process and the sub-process.

[0135] S4, determining the fitness of the second code, and updating the second code in the second process mapping coding array according to the fitness to generate a target process mapping coding array, and obtaining a process mapping optimization result, wherein the fitness is determined according to the process communication traffic and the process communication delay corresponding to the second code, and the fitness is used to represent the communication overhead of the process core executing the sub-process.

[0136] It should be noted that the specific examples in the embodiments can refer to the examples described in the above embodiments and optional implementation manners, and the embodiments will not be described here.

[0137] In addition, in combination with the process mapping optimization method based on the genetic algorithm in the above embodiments, the embodiments of the application can provide a storage medium for implementation. The storage medium has a computer program stored thereon; the computer program is executed by a processor to implement any one of the process mapping optimization methods based on the genetic algorithm in the above embodiments.

[0138] Those skilled in the art should understand that the technical features of the above embodiments can be combined in any way, and in order to make the description concise, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combinations of the technical features do not exist contradictory, it should be considered that it is within the scope of the present application.

[0139] The above embodiments only express several implementation manners of the application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the application, a number of modifications and improvements can be made, which are all within the scope of the application. Therefore, the protection scope of the patent of the application should be subject to the appended claims.

Claims

1. A process mapping optimization method based on genetic algorithm, characterized in that, The method comprises the following steps: acquiring communication data corresponding to a plurality of sub-processes of a target computing program, wherein the communication data comprises process communication volume and process communication delay; real number coding is performed on the process communication volume and process communication delay to generate a first process mapping coding array, wherein the first process mapping coding array comprises a plurality of first codes corresponding to the sub-processes, and the first codes are used to represent the correspondence between a first process core executing the sub-processes and the sub-processes; a genetic evolution operation is performed on the first codes based on a preset genetic algorithm, and a redundancy scheduling process is performed on the first codes after the genetic evolution operation to generate a second process mapping coding array with a plurality of second codes, wherein the genetic evolution operation at least comprises one of tournament selection, uniform crossover and mutation, and the second codes are used to represent the correspondence between a second process core executing the sub-processes and the sub-processes; the fitness of the second codes is determined, and the second codes in the second process mapping coding array are updated through the genetic evolution operation according to the fitness to generate a target process mapping coding array, thereby obtaining a process mapping optimization result, wherein the fitness is determined according to the process communication volume and the process communication delay corresponding to the second codes, and the fitness is used to represent the communication overhead of the corresponding process core executing the sub-processes; wherein the first codes comprise sub-process codes and process core codes, and the redundancy scheduling process performed on the first codes after the genetic evolution operation generates a second process mapping coding array with a plurality of second codes, which comprises: determining the sub-processes carried by the first process core according to all the process core codes corresponding to the first process core, thereby obtaining a first sub-process set; constructing a process communication volume matrix based on the process communication volume corresponding to each sub-process in the first sub-process set, and selecting the process communication volume with a row number value corresponding to the sub-process code of a target sub-process in the process communication volume matrix, thereby obtaining a first process communication volume set; acquiring a preset number of process communication volumes in descending order of communication volume value from the first process communication volume set, and determining the sub-process corresponding to each acquired process communication volume, thereby obtaining a candidate communication process set corresponding to the target sub-process, wherein the target sub-process and all the sub-processes in the candidate communication process set form a plurality of first process communication pairs; detecting a second process communication pair carried by the first process core carrying the target sub-process from a plurality of first process communication pairs, and updating the first sub-process set according to the number of the second process communication pair and a preset process scheduling rule to generate a second sub-process set, wherein the second sub-process set comprises correspondence information between the second process core and the sub-process carried by the second process core, and the second process core is used to execute the target sub-process; generating the second process mapping coding array according to the second sub-process set; The first sub-process set is updated according to the pair number of the second process communication pair and a preset process scheduling rule, to generate a second sub-process set, including: According to the pair number of the second process communication pair corresponding to the target sub-process, a scheduling level corresponding to each target sub-process is determined, wherein the scheduling level is used to represent the contribution degree of the in-core process communication amount of the target sub-process; The number of processes in a sub-process subset composed of the sub-processes carried by each first process core is determined, and in all the first process cores, at least one process core group including a first target core and a second target core is determined according to the number of processes, wherein the first target core includes the first process core corresponding to the number of processes greater than a preset average number of processes, and the second target core includes the first process core corresponding to the number of processes less than the average number of processes; In the first target core corresponding to the sub-process subset, the target sub-processes of a preset process scheduling number are selected in order from low to high according to the scheduling level, to obtain scheduling sub-processes, and the scheduling sub-processes are scheduled to the second target core, to generate the second sub-process set, wherein the process scheduling number includes the minimum deviation in the deviation of the number of processes from the average number of processes, the second process core includes the first target core and the second target core, and the sub-processes carried by the second process core include the sub-processes carried by the first target core after scheduling and the sub-processes carried by the second target core.

2. The method of claim 1, wherein, According to the second sub-process set, the second process mapping encoding array is generated, including: The first sub-process group and the second sub-process group are obtained by respectively determining the sub-processes carried by the first target core after scheduling and the sub-processes carried by the second target core; The first target core is process-mapped with the first sub-process group, the second target core and the second sub-process group, to generate the second process mapping encoding array.

3. The method of claim 1, wherein, In the process of updating the second encoding in the second process mapping encoding array by genetic evolution operation according to the fitness, the method further includes: According to the fitness, a plurality of target encodings with the minimum fitness are selected from the second encoding after each iteration update, wherein the number of target encodings is equal to the number of iteration updates; It is judged in turn whether the fitness corresponding to a plurality of target encodings changes, and according to the judgment result, the mutation rate corresponding to the mutation operation is adjusted according to a preset mutation surge rule.

4. The method according to claim 1, wherein According to the judgment result, the mutation rate corresponding to the mutation operation is adjusted according to a preset mutation surge rule, including: In the case where the judgment result is that the fitness does not change, the mutation rate is increased by a preset step; In the case where the judgment result is that the fitness changes, the current mutation rate is adjusted to a mutation rate initial value.

5. The method of claim 1, wherein, The fitness F is calculated according to the following formula: Among them, X represents the corresponding process mapping code group, and the element X of X i Represents the number of the process core that carries the child process i; M represents the process communication volume matrix constructed based on the process communication volume corresponding to each child process, M∈R n×n , n represents the number of subprocesses, the element M of M ij represents the communication volume from process i to process j; P represents the communication cost matrix, P∈R q×q , q represents the number of process cores, and the element P of P ij represents the process communication delay between process core i and process core j; represents the process communication delay of the process pair consisting of sub-process i and sub-process j, r(i) represents the number of sub-processes carried by the corresponding process core i; represents the minimum process communication delay in the process communication matrix; exp is an exponential function with the natural constant e as the base; r(X i ) represents the number of sub-processes carried by the process core that carries sub-process i; n / q represents the mean number of processes.

6. A process mapping optimization apparatus based on a genetic algorithm, characterized by, including: An obtaining module is configured to obtain communication data corresponding to a plurality of sub-processes of a target computing program, wherein the communication data comprises process communication volume and process communication latency; An encoding module is configured to perform real number encoding on the process communication volume and the process communication latency to generate a first process mapping encoding array, wherein the first process mapping encoding array comprises a plurality of first encodings corresponding to the sub-processes, and the first encodings are used to represent the corresponding relationship between a first process core executing the sub-processes and the sub-processes; The processing module is configured to perform genetic evolution operation on the first encoding based on a preset genetic algorithm, and perform redundancy scheduling processing on the first encoding after the genetic evolution operation to generate a second process mapping encoding array with multiple second encodings, wherein the genetic evolution operation at least includes one of tournament selection, uniform crossover and mutation, the second encoding is used to represent the corresponding relationship between a second process core for executing a sub-process and the sub-process, the first encoding includes sub-process encoding and process core encoding, the processing module is further configured to determine the sub-process carried by the first process core corresponding to all the process core encodings according to the first encoding to obtain a first sub-process set; construct a process communication matrix based on the process communication of each sub-process in the first sub-process set, and obtain a first process communication set by selecting the process communication with the row number value of the sub-process encoding corresponding to the target sub-process in the process communication matrix; obtain a preset number of process communications in descending order of communication value from the first process communication set, and determine the sub-process corresponding to each obtained process communication to obtain a candidate communication process set corresponding to the target sub-process, wherein the target sub-process and all the sub-processes of the candidate communication process set form multiple first process communication pairs; detect a second process communication pair carried by the first process core carrying the target sub-process in the multiple first process communication pairs, and update the first sub-process set according to the number of the second process communication pair and a preset process scheduling rule to generate a second sub-process set, wherein the second sub-process set includes the corresponding relationship information between the second process core and the sub-process carried by the second process core, and the second process core is used to execute the target sub-process; generate the second process mapping encoding array according to the second sub-process set; the processing module is further configured to determine a scheduling level corresponding to each target sub-process according to the number of the second process communication pair corresponding to the target sub-process, wherein the scheduling level is used to represent the contribution degree of the intra-core process communication of the target sub-process; determine the number of processes in a sub-process subset composed of the sub-process carried by each first process core, and determine at least one process core group including a first target core and a second target core in all the first process cores according to the number of processes, wherein the first target core includes the first process core with a corresponding number of processes greater than a preset average number of processes, and the second target core includes the first process core with a corresponding number of processes less than the average number of processes.In the sub-process subset corresponding to the first target core, the target sub-processes are selected in the order from low to high of the scheduling level according to the preset number of process scheduling, to obtain scheduling sub-processes, and the scheduling sub-processes are scheduled to the second target core to generate the second sub-process set, wherein the number of process scheduling includes the minimum deviation in the deviation of the process number and the process number average, the second process core includes the first target core and the second target core, and the sub-processes carried by the second process core include the sub-processes carried by the first target core after completion of scheduling and the sub-processes carried by the second target core. A generating module is configured to determine an adaptability corresponding to the second encodings, and perform a genetic evolution operation on the second encodings in the second process mapping encoding array according to the adaptability to update the second encodings, to generate a target process mapping encoding array, and obtain a process mapping optimization result, wherein the adaptability is determined according to the process communication volume and the process communication latency corresponding to the second encodings, and the adaptability is used to represent the communication overhead of the corresponding process core executing the sub-processes. 7.An electronic device comprising a memory and a processor, the electronic device characterized by, The memory stores a computer program, and the processor is configured to run the computer program to perform the steps of the process mapping optimization method based on the genetic algorithm in any one of claims 1 to 5.

8. A storage medium having stored thereon a computer program, characterized in that The computer program is executed by the processor to implement the steps of the process mapping optimization method based on the genetic algorithm in any one of claims 1 to 5.

Images