Process migration method and device, computing equipment cluster and related equipment

Abstract

The invention provides a process migration method and device, a computing equipment cluster and related equipment. When a first single board in the computing device cluster meets the migration triggering condition, whether each process in a plurality of processes included in a target job running on the first single board is in a migration consistency state or not is detected, and no ongoing communication process exists between the process in the migration consistency state and other processes; and when each process in the plurality of processes is in the migration consistency state, migrating the plurality of processes to at least one second single board included in the computing device cluster. Wherein the plurality of computing nodes on the first single board execute the plurality of processes included in the target job in parallel. Therefore, the computing node on the first single board does not need to be in an idle state for a long time after waiting for the completion of the execution of the target job, so that the operation and maintenance efficiency can be improved; in addition, the dependency of process migration on a bottom driver and a communication function library can be reduced.

Description

Technical Field

[0001] The present application relates to the technical field of fault handling, and in particular to a process migration method, apparatus, computing device cluster and related equipment. Background Art

[0002] At present, the integration density of computing nodes on a single board is getting higher and higher. For example, 10 computing nodes can be integrated on a single board. Among them, a computing node refers to a node composed of computing resources and memory resources, which can provide corresponding computing power to process jobs. High-density single boards can support more computing power within a limited physical space. Accordingly, multiple computing nodes on a single board can execute multiple processes in parallel. The multiple processes can belong to the same job, such as a message passing interface (MPI) job; or, the multiple processes can belong to different jobs.

[0003] In actual application scenarios, it is often necessary to operate and maintain the single board. For example, among the multiple computing nodes integrated on the single board, it is inevitable that some computing nodes will have hardware failures. At this time, the single board will no longer receive new jobs and will continue to execute the jobs that have been scheduled to the single board (specifically, the process corresponding to the execution of the job). When all the jobs on the single board are executed, the single board will be powered off for maintenance, such as replacing the processor or memory bar in the failed computing node on the single board. However, while waiting for the single board to complete processing all jobs, many computing nodes on the single board will be idle for a long time, resulting in serious resource waste. Summary of the invention

[0004] In view of this, the embodiment of the present application provides a process migration method for alleviating the problem of resource waste and improving operation and maintenance efficiency during the operation and maintenance of a single board. The present application also provides a corresponding apparatus, a computing device cluster, a computer-readable storage medium, and a computer program product.

[0005] In the first aspect, the embodiment of the present application provides a process migration method, which is applied to a computing device cluster including a first single board and at least one second single board. In actual application, the computing device cluster may also include other types of devices. Wherein, the first single board and at least one second single board each include multiple computing nodes, such as 16 computing nodes, etc., and the multiple computing nodes on the first single board execute multiple processes included in the target job in parallel, and the multiple processes may be all or part of the processes of the target job. In the process of migrating multiple processes on the first single board, when the first single board meets the migration trigger condition, it is detected whether each of the multiple processes running on the first single board is in a migration consistency state, wherein there is no ongoing communication process between the process in the migration consistency state and other processes, such as the process does not have a communication behavior with other processes, or the communication behavior ends, etc., so that when each of the multiple processes is in a migration consistency state, the multiple processes are migrated to at least one second single board in the computing device cluster, specifically, all of the multiple processes can be migrated to one second single board, or the multiple processes can be migrated to multiple different second single boards.

[0006] In this way, when the first single board is operated and maintained, it is not necessary to wait for the first single board to complete the target job, but to migrate the multiple processes included in the target job running on the first single board to other single boards, so that the computing nodes on the first single board do not need to be idle for a long time because of waiting for the target job to be executed, so as to avoid the problem of serious waste of resources on the first single board, so as to effectively shorten the operation and maintenance time of the first single board and improve the operation and maintenance efficiency. In addition, the multiple processes included in the target job are in a migration consistency state, that is, there is no data currently being communicated between the multiple processes, so that in the process of migrating the multiple processes included in the target job, it is not necessary to modify the underlying driver or modify the communication function library to obtain the data generated when the processes communicate with each other, so as to reduce the dependence of the process migration on the underlying driver and the communication function library. Further, when all the processes included in the target job are executed by the computing nodes on different single boards respectively, it is only necessary to migrate the processes located on the first single board to other single boards without migrating all the processes included in the target job, which can effectively reduce the number of processes required to be migrated, so as to effectively reduce the cost required for single board operation and maintenance, and reduce the impact of single board operation and maintenance on the execution of the target job.

[0007] In a possible implementation, when each of the multiple processes is in a migration consistency state, specifically, the code execution position corresponding to each of the multiple processes is located in the migratable interval, and the total amount of messages received between the process pairs with communication behaviors in the multiple processes is equal to the total amount of messages sent, wherein the program code located in the migratable interval does not include communication functions. In this way, there is currently no data being communicated between the multiple processes on the first single board, which makes it unnecessary to modify the underlying driver or modify the communication function library to obtain the data generated when the processes communicate with each other during the migration of the multiple processes included in the target job, thereby reducing the dependence of the process migration on the underlying driver and the communication function library.

[0008] In a possible implementation, when it is detected that the first process among multiple processes is not in a migration consistency state, the first process is continued to be executed until the first process is in a migration consistency state; wherein the code execution position corresponding to the first process that is not in a migration consistency state is located in a non-migratable interval, and the program code located in the non-migratable interval includes a communication function, or the total amount of messages received and sent between the first process that is not in a migration consistency state and the second process among the multiple processes is not equal. In this way, by detecting whether the execution position of the code is located in a communication function, and detecting the total amount of messages received and sent between the first process and other processes, it can be determined whether there are communication messages being transmitted between the first process and other processes, so that the first process can be subsequently migrated when there is no ongoing communication process between the first process and other processes, thereby reducing the dependence of the first process migration on the underlying driver and the communication function library.

[0009] In a possible implementation, a computing device cluster includes a coordination process, an agent process, and a transmission process, wherein the agent process and the transmission process are located on a first single board, and the coordination process is used to instruct the agent process to detect the migration status of a first process among multiple processes running on the first single board, and the number of message interactions between the first process and other processes among the multiple processes, wherein the migration status is used to identify whether the code execution position of the first process is located in a migration interval; and the coordination process is also used to determine that the first process is in a migration consistency state based on the migration status and the number of message interactions, and when each of the multiple processes is in the migration consistency state, instruct the transmission process to migrate the first process to the second single board. In this way, the computing device cluster can utilize the coordination process, the agent process, and the transmission process to achieve the migration of multiple processes included in the target job running on the first single board to other single boards without modifying the underlying driver and the communication function library, thereby reducing the dependence of process migration on the underlying driver and the communication function library.

[0010] In a possible implementation, the proxy process shares one or more of the following variables with the first process: an interval variable, used to indicate whether the code execution location corresponding to the first process is in a migration interval or a non-migratable interval; a step variable, used to indicate whether to continue executing the first process; a request variable, used to indicate whether there is a migration request to migrate the first process; a message record variable, used to indicate the number of messages sent and received by the first process. In this way, the proxy process can obtain relevant information of the first process by accessing these shared variables, or the proxy process can control the first process by setting the shared variables, such as controlling the first process to interrupt or continue execution.

[0011] In a possible implementation, the migration triggering condition for triggering the computing device cluster to migrate multiple processes on the first single board may include a hardware failure of a computing node on the first single board, or the computing device cluster obtains a migration request for multiple processes, which may be sent to the computing device cluster by an administrator through a login node, etc. In this way, by manually intervening as an administrator to determine the migration of processes on the first single board, it is possible to avoid frequent execution of process migration processes in the computing device cluster due to failures of multiple single boards in the computing device cluster, thereby minimizing the impact of single board operation and maintenance on the execution of jobs by the computing device cluster.

[0012] In a possible implementation manner, the target job running on the first board is an MPI job, or the target job may also be another type of job including multiple processes.

[0013] In the second aspect, the present application provides a process migration device, which is applied to a computing device cluster, wherein the computing device cluster includes a first single board and at least one second single board, the first single board and the at least one second single board each include multiple computing nodes, and the multiple computing nodes on the first single board execute multiple processes included in the target job in parallel; the process migration device includes: a coordination module, which is used to detect whether each of the multiple processes running on the first single board is in a migration consistency state when the first single board meets the migration trigger condition, wherein there is no ongoing communication process between the process in the migration consistency state and other processes; a transmission module, which is used to migrate the multiple processes to at least one second single board when each of the multiple processes is in a migration consistency state.

[0014] In one possible implementation, each of the multiple processes is in a migration consistency state, including: the code execution location corresponding to each of the multiple processes is located in a migratable interval, and the total amount of messages received between process pairs that have communication behaviors in the multiple processes is equal to the total amount of messages sent; wherein the program code located in the migratable interval does not include communication functions.

[0015] In a possible implementation, the process migration device also includes: an agent module, which is used to continue executing the first process among multiple processes until the first process is in a migration consistency state when it is detected that the first process among multiple processes is not in a migration consistency state; wherein the code execution position corresponding to the first process that is not in a migration consistency state is located in a non-migratable interval, and the program code located in the non-migratable interval includes a communication function, or the total amount of messages received between the first process that is not in a migration consistency state and the second process among the multiple processes is not equal to the total amount of messages sent.

[0016] In one possible implementation, a computing device cluster includes a coordination process, an agent process, and a transmission process, wherein the agent process and the transmission process are located on a first single board; a coordination module is used to use the coordination process to instruct the agent process in the agent module to detect the migration status of a first process among multiple processes running on the first single board, and the number of message interactions between the first process and other processes among the multiple processes, wherein the migration status is used to identify whether the code execution position of the first process is located in the migration interval; the coordination module is used to use the coordination process to determine that the first process is in a migration consistency state based on the migration status and the number of message interactions, and when each of the multiple processes is in a migration consistency state, instruct the transmission process in the transmission module to migrate the first process to a second single board.

[0017] In one possible implementation, the proxy process shares one or more of the following variables with the first process: an interval variable, used to indicate whether the code execution location corresponding to the first process is in a migratable interval or a non-migratable interval; a step variable, used to indicate whether to continue executing the first process; a request variable, used to indicate whether there is a migration request to migrate the first process; a message record variable, used to indicate the number of messages sent and the number of messages received by the first process.

[0018] In a possible implementation, the migration triggering condition includes that a computing node on the first board has a hardware failure, or that the computing device cluster obtains migration requests for multiple processes.

[0019] In a possible implementation, the target job is an MPI job.

[0020] It is worth noting that the process migration device provided in the second aspect corresponds to the process migration method provided in the first aspect. Therefore, the technical effects of the second aspect and any implementation method of the second aspect can refer to the technical effects of the first aspect or the corresponding implementation method of the first aspect.

[0021] In a third aspect, the present application provides a computing device, which includes a processor and a memory, the memory stores program instructions, and the processor runs the program instructions to execute the process migration method in the above-mentioned first aspect or any possible implementation of the first aspect.

[0022] In a fourth aspect, the present application provides a computing device cluster, which includes a first single board, at least one second single board, and a process migration device as described in the above-mentioned first aspect or any possible implementation of the first aspect; wherein the first single board and the at least one second single board each include multiple computing nodes.

[0023] In a fifth aspect, the present application provides a computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is executed on at least one computing device, the at least one computing device executes the method described in the first aspect or any one of the implementations of the first aspect.

[0024] In a sixth aspect, the present application provides a computer program product comprising instructions, which, when executed on at least one computing device, enables the at least one computing device to execute the method described in the first aspect or any one of the implementations of the first aspect.

[0025] Based on the implementations provided in the above aspects, the present application can also be further combined to provide more implementations. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a structural diagram of a computing device cluster;

[0027] Figure 2 It is a schematic diagram of a single board including multiple computing nodes;

[0028] Figure 3 A schematic diagram of the structure of an exemplary computing device cluster provided in an embodiment of the present application;

[0029] Figure 4 The process migration device 500 provided in the embodiment of the present application includes a schematic diagram of multiple processes;

[0030] Figure 5 A flowchart of a process migration method provided in an embodiment of the present application;

[0031] Fig. 6A A schematic diagram of dividing the program code of a process into a migratable interval and a non-migratable interval provided in an embodiment of the present application;

[0032] Figure 6B A schematic diagram of including offset addresses of multiple variables in a data structure defined in a configuration file;

[0033] Figure 7 A schematic diagram of migrating all mirror data corresponding to processes 1 to 9 to the same board;

[0034] Figure 8 A schematic diagram of migrating all mirror data corresponding to processes 1 to 9 to different boards;

[0035] Fig. 9 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] The scheme in the embodiments provided in this application will be described below in conjunction with the drawings in this application.

[0037] See also Figure 1 , which shows a schematic diagram of the structure of a computing device cluster. Figure 1 As shown, the computing device cluster 10 includes a job management system and multiple boards. Further, the computing device cluster 10 may also include multiple login nodes and multiple storage devices. For ease of understanding and description, Figure 1 The example includes two login nodes (login node 101 and login node 102), a job management system 200, four single boards (single boards 301 to 304), and two storage devices (storage device 401 and storage device 402), and different devices can communicate through a high-speed interconnection network.

[0038] Each login node may be implemented by a computing device, such as a server, and is used to connect with a client on the user side, which may be, for example, a web browser, or an application running on a user side device. In this way, the user may log in to the login node through the client, such as logging in to a user account, and submit a job on the login node.

[0039] The job management system 200 is used to receive jobs submitted by users through the login node 101, and schedule corresponding computing resources for the jobs in the computing device cluster 10. Specifically, the jobs can be scheduled to the corresponding boards, and at least one computing node on the boards can be designated to execute the jobs. In addition, the job management system 200 can also schedule corresponding storage resources (and network resources) for the jobs, such as scheduling part of the storage area in the storage device 401 to the jobs, so as to meet the storage resource requirements for executing the jobs.

[0040] Each single board may include multiple computing nodes, which may be composed of processors and memory and can provide corresponding computing power. For example, Figure 2The single board 301 shown may include 10 computing nodes, each of which is composed of a central processing unit (CPU) and a memory bar. Different computing nodes may be connected via a bus or a high-speed interconnection network, and the single board may provide unified power supply, cooling, interconnection and other capabilities for multiple computing nodes thereon. The job management system 200 may schedule one or more jobs submitted by a user to multiple computing nodes on the single board, and the multiple computing nodes may execute the jobs. In actual application, multiple single boards may be stored in a computer cabinet, and the computing device cluster 10 may include one or more computer cabinets.

[0041] Each storage device is used to store data, for example, intermediate data or execution results generated during the execution of a job. Exemplarily, multiple storage devices can be used to build a high-performance parallel file system (PFS) or to build other types of systems. Multiple storage devices can be stored in a storage cabinet, and the computing device cluster 10 can include one or more storage cabinets.

[0042] In actual application scenarios, among the multiple computing nodes integrated on each single board, it is inevitable that some computing nodes will have hardware failures, or some single boards in the computing device cluster 10 need to be replaced / checked. Figure 2 As shown, computing nodes 1 to 3 are used to execute job 1, which includes multiple processes, and each computing node executes at least one process included in job 1; the remaining computing nodes do not execute jobs and are in an idle state. If computing node 10 fails, since computing node 10 is difficult to remove separately from single board 301, during the process of repairing the single board 301, the job management system 200 will isolate the resources on the entire single board 301 and will no longer schedule new jobs to single board 301. At the same time, the computing nodes on single board 301 will continue to execute job 1. When all jobs on single board 301 are executed, single board 301 will be powered off for maintenance, such as replacement of the CPU or memory bar in computing node 10 by the operation and maintenance personnel.

[0043] However, it usually takes a long time for single board 301 to execute job 1, which means that during the period from the failure of computing node 10 to the power-off maintenance of single board 301, computing nodes 4 to 9 that are not executing jobs on single board 301 will be idle for a long time, resulting in serious waste of resources.

[0044] To this end, this application provides Figure 3 The computing device cluster 20 shown in FIG. Figure 1Based on the computing device cluster 10 shown, Figure 3 The computing device cluster 20 shown has a newly added process migration device 500, which can be connected to the job management system 200 and each single board respectively. When the single board 301 meets the migration trigger condition, such as a computing node on the single board 301 reports a fault or receives a fault handling instruction from the administrator for the single board 301, the process migration device 500 detects whether the various processes included in the job 1 running on the single board 301 are in a migration consistency state, wherein there is no ongoing communication process between the process in the migration consistency state and other processes, such as no communication behavior or the communication behavior ends, that is, there is no data being transmitted between the processes. When it is detected that each process is in a migration consistency state, the process migration device 500 migrates the multiple processes to other single boards. For example, the multiple processes included in job 1 can all be migrated to single board 302; or, the multiple processes included in job 1 can be migrated to single board 302 to single board 304 respectively, wherein different processes are migrated to different single boards.

[0045] In this way, when operating and maintaining single board 301 (such as operating and maintaining some computing nodes with hardware failures on single board 301), instead of waiting for single board 301 to complete job 1, multiple processes included in job 1 running on single board 301 are migrated to other single boards, which makes it unnecessary for the computing nodes on single board 301 to be idle for a long time because of waiting for job 1 to be completed, thereby avoiding the problem of serious waste of resources on single board 301. Usually, after single board 301 completes process migration, single board 301 can be directly powered off for maintenance, which can effectively shorten the operation and maintenance time of single board 301 and improve operation and maintenance efficiency.

[0046] Moreover, the multiple processes included in Job 1 are in a migration consistency state, that is, there is currently no data being communicated between the multiple processes. This means that in the process of migrating the multiple processes included in Job 1, there is no need to modify the underlying driver or the communication function library to obtain the data generated when the processes communicate with each other, thereby reducing the dependence of process migration on the underlying driver and the communication function library.

[0047] In addition, in actual application scenarios, the number of processes included in job 1 may be large, such as thousands of processes, and the multiple processes included in job 1 are respectively executed by computing nodes on different boards. When single board 301 needs to be operated and maintained, only part of the processes included in job 1 need to be migrated to other boards, without migrating all the processes included in job 1. This can effectively reduce the number of processes required to be migrated, thereby effectively reducing the cost required for single board operation and maintenance and reducing the impact of single board operation and maintenance on the execution of job 1.

[0048] For example, the above Figure 3 The process migration device 500 shown may be implemented by software or hardware.

[0049] In one example, when implemented by software, the process migration apparatus 500 may include code running on a computing device. Figure 4 As shown, the process migration device 500 may specifically include a coordination module 501, an agent module 502, and a transmission module 503. Each module included in the process migration device 500 may be implemented, for example, by a process, a thread, or by other means, such as by a virtual machine or a container. The agent module 502 and the transmission module 503 may be deployed on the single board 301 (and other single boards), such as Figure 4 As shown, the coordination module 501 can be deployed on a computing device independent of the single board, such as a server separately set up in the computing device cluster 20. The coordination module 501 is used to control and manage the proxy module 502 and the transmission module 503. Specifically, it can instruct the proxy module 502 to detect whether multiple processes on the single board 301 are in a migration consistency state, and when multiple processes are in a migration consistency state, instruct the transmission module 503 to migrate the multiple processes to other single boards. The coordination module 501, the proxy module 502, and the transmission module 503 can collaboratively complete the fault handling for the single board 301.

[0050] In another example, when implemented by hardware, the process migration device 500 may be implemented by a computing device including a processor, such as a server, an accelerator card, etc. The processor may be a CPU, an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), a system on chip (SoC), a software-defined infrastructure (SDI) chip, an artificial intelligence (AI) chip, a data processing unit (DPU), or any combination thereof. In addition, the number of processors included in the process migration device 500 may be one or more, and the type of processors included may be one or more. The number and type of processors may be set according to the business requirements of the actual application, and this embodiment does not limit this.

[0051] It is worth noting that the above Figure 3 The computing device cluster 20 shown is only used as an exemplary description. In other possible implementations, the computing device cluster 20 may include any number of boards and any number of storage devices; or, the computing device cluster 20 may have more types of devices, such as devices for storing scheduling logs, etc., to support the computing device cluster 20 with more functions; or, the computing device cluster 20 may have fewer devices, such as the computing device cluster 20 may not have login nodes, storage devices, etc. In addition, the above-mentioned process migration method can be applied to other types of clusters, and the specific structure of the cluster and its applicable application scenarios are not limited in this application.

[0052] Next, various non-limiting specific implementation methods of the process migration are described in detail.

[0053] See also Figure 5 , is a flow chart of a process migration method in an embodiment of the present application. The method can be applied to the above Figure 3 The computing device cluster 20 shown in FIG. 1 may also be applied to computing device clusters of other possible structures. Figure 3 The computing device cluster 20 is shown, and migrating multiple processes included in the target job on the single board 301 to other single boards is used as an example for explanation. Figure 5 The process migration method shown may specifically include:

[0054] S501: The job management system 200 determines that a migration trigger condition is satisfied for the board 301.

[0055] In actual application, the user may submit a target job to the computing device cluster 20, specifically by submitting the job to the job management system 200 through the login node 101, so as to request the use of the high performance computing service of the computing device cluster 20. The target job includes multiple processes, such as an MPI job, or other types of jobs including multiple processes, which are not limited in this embodiment.

[0056] After receiving the target job, the job management system 200 can schedule the computing nodes on one or more boards to the target job, so as to use multiple computing nodes to execute multiple processes included in the target job in parallel. Figure 2 The computing nodes 1 to 3 in the single board 301 execute the multiple processes included in the target job in parallel, and each computing node executes at least one process of the target job. In addition, the multiple processes executed on the single board 301 can be all the processes included in the target job, or part of the processes included in the target job (the remaining part of the processes can be scheduled to other single boards). For ease of description, the multiple processes included in the target job on the single board 301 are briefly described as the multiple processes included in the target job.

[0057] In actual application scenarios, it is usually necessary to operate and maintain the single board 301. For example, during the operation of multiple computing nodes on the single board 301, it is inevitable that some computing nodes will have hardware failures, such as physical damage to the memory bars in some computing nodes. Alternatively, the operation and maintenance personnel may need to replace some single boards in the computing device cluster 20, such as replacing a single board including 16 computing nodes with a single board including 31 computing nodes; or the operation and maintenance personnel may need to power off some single boards in the computing device cluster 20 for inspection.

[0058] For ease of understanding, this embodiment takes the hardware failure of some computing nodes on the single board 301 as an example. The failed computing node can actively report the failure information to the job management system 200; or, the job management system 200 learns about the failed computing node by polling the computing node or other methods. Among them, when the computing node with hardware failure is a computing node executing a process, the job management system 200 can use the computing node that has not failed on the single board 301 or other single boards to resume the execution of the process. Specifically, it can use the process data recorded by the computing node before the failure to restart the process on other computing nodes to avoid the process not being executed and causing the target job to fail to execute.

[0059] In this embodiment, the following implementation examples for determining whether the board 301 meets the migration triggering condition are provided.

[0060] In the first implementation example, after sensing that a computing node on the board 301 fails, the job management system 200 can determine that the board 301 meets the migration trigger condition, so as to trigger the process of migrating multiple processes on the board 301 to other boards.

[0061] In the second implementation example, after the job management system 200 senses that a computing node on the single board 301 has failed, the fault information of the computing node can be fed back to the administrator, for example, by presenting an interactive interface to the administrator through the login node 101, and the interactive interface includes the name of the failed computing node, the cause of the failure, the failure time and other related information. In this way, the administrator determines whether to power off the single board 301 for maintenance based on the information presented by the login node 101, and after determining to maintain the single board 301, the migration request for multiple processes on the single board 301 can be sent to the job management system 200 through the login node 101, so that the job management system 200 can determine that the single board 301 meets the migration triggering condition after receiving the migration request. In this way, by determining to repair the single board 301 through the human intervention of the administrator, it is possible to avoid the frequent execution of the process migration process in the computing device cluster 20 due to the failure of multiple single boards in the computing device cluster 20, thereby minimizing the impact of the fault repair on the execution of jobs by the computing device cluster 20. For example, for some boards with computing node failures, the administrator may not need to repair them, or may repair them within a specified time period (such as a time period when the load of the computing device cluster 20 is relatively small).

[0062] After determining that the single board 301 meets the migration triggering condition, the job management system 200 may notify the process migration device 500 so that the process migration device 500 performs the subsequent process migration process for the single board 301. In this embodiment, the coordination module 501, the agent module 502, and the transmission module 503 included in the process migration device 500 cooperate to complete the process of migrating the process on the single board 301. Exemplarily, the coordination module 501, the agent module 502, and the transmission module 503 may be implemented by an out-of-band daemon process independent of the process of the target job, that is, the coordination module 501 may be implemented by a coordination process, the agent module 502 may be implemented by an agent process, and the transmission module 503 may be implemented by a transmission process.

[0063] S502: The coordination module 501 sends a detection signal to the proxy module 502, which is used to instruct the proxy module 502 to detect the migration status of each process on the single board 301 and the number of message interactions between each process and other processes, wherein the migration status of each process is used to identify whether the code execution position of the process is in the migration interval.

[0064] S503 : the proxy module 502 performs status detection on each process on the single board 301 to obtain a detection result, which is used to indicate the migration status and message interaction quantity of each process, and feeds back the detection result to the coordination module 501 .

[0065] In this embodiment, the program codes corresponding to each process included in the target job can be divided into two types of intervals, one is a migratable interval and the other is a non-migratable interval. Fig. 6A As shown, the program code in the migratable interval does not include the communication function, and the program code in the non-migratable interval includes the communication function, such as the non-migratable interval 2 includes the communication function k, and the non-migratable interval 4 includes the communication function k+1, etc. When the process executes to the migratable interval, the process does not currently exchange data with other processes. On the contrary, when the process executes to the non-migratable interval, the process can communicate data with other processes based on the communication function when being executed; at this time, if the process is directly migrated from the single board 301 to other single boards, it is usually necessary to modify the underlying driver to obtain the communication message being transmitted between the process and other processes through the pipeline, and it is necessary to modify the communication function library to obtain the state data cached in the internal buffer of the communication function library (such as the communication progress of the process, etc.) and other information, so as to avoid the loss of communication messages between processes due to process migration, which will increase the dependence of process migration on the underlying driver or communication function library.

[0066] The coordination module 501 can instruct the proxy module 502 to detect whether the current code execution position of each process of the target job (on the single board 301) is located in the migratable interval. In specific implementation, the coordination module 501 can send a detection signal to the proxy module 502 to detect the migratable state of each process of the target job. The proxy module 502 can detect the current code execution position of each process and determine whether the code execution position is located in the communication function. If so, it is determined that the code execution position is located in the non-migratable interval, and the migratable state is used to identify this. At this time, the value of the migratable state can be, for example, "un-migratable". If not, it is determined that the code execution position is located in the migratable interval. At this time, the value of the migratable state can be, for example, "migratable". Then, the proxy module 502 can feed back the migratable state of each process to the coordination module 501.

[0067] As an implementation example, the proxy module 502 can share interval variables and request variables with the first process (one of the processes included in the target job) on the single board 301. The interval variable is used to identify whether the code execution position corresponding to the first process is located in the migratable interval or the non-migratable interval, and the request variable is used to identify whether there is a request to migrate the first process. After receiving the detection signal, the proxy module 502 can update the value of the request variable. For example, the request variable can be defined as "have_request", and the value of the request variable can be updated from "False" to "True" to represent the current request to migrate the first process. In addition, the proxy module 502 will also send an update signal to the first process to instruct the first process to update the interval variable. The first process (specifically, the computing node executing the first process) responds to the update signal, checks the request variable, and determines that the value of the request variable is "True". Then, the first process can determine whether the code execution position is located in the migratable interval, and update the value of the interval variable according to the determination result. For example, an interval variable can be defined as "interval". Then, when the code execution location is in a migratable interval, the value of the interval variable "interval" can be updated to "migratable"; when the code execution location is in a non-migratable interval, the value of the interval variable "interval" can be updated to "un-migratable". In this way, the proxy module 502 can access the value of the shared interval variable to obtain the migratable state of the first process.

[0068] In actual application, the administrator may provide a configuration file to the computing device cluster 20, in which the addresses of multiple variables shared between the proxy module 502 and the first process are defined. For example, the configuration file in json format provided by the administrator may define the following: Figure 6BThe data structure shown in FIG. 5 is a data structure in which a shared key (shared_key) is defined, and the shared key is used to support the proxy module 502 and the first process to share the same memory area. In addition, the data structure also defines "have_request_offset" and "interval_offset", where "have_request_offset" is the offset address of the request variable "have_request" and "interval_offset" is the offset address of the interval variable "interval". In this way, the proxy module 502 and the first process can modify the values ​​of the request variable and the interval variable according to the offset addresses indicated by "have_request_offset" and "interval_offset" defined in the data structure.

[0069] In addition, the detection signal sent by the coordination module 501 to the proxy module 502 is also used to instruct the proxy module 502 to feedback the number of message interactions between each process of the target job (on the single board 301) and other processes, and the number of message interactions includes the total number of messages sent by the process to each other process, and the total number of messages received by the process from each other process. For example, process A sends 10 messages to process B and receives 8 messages from process B, and process A also sends 15 messages to process C and receives 5 messages from process B. Then the number of message interactions of process A fed back by the coordination module 501 to the proxy module 502 can be shown in the following Table 1.

[0070] Table 1

[0071] Process B Process C send 10 15 take over 8 5

[0072] As an implementation example, the proxy module 502 can share a message record variable with the first process on the single board 301. The message record variable can include at least one group of variables, each group of variables corresponds to other processes included in the target job that communicates with the first process, such as the second process. Each group of variables can include a variable "R" and a variable "S. Taking a group of variables corresponding to the second process as an example, the "R" in the group of variables is used for the number of messages sent by the first process to the second process, and the "S" in the group of variables is used to identify the number of messages received by the first process from the second process. After sending or receiving a message, the first process will locally count the number of messages communicated with the first process, and use the number of messages to update the value of the message record variable in real time. For example, the first process can be based on Figure 6BThe message recording mode indicated by "msg_prof" defined in the data structure shown in the figure modifies the value of the message recording variable, wherein the "RS" mode characterizes the number of received communication messages and the number of sent communication messages recorded in sequence; the "SR" mode characterizes the number of sent communication messages and the number of received communication messages recorded in sequence. In this way, the proxy module 502 can obtain the number of message interactions corresponding to the first process by accessing the value of the message recording variable. Then, the proxy module 502 can feed back the value of the accessed message recording variable to the coordination module 501.

[0073] In actual application, the single board 301 can create a proxy module 502 for each process running on it. The proxy module 502 can be when the multiple processes included in the target job are scheduled to the single board 301, and the job management system 200 instructs the single board 301 to create corresponding proxy modules 502 for the multiple processes. Each proxy module 502 is used to detect the migration status and message interaction number of a process of the target job. Then, the coordination module 501 can send a detection signal to the proxy module 502 corresponding to each process on the single board 301. Correspondingly, each proxy module 502 can return information such as the migration status and message interaction number of a corresponding process to the coordination module 501.

[0074] Alternatively, only one proxy module 502 may be created on the single board 301, so that the proxy module 502 can be responsible for detecting the migration states and message interaction quantities corresponding to multiple processes included in the target job running on the single board 301, which is not limited in this embodiment.

[0075] S504: The coordination module 501 determines whether the multiple processes included in the target job are in a migration consistency state according to the received detection result.

[0076] Among them, there is no ongoing communication process between the process in the migration consistency state and other processes in the multiple processes, such as there is no communication behavior, or the communication behavior is terminated, that is, the code logic currently executed by the process is not the logic for data interaction with other processes.

[0077] In a possible implementation, for the first process included in the target job, the coordination module 501 can determine whether the code execution position of the first process is located in the migratable interval according to the migratable state included in the detection result, and the coordination module 501 also determines whether the number of messages that have been exchanged between the first process and other processes is the same according to the number of message interactions included in the detection result, specifically to verify whether the number of messages sent by the first process to other processes is the same as the number of messages received by other processes, and to verify whether the number of messages received by the first process is the same as the number of messages sent by other processes. In this embodiment, it is defined that when the code execution position of the first process is located in the migratable interval and the number of messages that have been exchanged between the first process and other processes is the same, the first process is in a migration consistency state. On the contrary, if the code execution position of the first process is located in the non-migratable interval, or the number of messages sent by the first process is inconsistent with the number of messages received by other processes, or the number of messages received by the first process is inconsistent with the number of messages sent by other processes, then it is defined that the first process is not in a migration consistency state.

[0078] Referring to the above implementation, the coordination module 501 may verify whether all processes included in the target job are in a migration consistency state according to the detection result.

[0079] Furthermore, when some processes have not entered the migration consistency state, it indicates that data is currently being exchanged between these processes. At this time, the coordination module 501 can use the proxy module 502 to control the computing nodes on the single board 301 to continue to execute these processes, so that the data exchange process between these processes ends and enters the migration consistency state. For ease of understanding, this embodiment takes the first process and the second process included in the target job not entering the migration consistency state as an example for explanation. Figure 5 The method flow shown continues to include:

[0080] S505: When it is determined that the first process and the second process included in the target job have not entered the migration consistency state, the coordination module 501 sends a pace signal to the agent module 502, where the pace signal is used to instruct the agent module 502 to push the first process to continue execution.

[0081] S506: The proxy module 502 instructs the computing node on the board 301 to continue executing the first process.

[0082] At this time, the value of the interval variable shared between the proxy module 502 and the first process is "un-migratable".

[0083] In a possible implementation, the proxy module 502 and the first process may share a step variable, which is used to identify whether to execute the first process. When the proxy module 502 receives the step signal, it may update the value of the step variable, which may be defined as "step", so that the proxy module 502 may update the value of the step variable from "False" to "True". Figure 6B The computing node executing the first process, when determining that the value of the step variable is "True", continues to execute the first process so that the code execution position of the first process enters the migratable interval from the non-migratable interval, as shown by Fig. 6A The non-migratable interval 2 shown enters the migratable interval 3, etc. Then, the computing node executing the first process can re-update the value of the step variable from "True" to "False", and update the value of the shared interval variable from "un-migratable" to "migratable". In this way, the proxy module 502 can determine that the first process has currently completed execution by detecting the value of the step variable, and determine that the code execution position of the first process is in the migratable interval by detecting the value of the interval variable. In actual application, the communication function included in the program code of the first process can be encapsulated to increase the function of recording the switching between the migratable interval and the non-migratable interval before entering the communication function and after exiting the communication function.

[0084] Exemplarily, while the computing node continues to execute the first process, it can also update the value of the message record variable shared between the proxy module 502 and the first process according to the number of messages interacted with the second process, so as to record the number of messages interacted between the first process and the second process. In actual application, the communication function included in the program code of the first process can be encapsulated to add the function of recording the number of sent and received communication messages in the communication function.

[0085] S507 : After the first process is continued to be executed, the proxy module 502 obtains the migration status of the first process and the number of message interactions between the first process and the second process, and feeds them back to the coordination module 501 .

[0086] In specific implementation, the proxy module 502 can determine the migration status of the first process by accessing the shared interval variable, and determine the number of message interactions between the first process and the second process by accessing the shared message record variable, so that the updated migration status and the number of message interactions can be fed back to the coordination module 501.

[0087] S508: The coordination module 501 determines that the first process is in a migration consistency state according to the newly received migration status and the number of message interactions of the first process.

[0088] The specific implementation process of step S508 can be found in the relevant description of the aforementioned step S504, which will not be repeated here.

[0089] In actual application, if the coordination module 501 determines that the first process is still not in a migration consistency state based on the newly received migratable state and the number of message interactions of the first process, such as when the computing node is already in the migratable interval at the code execution position of the first process, the first process may still not receive all the messages sent by the second process, such as some messages sent by the second process have not been fully transmitted to the first process due to transmission congestion, etc. Then, the coordination module 501 can repeat the above steps S505 to S508 until the first process is in a migration consistency state.

[0090] For example, the coordination module 501 may send a pacing signal to the proxy module 502 again, so as to instruct the computing node to continue executing the first process to the next interval through the proxy module 502. At this time, the code execution position of the first process will enter the non-migratable interval from the migratable interval again, such as from Fig. 6A The migratable interval in the first process enters the non-migratable interval 4. Therefore, the coordination module 501 will determine that the first process is still not in a migration consistency state, and continue to send the next step signal to the proxy module 502, so that the proxy module 502 will again instruct the computing node to continue executing the first process. In this way, the code execution position of the first process will enter the migratable interval 5 from the non-migratable interval 4. At this time, the coordination module 501 can determine whether the first process is in a migration consistency state based on the migratable state and the number of message interactions when the code execution position of the first process is in the migratable interval 5. If not, the coordination module 501 can repeat the above process until it is determined that the first process is in a migration consistency state. In this way, after multiple iterations of executing the first process, the first process will eventually be in a migration consistency state, that is, the code execution position of the first process is in the migratable interval, and the number of messages received by the first process is consistent with the number of messages sent by the second process.

[0091] Similarly, the coordination module 501 may refer to the process described in the above steps S505 to S508 to control the computing nodes on the board 301 to continue executing the second process (and other processes that are not in a migration consistency state) until the second process is in a migration consistency state.

[0092] S509: When it is determined that all the multiple processes included in the target job on the board 301 are in a migration consistency state, the coordination module 501 may instruct the transmission module 503 to migrate the multiple processes included in the target job to other boards.

[0093] It can be understood that since the multiple processes included in the target job are in a migration consistency state, when the transmission module 503 migrates the multiple processes to other boards, there is no data for communication between the multiple processes. At this time, the transmission module 503 does not need to modify the underlying driver and communication function library, thereby reducing the dependence of process migration on the underlying driver and communication function library.

[0094] As an implementation example of a migration process, the coordination module 501 can suspend the target job on the single board 301, that is, interrupt the execution of multiple processes included in the target job to prepare for the migration of multiple processes included in the target job. For example, for the first process included in the target job, a signal processing function for executing the interrupt can be added to the first process, and when the coordination module 501 determines that the first process is in a migration consistency state, the coordination module 501 can send an interrupt signal to the first process through the proxy module 502. After receiving the interrupt signal, the first process can execute the corresponding signal processing function, and set the values ​​of multiple variables such as defined interval variables and step variables in the signal processing function to preset values, so that the computing node suspends the execution of the first process according to the values ​​of multiple variables such as the interval variables and step variables. In this way, the interruption of the execution of the first process can be realized. Then, the coordination module 501 can generate a mirror signal and send the mirror signal to the proxy module 502.

[0095] After receiving the mirror signal, the proxy module 502 can first close the connection between the multiple processes to be migrated and other processes. For example, the target job includes at least one process on other single boards in addition to the multiple processes on the single board 301. Therefore, the proxy module 502 can disconnect the connection between the multiple processes on the single board 301 and the processes on other single boards to avoid the communication between the processes on other single boards and the processes on the single board 301. Then, the proxy module 502 can generate the mirror data (i.e., the copies of the multiple processes) of the multiple processes on the single board 301, and save the mirror data on the single board 301. Wherein, the mirror data includes the program code of the multiple processes, the results (such as temporary data, etc.) generated by the multiple processes during execution, etc. The proxy module 502 notifies the coordination module 501 that the mirror data of the multiple processes has been saved on the single board 301.

[0096] Next, the coordination module 501 notifies the transmission module 503 to transmit the mirror data. Accordingly, the transmission module 503 transmits the mirror data stored on the single board 301 to other single boards. Among them, the mirror data of multiple processes can be migrated to the same single board. For example, Figure 7 As shown, the transmission module 503 can migrate all the mirror data corresponding to processes 1 to 9 to the single board 302, etc. In this way, multiple subsequent processes can still continue to run on the same single board, which can reduce the communication delay between the multiple processes, thereby improving the execution efficiency of the multiple processes, that is, improving the performance of the target job. Alternatively, the mirror data of multiple processes can be transmitted to different single boards, for example, Figure 8 As shown, the mirror data of processes 1 to 5 included in the target job can be transmitted to single board 302, the mirror data of processes 6 and 7 included in the target job can be transmitted to single board 303, and the mirror data of processes 8 and 9 included in the target job can be transmitted to single board 304. In this way, by using computing nodes on multiple single boards to execute multiple processes included in the target job, resource fragmentation on single board 302 to single board 304 can be reduced, thereby improving resource utilization of single board 302 to single board 304. Among them, the administrator can configure which single board or boards the mirror data of multiple processes included in the target job are migrated to, such as the administrator can configure which single board or boards the multiple processes are migrated to by logging in to the node; or, the job management system 200 can automatically allocate resources according to the number of available resources on each single board to determine the single board where the multiple processes are located after the process migration, which is not limited in this embodiment. Generally, before migrating multiple processes of the target job, the job management system 200 may first determine whether the number of available resources on the single board 302 to the single board 304 meets the resource requirements of the multiple processes of the target job. If so, the multiple processes are determined to be migrated; if not, the migration process may not be performed, and the administrator is prompted that the current available resources are insufficient and the migration fails.

[0097] After completing the transmission of the mirror data, the transmission module 503 notifies the coordination module 501 that the transmission of the mirror data is completed, and the coordination module 501 notifies it to the job management system 200. The job management system 200 notifies the coordination module 501 to start the migration process, so that the coordination module 501 can instruct the agent module 502 to use the mirror data to start the multiple processes included in the target job, such as using the mirror data to start all the multiple processes migrated to the single board 302, or using the mirror data to start the multiple processes migrated to the single board 302 to the single board 304, etc., and instruct the agent module 502 to establish a connection between the started process and the other processes included in the target job, so as to restore the target job to continue running. After starting the job, the agent module 502 can feed back the startup status of each process to the coordination module 501, and the coordination module 501 further feeds it back to the job management system 200, so that the job management system 200 can notify the administrator whether the multiple processes included in the target job are successfully migrated or failed to migrate.

[0098] When multiple processes on the single board 301 are successfully migrated to other single boards, the single board 301 can be directly powered off for maintenance, such as the administrator replacing the hardware in the computing node that fails on the single board 301. In this way, the fault repair of the single board 301 can be achieved.

[0099] It is worth noting that the above Figure 5 The process migration method shown is explained by assuming that only the target job is running on the single board 301 and multiple processes of the target job are migrated. When multiple jobs are running on the single board 301, the process migration device 500 can migrate all processes included in all jobs to other single boards, and all jobs can be migrated to the same single board or to different single boards. The specific implementation process can refer to the above description of the relevant parts of migrating the process of the target job, and will not be repeated here.

[0100] Above Figure 5 In the illustrated embodiment, the process migration apparatus 500 involved in the process migration process may be software configured on a computing device, and by running the software on the computing device, the computing device may implement the functions of the process migration apparatus 500. Below, based on the perspective of hardware device implementation, the process migration apparatus 500 involved in the process migration process is described in detail.

[0101] Fig. 9 The schematic diagram of the structure of a computing device is shown. The process migration device 500 can be deployed on the computing device. The computing device can be a computing device deployed separately in a computing device cluster. Specifically, it can be used to implement the above Figure 5 The function of the process migration device 500 in the illustrated embodiment.

[0102] like Fig. 9 As shown, computing device 900 includes processor 910, memory 920, communication interface 930 and bus 940. Processor 910, memory 920 and communication interface 930 communicate through bus 940. Bus 940 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig. 9 The communication interface 930 is used for external communication, for example, receiving migration requests for multiple processes on the board 301 sent by an administrator through a login node.

[0103] Among them, the processor 910 can be a central processing unit (CPU), an application specific integrated circuit (ASIC), a graphics processing unit (GPU) or one or more integrated circuits. The processor 910 can also be an integrated circuit chip with signal processing capabilities. In the implementation process, the functions of the server 200 can be completed by the hardware integrated logic circuit or software instructions in the processor 910. The processor 910 can also be a general processor, a digital signal process (DSP), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, and can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application. Among them, the general processor can be a microprocessor or the processor can also be any conventional processor, etc., and the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to execute, or it can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 920 , and the processor 910 reads the information in the memory 920 and completes part or all of the functions in the server 200 in combination with its hardware.

[0104] The memory 920 may include a volatile memory, such as a random access memory (RAM), or a non-volatile memory, such as a read-only memory (ROM), a flash memory, a HDD, or a SSD.

[0105] The memory 920 stores executable codes, and the processor 910 executes the executable codes to execute the method executed by the aforementioned process migration device 500 .

[0106] Specifically, in implementing Figure 5 In the case of the embodiment shown, and Figure 5 In the case where the process migration device 500 described in the illustrated embodiment is implemented by software, the execution Figure 5 The software or program code required for the functions of the process migration device 500 is stored in the memory 920. The interaction between the process migration device 500 and other devices is realized through the communication interface 930. The processor is used to execute the instructions in the memory 920 to implement the method executed by the process migration device 500.

[0107] In addition, an embodiment of the present application also provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on one or more computing devices, the one or more computing devices execute the method executed by the process migration apparatus 500 of the above embodiment.

[0108] In addition, the embodiment of the present application further provides a computer program product, and when the computer program product is executed by one or more computing devices, the one or more computing devices execute any of the aforementioned process migration methods. The computer program product may be a software installation package, and when any of the aforementioned process migration methods is required, the computer program product may be downloaded and executed on a computer.

[0109] It should also be noted that the embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the embodiment drawings provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0110] The terms "first", "second", etc. in the specification, claims and the above drawings of this application are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances. This is just a way of distinguishing objects with the same attributes when describing the embodiments of this application.

[0111] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0112] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0113] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A process migration method, characterized in that: The method is applied to a computing device cluster, the computing device cluster includes a first single board and at least one second single board, the first single board and the at least one second single board each include a plurality of computing nodes, and the plurality of computing nodes on the first single board execute a plurality of processes included in a target job in parallel; The method comprises: When the first board meets the migration trigger condition, detecting whether each of the multiple processes running on the first board is in a migration consistency state, wherein there is no ongoing communication process between the process in the migration consistency state and other processes; When each of the multiple processes is in the migration consistency state, the multiple processes are migrated to the at least one second board.

2. The method according to claim 1, characterized in that Each of the multiple processes is in the migration consistency state, including: The code execution position corresponding to each process in the plurality of processes is located in the migration range, and the total amount of messages received between the process pairs having communication behaviors in the plurality of processes is equal to the total amount of messages sent; The program code in the migratable area does not include a communication function.

3. The method according to claim 2, characterized in that The method further comprises: When it is detected that a first process among the multiple processes is not in the migration consistency state, continue to execute the first process until the first process is in the migration consistency state; Among them, the code execution position corresponding to the first process that is not in the migration consistency state is located in a non-migratable interval, the program code located in the non-migratable interval includes a communication function, or the total amount of messages received between the first process that is not in the migration consistency state and the second process among the multiple processes is not equal to the total amount of messages sent.

4. The method according to claim 2 or 3, characterized in that: The computing device cluster includes a coordination process, an agent process, and a transmission process, and the agent process and the transmission process are located on the first board; The coordination process is used to instruct the agent process to detect the migration status of the first process among the multiple processes running on the first single board and the number of message interactions between the first process and other processes among the multiple processes, wherein the migration status is used to identify whether the code execution position of the first process is located in the migration interval; The coordination process is also used to determine that the first process is in the migration consistency state based on the migration state and the number of message interactions, and when each of the multiple processes is in the migration consistency state, instruct the transmission process to migrate the first process to the second board.

5. The method according to claim 4, characterized in that The proxy process shares one or more of the following variables with the first process: An interval variable, used to indicate that the code execution position corresponding to the first process is in a migratable interval or a non-migratable interval; A step variable, used to indicate whether to continue executing the first process; a request variable, used to indicate whether there is a migration request to migrate the first process; The message record variable is used to indicate the number of messages sent and the number of messages received by the first process.

6. The method according to any one of claims 1 to 5, characterized in that: The migration triggering condition includes that a hardware failure occurs in a computing node of the first board, or the computing device cluster obtains a migration request for the multiple processes.

7. The method according to any one of claims 1 to 6, characterized in that: The target job is a message passing interface MPI job.

8. A process migration device, characterized in that: The process migration device is applied to a computing device cluster, the computing device cluster includes a first single board and at least one second single board, the first single board and the at least one second single board each include a plurality of computing nodes, and the plurality of computing nodes on the first single board execute a plurality of processes included in a target job in parallel; The process migration device comprises: a coordination module, configured to detect, when the first board meets a migration trigger condition, whether each of the multiple processes running on the first board is in a migration consistency state, wherein there is no ongoing communication process between the process in the migration consistency state and other processes; The transmission module is used to migrate the multiple processes to the at least one second board when each process in the multiple processes is in the migration consistency state.

9. The process migration device according to claim 8, characterized in that: Each of the multiple processes is in the migration consistency state, including: The code execution position corresponding to each process in the plurality of processes is located in the migration range, and the total amount of messages received between the process pairs having communication behaviors in the plurality of processes is equal to the total amount of messages sent; The program code in the migratable area does not include a communication function.

10. The process migration device according to claim 9, characterized in that: The process migration device also includes: an agent module, configured to, when detecting that a first process among the multiple processes is not in the migration consistency state, continue to execute the first process until the first process is in the migration consistency state; Among them, the code execution position corresponding to the first process that is not in the migration consistency state is located in a non-migratable interval, the program code located in the non-migratable interval includes a communication function, or the total amount of messages received between the first process that is not in the migration consistency state and the second process among the multiple processes is not equal to the total amount of messages sent.

11. The process migration device according to claim 10, characterized in that: The computing device cluster includes a coordination process, an agent process, and a transmission process, and the agent process and the transmission process are located on the first board; The coordination module is used to use the coordination process to instruct the agent process in the agent module to detect the migration state of the first process among the multiple processes running on the first single board and the number of message interactions between the first process and other processes among the multiple processes, wherein the migration state is used to identify whether the code execution position of the first process is located in the migration interval; The coordination module is used to use the coordination process to determine that the first process is in the migration consistency state according to the migratable state and the number of message interactions, and when each of the multiple processes is in the migration consistency state, instruct the transmission process in the transmission module to migrate the first process to the second board.

12. The process migration device according to claim 11, characterized in that: The proxy process shares one or more of the following variables with the first process: An interval variable, used to indicate that the code execution position corresponding to the first process is in a migratable interval or a non-migratable interval; A step variable, used to indicate whether to continue executing the first process; a request variable, used to indicate whether there is a migration request to migrate the first process; The message record variable is used to indicate the number of messages sent and the number of messages received by the first process.

13. The process migration device according to any one of claims 8 to 12, characterized in that: The migration triggering condition includes that a hardware failure occurs in a computing node of the first board, or the computing device cluster obtains a migration request for the multiple processes.

14. The process migration device according to any one of claims 8 to 13, characterized in that: The target job is a message passing interface MPI job.

15. A computing device, characterized in that: The computing device comprises a processor and a memory, wherein the memory stores program instructions, and the processor runs the program instructions to execute the method according to any one of claims 1 to 7.

16. A computing device cluster, characterized in that: The computing device cluster comprises a first single board, at least one second single board, and a process migration device as claimed in any one of claims 8 to 14; Wherein, the first single board and the at least one second single board each include a plurality of computing nodes.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, which, when executed on at least one computing device, enable the at least one computing device to perform the method according to any one of claims 1 to 7.

Images