Parallel sorting fault node task migration method, device, equipment and storage medium

Abstract

The present invention belongs to the field of geophysical exploration technology, and discloses a parallel sorting fault node task migration method, device, equipment and storage medium, wherein the parallel sorting fault node task migration method first initializes valid nodes to form a valid node list, and according to the valid node list, the main process communicates with each background service process, and each background service process creates and starts each sub-service process, and the main process sets a keep-alive mechanism for each sub-service process, monitors the node status of the sub-service process, and processes abnormal nodes, and records the number of times each sub-job has been run. When a certain sub-job fails after being repeatedly run three times, the parallel sorting job is terminated. The present invention is suitable for parallel sorting fault node task migration, can provide a fault-tolerant mechanism for parallel sorting of seismic data, migrate erroneous sub-tasks, and provide support for the processing production efficiency of massive seismic data.

Description

Technical Field

[0001] The present invention belongs to the technical field of geophysical exploration, and relates to a parallel sorting fault node task migration method, device, equipment and storage medium. Background Art

[0002] With the continuous advancement of exploration technology, more and more acquisition technologies have been developed, reducing exploration costs and improving exploration efficiency. For example, ultra-efficient aliasing acquisition technology collects data through full-node instruments, significantly improving exploration efficiency and accelerating the rapid growth of collected data volume. Some projects download aliasing data at the daily level of hundreds of TB, and the total data volume reaches PB.

[0003] Although acquisition technology has developed rapidly, massive seismic data has brought great pressure and challenges to the processing business. The process of extracting tracks (seismic data sorting) in the production of seismic data projects is a key step in seismic data processing. Extracting tracks is an IO-intensive processing task that requires a large amount of hardware resources. At present, there are two ways to sort seismic data in the industry: keyword indexing and multi-node parallelization. The indexing method requires creating an index for each seismic data first, and using the disk file random positioning reading method according to the data track number in each index. However, the data track reading efficiency is low and it will reduce the access bandwidth of the storage system. Therefore, the keyword indexing method is not suitable for sorting massive seismic data. The multi-node parallel method has a complex sorting method and needs to use memory and temporary files to convert random reading and writing into sequential reading and writing, but the sorting efficiency is high. The GeoEast system already has a seismic data parallel extraction module (PGeoSortIn), which is a multi-node parallel module. However, the current version of the PGeoSortIn module does not have a fault-tolerant function during operation. Once an abnormal process on a cluster node fails, the entire extraction operation will fail and the operation must be re-run, affecting the stability of seismic data sorting and reducing the work efficiency of production users.

[0004] At present, for efficient sorting of massive seismic data, hundreds of nodes must be coordinated to complete the task in a suitable time. At the same time, the sorting module is required to continue to operate normally when a node fails. Therefore, it is very necessary to provide a parallel sorting fault node task migration method that can tolerate faulty tasks. Summary of the invention

[0005] The purpose of the present invention is to provide a parallel sorting fault node task migration method, which supports the processing production efficiency of massive seismic data by migrating the sub-process tasks that have errors in the parallel sorting operation of seismic data;

[0006] Another object of the present invention is to provide a parallel sorting fault node task migration device, equipment and storage medium.

[0007] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0008] A parallel sorting fault node task migration method comprises the following steps:

[0009] S1. Initialize valid nodes to form a valid node list. This step specifically includes:

[0010] S1.1. Parse the node file provided by the multi-node parallel sorting job in the GeoEast system, remove duplicate nodes, and form a node list container AllHosts;

[0011] S1.2, initialize the invalid node container FailHosts to empty;

[0012] S1.3, the main process creates a child process for each node according to the node list container AllHosts, starts the background service process on each child process, and adds the node that fails to start the background service process to the invalid node container FailHosts;

[0013] S1.4, remove the nodes in the invalid node container FailHosts from the node list container AllHosts to form a valid node list;

[0014] S2. According to the valid node list, the main process communicates with each background service process, and each background service process creates and starts each sub-service process; the main process sets a keep-alive mechanism for each sub-service process;

[0015] S3, monitor the node status of the sub-service process, when the main process and each sub-service process communicate and connect to each readable event or writable event occurs, the main process uses epoll to perform parallel sorting operations;

[0016] S4. When there is an abnormality in the communication connection between the main process and the sub-service process, the main process removes the node corresponding to the communication connection from the valid node list, cleans up the task information of the sub-service process recording the node, and then selects a node with the least tasks to re-run the sub-service process task;

[0017] S5. The number of times each sub-job has been run is recorded in the main process. When a sub-job fails after being run three times repeatedly, the parallel sorting job is terminated.

[0018] As a limitation, step S1.3 specifically includes:

[0019] S1.3.1. The main process checks whether the current node is in the invalid node container FailHosts according to the node list container AllHosts. If not, execute step S1.3.2. If yes, execute step S1.3.5.

[0020] S1.3.2. Create a child process for the current node;

[0021] S1.3.3. Use ssh in the child process to start the background service process, and set the startup timeout to 120 seconds. If the startup of the background service process times out, execute step S1.3.4, otherwise execute step S1.3.5;

[0022] S1.3.4. The node corresponding to the child process that times out when starting the background service process is considered a failed node and is added to the invalid node container FailHosts.

[0023] S1.3.5. If all nodes have been processed, this step ends; otherwise, process the next node and execute step S1.3.1.

[0024] As a further limitation, step S2 specifically includes:

[0025] S2.1. According to the valid node list, the main process communicates with each background service process, specifies the type of the sub-process, and each background service process creates and starts each sub-service process;

[0026] S2.2, save the communication connection descriptor socket between the main process and each background service process into the communication connection descriptor container Sockets;

[0027] S2.3, record the communication connection descriptor socket and the node information of the sub-service process into the node information container MHosts;

[0028] S2.4. Enable a keep-alive mechanism and set a time for the communication connection between the main process and each background service process in the communication connection descriptor container Sockets.

[0029] As a further limitation, step S3 specifically includes:

[0030] S3.1. Create epoll using the total number of child processes as a parameter;

[0031] S3.2. Add the writable events for the communication connection between the main process and each sub-service process to epoll;

[0032] S3.3, wait for the main process to communicate with each sub-service process. The readable event or the writable event occurs. If the writable event occurs, execute step S3.4. If the readable event occurs, execute step S3.5.

[0033] S3.4, send subtask information for the corresponding main process and sub-service process communication connection. If the sending is successful, add the readable event of the current communication connection to epoll and execute step S3.6. If the sending fails abnormally, execute step S4; if step S4 fails to execute, execute step S3.7, otherwise, execute step S3.6;

[0034] S3.5, receive information from the corresponding main process and the sub-service process communication connection. If the reception is successful, the task executed by the sub-service process is completed, and the communication connection is removed from epoll, and step S3.6 is executed. If the reception fails abnormally, step S4 is executed; if step S4 fails to execute, step S3.7 is executed, otherwise, step S3.6 is executed;

[0035] S3.6, if the communication connection between the main process and each sub-service process is completed, this stage ends, otherwise, execute step S3.3;

[0036] S3.7, Parallel sorting operation fails.

[0037] As a further limitation, step S4 specifically includes:

[0038] S4.1. When there is an abnormality in the communication connection between the main process and the sub-service process, remove the abnormal communication connection from epoll;

[0039] S4.2. According to the background service process corresponding to the sub-service process, query the communication connection descriptor container Sockets, obtain the communication connection descriptor socket corresponding to the abnormal communication connection, and close the abnormal communication connection;

[0040] S4.3, query the node information container MHosts, obtain the node name where the abnormal communication connection is located, remove the node from the valid node list, if the number of nodes in the valid node list after removal is 0, then step S4 ends and returns failure, otherwise execute step S4.4;

[0041] S4.4. The main process cleans up the task information of the sub-service process of the node recorded;

[0042] S4.5, select a node for re-running the sub-service process task corresponding to the abnormal communication connection, first count the number of sub-service processes running on each node in the valid node list, then sort them, and select a node ReplaceNode with the least tasks;

[0043] S4.6. Create a new child process for the node ReplaceNode. Use ssh in the new child process to start the background service process. Start the new background service process on the new child process. Create and start a new child service process through the new background service process. Then create a communication connection between the main process and the new child service process, set a keep-alive mechanism, and set the service type of the child process.

[0044] S4.7, add the writable event connection of the communication connection between the main process and the new sub-service process to epoll. If it is in the rough selection task stage of the parallel sorting operation, execute step S4.8, otherwise, step S4 ends and returns success;

[0045] S4.8. Recreate the parameter files of each sub-job, update the information of each rough selection service node recorded in each sub-job parameter file, and update the data mode output in each sub-job parameter file to the overwrite mode; the sub-job is the job corresponding to the sub-job process started by the main process later.

[0046] As a further limitation, step S5 specifically includes:

[0047] S5.1. The main process evenly distributes the sub-jobs to each node in the valid node list according to the number of sub-jobs.

[0048] S5.2. The main process creates a subprocess for each subjob, and the subjob is started in the subprocess using ssh;

[0049] S5.3, initialize the number of times each sub-job runs to 1, create a sub-job status queue JobsRunning, and add each sub-job to the sub-job status queue JobsRunning;

[0050] S5.4. Start the sub-job process through the main process and monitor the status of each sub-job process; specifically, the following steps are included:

[0051] S5.4.1, loop query the status of the sub-job process corresponding to each sub-job, if the current sub-job process is running, query the sub-job process corresponding to the next sub-job, if the current sub-job process ends normally, execute step S5.4.2, if the current sub-job process exits abnormally, execute step S5.4.3;

[0052] S5.4.2. Remove the sub-job corresponding to the sub-job process from the sub-job running queue JobsRunning, stop monitoring the sub-job, and execute step S5.4.1;

[0053] S5.4.3. Query the number of times the sub-job has been run. If it has been run three times, the parallel sorting job has failed. Otherwise, execute step S5.4.4.

[0054] S5.4.4. Modify the job parameter file corresponding to the sub-job process, and update the data mode output in the job parameter file to the overwrite mode;

[0055] S5.4.5. Remove the node of the sub-job corresponding to the sub-job process from the valid node list. If the number of nodes in the valid node list is 0, the parallel sorting operation fails. Otherwise, execute step S5.4.6.

[0056] S5.4.6. Query a node with the least number of running sub-job processes from the valid node list;

[0057] S5.4.7. The main process creates a sub-process for the sub-job corresponding to the sub-job process. The sub-job is started in the sub-process using ssh mode to monitor the sub-job process.

[0058] S5.4.8. Update the number of times the sub-job runs;

[0059] S5.4.9, check the number of elements in the sub-job running queue JobsRunning. If it is 0, it means that all sub-jobs are completed and step S5 ends. Otherwise, execute step S5.4.10;

[0060] S5.4.10. Set the waiting sleep time to 3 seconds. If the time is up, execute step S5.4.1.

[0061] The present invention also provides a parallel sorting fault node task migration device, comprising:

[0062] The valid node list module is used to initialize valid nodes and form a valid node list;

[0063] The sub-service process keep-alive mechanism module is used to establish a communication connection between the main process and each background service process based on the valid node list, and each background service process creates and starts each sub-service process; the main process sets a keep-alive mechanism for each sub-service process;

[0064] The sub-service process monitoring module is used to monitor the node status of the sub-service process. When a readable event or a writable event occurs between the main process and each sub-service process, the main process uses epoll to perform parallel sorting operations.

[0065] The abnormal node processing module is used to remove the node corresponding to the communication connection from the valid node list through the main process when there is an abnormality in the communication connection between the main process and the sub-service process, and clean up the task information of the sub-service process recording the node, and then select a node with the least tasks to re-run the sub-service process task;

[0066] The sub-job redo module is used to record the number of times each sub-job process has been run in the main process. When a sub-job fails after being run three times repeatedly, the parallel sorting job is terminated.

[0067] The present invention also provides a computer device, which includes a processor and a memory, wherein the memory is used to store at least one computer program, and the at least one computer program is loaded by the processor and executes the above-mentioned parallel fault node sorting task migration method.

[0068] The present invention also provides a storage medium, which is used to store at least one section of a computer program, and the at least one section of the computer program is used to execute the above-mentioned parallel sorting fault node task migration method.

[0069] Due to the adoption of the above technical solution, the present invention has the following technical advances compared with the prior art:

[0070] (1) The present invention uses task migration to restart tasks on abnormal nodes, allowing multi-node parallel sorting operations to continue running, improving the stability of massive seismic data sorting and ensuring the efficiency of field processing and project production parallel sorting;

[0071] (2) The present invention can monitor the node status of the sub-service process through the main process. When there is an abnormality in the communication connection between the main process and the sub-service process, the main process removes the node corresponding to the communication connection from the valid node list, thereby ensuring the validity of the received sub-process information;

[0072] (3) The present invention improves the work efficiency of seismic data processing personnel, effectively reduces the cost of seismic data processing services, and plays a certain supporting role in the development of the company's main business of seismic data acquisition and processing.

[0073] In summary, the present invention is suitable for migrating tasks of parallel sorting faulty nodes, can provide a fault-tolerant mechanism for parallel sorting of seismic data, migrate erroneous subtasks, and provide support for the processing efficiency of massive seismic data. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 The figure is a flow chart of the overall method of the parallel sorting fault node task migration method in Embodiment 1 of the present invention;

[0075] Figure 2 It is a structural block diagram of a device for parallel sorting fault node task migration in Embodiment 2 of the present invention;

[0076] Figure 3 FIG. 2 is a schematic diagram showing the structure of a computer device in Embodiment 2 of the present invention;

[0077] Figure 4The figure is a schematic diagram of the structure of the computer storage medium in the second embodiment of the present invention. DETAILED DESCRIPTION

[0078] In order to better explain the present invention and facilitate understanding, the present invention is described in detail below through specific implementation modes in conjunction with the accompanying drawings.

[0079] Example 1

[0080] like Figure 1 As shown, this embodiment is a parallel sorting fault node task migration method, comprising the following steps:

[0081] S1. Initialize valid nodes and form a valid node list;

[0082] The purpose of this step is to initialize valid nodes and put them into the valid node list when the main process starts the background service process on the child process of each node. When the background service process of each child process is started later, it will only be started on the valid node. This step specifically includes:

[0083] S1.1. Parse the node file provided by the multi-node parallel sorting job in the GeoEast system, remove duplicate nodes, and form a node list container AllHosts;

[0084] S1.2, initialize the invalid node container FailHosts to empty;

[0085] S1.3, the main process creates a child process for each node according to the node list container AllHosts, starts the background service process on each child process, and adds the node that fails to start the background service process to the invalid node container FailHosts;

[0086] This step specifically includes:

[0087] S1.3.1. The main process checks whether the current node is in the invalid node container FailHosts according to the node list container AllHosts. If not, execute step S1.3.2. If yes, execute step S1.3.5.

[0088] S1.3.2. Create a child process for the current node;

[0089] S1.3.3. Use ssh in the child process to start the background service process, and set the startup timeout to 120 seconds. If the startup of the background service process times out, execute step S1.3.4, otherwise execute step S1.3.5;

[0090] S1.3.4. The node corresponding to the child process that times out when starting the background service process is considered a failed node and is added to the invalid node container FailHosts.

[0091] S1.3.5. If all nodes have been processed, this step ends; otherwise, the next node is processed and step S1.3.1 is executed;

[0092] S1.4, remove the nodes in the invalid node container FailHosts from the node list container AllHosts to form a valid node list;

[0093] S2. According to the valid node list, the main process communicates with each background service process, and each background service process creates and starts each sub-service process; the main process sets a keep-alive mechanism for each sub-service process;

[0094] In this step, the main process sets a keep-alive mechanism for each sub-service process. When the sub-service process host is unreachable or there is a network problem, the main process can detect it in time and perform corresponding fault tolerance processing;

[0095] This step specifically includes:

[0096] S2.1. According to the valid node list, the main process communicates with each background service process, specifies the type of the sub-process, and each background service process creates and starts each sub-service process;

[0097] S2.2, save the communication connection descriptor socket between the main process and each background service process into the communication connection descriptor container Sockets;

[0098] S2.3, record the communication connection descriptor socket and the node information of the sub-service process into the node information container MHosts;

[0099] S2.4, enable the keep-alive mechanism and set the time for the communication connection between the main process and each background service process in the communication connection descriptor container Sockets;

[0100] S3, monitor the node status of the sub-service process, when the main process and each sub-service process communicate and connect to each readable event or writable event occurs, the main process uses epoll to perform parallel sorting operations;

[0101] In this step, when a readable event or a writable event occurs between the main process and each sub-service process, the main process uses epoll to perform parallel sorting operations. Since the number of sub-service processes that communicate with the main process at the same time can reach thousands, the Linux system IO multiplexing mechanism epoll can provide an efficient mechanism to monitor the communication connection events between the main process and the sub-service process, and can efficiently handle a large number of concurrent connections;

[0102] This step specifically includes:

[0103] S3.1. Create epoll using the total number of child processes as a parameter;

[0104] S3.2. Add the writable events for the communication connection between the main process and each sub-service process to epoll;

[0105] S3.3, wait for the main process to communicate with each sub-service process. The readable event or the writable event occurs. If the writable event occurs, execute step S3.4. If the readable event occurs, execute step S3.5.

[0106] S3.4, send subtask information for the corresponding main process and sub-service process communication connection. If the sending is successful, add the readable event of the current communication connection to epoll and execute step S3.6. If the sending fails abnormally, execute step S4; if step S4 fails to execute, execute step S3.7, otherwise, execute step S3.6;

[0107] S3.5, receive information from the corresponding main process and the sub-service process communication connection. If the reception is successful, the task executed by the sub-service process is completed, and the communication connection is removed from epoll, and step S3.6 is executed. If the reception fails abnormally, step S4 is executed; if step S4 fails to execute, step S3.7 is executed, otherwise, step S3.6 is executed;

[0108] S3.6, if the communication connection between the main process and each sub-service process is completed, this stage ends, otherwise, execute step S3.3;

[0109] S3.7, Parallel sorting operation fails;

[0110] S4. When there is an abnormality in the communication connection between the main process and the sub-service process, the main process removes the node corresponding to the communication connection from the valid node list, cleans up the task information of the sub-service process recording the node, and then selects a node with the least tasks to re-run the sub-service process task;

[0111] This step specifically includes:

[0112] S4.1. When there is an abnormality in the communication connection between the main process and the sub-service process, remove the abnormal communication connection from epoll;

[0113] S4.2. According to the background service process corresponding to the sub-service process, query the communication connection descriptor container Sockets, obtain the communication connection descriptor socket corresponding to the abnormal communication connection, and close the abnormal communication connection;

[0114] S4.3, query the node information container MHosts, obtain the node name where the abnormal communication connection is located, remove the node from the valid node list, if the number of nodes in the valid node list after removal is 0, then step S4 ends and returns failure, otherwise execute step S4.4;

[0115] S4.4. The main process cleans up the task information of the sub-service process of the node recorded;

[0116] S4.5, select a node for re-running the sub-service process task corresponding to the abnormal communication connection, first count the number of sub-service processes running on each node in the valid node list, then sort them, and select a node ReplaceNode with the least tasks;

[0117] S4.6. Create a new child process for the node ReplaceNode. Use ssh in the new child process to start the background service process. Start the new background service process on the new child process. Create and start a new child service process through the new background service process. Then create a communication connection between the main process and the new child service process, set a keep-alive mechanism, and set the service type of the child process.

[0118] S4.7, add the writable event connection of the communication connection between the main process and the new sub-service process to epoll. If it is in the rough selection task stage of the parallel sorting operation, execute step S4.8, otherwise, step S4 ends and returns success;

[0119] S4.8. Recreate the parameter files of each sub-job, update the information of each rough selection service node recorded in each sub-job parameter file, and update the data mode output in each sub-job parameter file to the overwrite mode; wherein the sub-job is the job corresponding to the sub-job process started by the main process later;

[0120] S5. Record the number of times each sub-job has been run in the main process. When a sub-job fails after being run three times repeatedly, the parallel sorting job is terminated. The sub-job process is a job task running on each node started by the main process.

[0121] This step specifically includes:

[0122] S5.1. The main process evenly distributes the sub-jobs to each node in the valid node list according to the number of sub-jobs.

[0123] S5.2. The main process creates a subprocess for each subjob, and the subjob is started in the subprocess using ssh;

[0124] S5.3, initialize the number of times each sub-job runs to 1, create a sub-job status queue JobsRunning, and add each sub-job to the sub-job status queue JobsRunning;

[0125] S5.4. Start the sub-job process through the main process and monitor the status of each sub-job process; specifically, the following steps are included:

[0126] S5.4.1, loop query the status of the sub-job process corresponding to each sub-job, if the current sub-job process is running, query the sub-job process corresponding to the next sub-job, if the current sub-job process ends normally, execute step S5.4.2, if the current sub-job process exits abnormally, execute step S5.4.3;

[0127] S5.4.2. Remove the sub-job corresponding to the sub-job process from the sub-job running queue JobsRunning, stop monitoring the sub-job, and execute step S5.4.1;

[0128] S5.4.3. Query the number of times the sub-job has been run. If it has been run three times, the parallel sorting job has failed. Otherwise, execute step S5.4.4.

[0129] S5.4.4. Modify the job parameter file corresponding to the sub-job process, and update the data mode output in the job parameter file to the overwrite mode;

[0130] S5.4.5. Remove the node of the sub-job corresponding to the sub-job process from the valid node list. If the number of nodes in the valid node list is 0, the parallel sorting operation fails. Otherwise, execute step S5.4.6.

[0131] S5.4.6. Query a node with the least number of running sub-job processes from the valid node list;

[0132] S5.4.7. The main process creates a sub-process for the sub-job corresponding to the sub-job process. The sub-job is started in the sub-process using ssh mode to monitor the sub-job process.

[0133] S5.4.8. Update the number of times the sub-job runs;

[0134] S5.4.9, check the number of elements in the sub-job running queue JobsRunning. If it is 0, it means that all sub-jobs are completed and step S5 ends. Otherwise, execute step S5.4.10;

[0135] S5.4.10. Set the waiting sleep time to 3 seconds. If the time is up, execute step S5.4.1.

[0136] In order to verify the effectiveness of this embodiment, this embodiment is deployed on a high-performance cluster and experiments are conducted on the cluster for many times. Each node in the cluster is equipped with two Intel(R) Xeon(R) E5-2660 v3 2.6GHz CPUs, each processor contains 10 physical cores, each node has a total of 20 physical cores, 128GB of memory, and shared storage is Sugon storage. The version of the node operating system is Red Hat Enterprise Linux Server release 6.8 (Santiago). The data scale of the test is the actual data of the processing project, with a size of 32TB~512TB, and three sorting keywords are used for sorting, line-cmp-offset. Through the experiment, the operating efficiency of 128 nodes and 256 nodes as shown in Table 1 is obtained. It can be seen from Table 1 that although the task migration mechanism is added, the stability of the parallel sorting operation has been greatly improved, and the parallel sorting operation task ends normally, so this embodiment can improve the work efficiency of seismic data processing personnel.

[0137] Table 1 Operation efficiency of 128 nodes and 256 nodes

[0138]

[0139] Example 2

[0140] This embodiment is a parallel sorting fault node task migration device, such as Figure 2 FIG. 1 is a block diagram of a parallel fault node sorting task migration device provided in accordance with Embodiment 1. The device includes:

[0141] The valid node list module is used to initialize valid nodes and form a valid node list;

[0142] The sub-service process keep-alive mechanism module is used to establish a communication connection between the main process and each background service process based on the valid node list, and each background service process creates and starts each sub-service process; the main process sets a keep-alive mechanism for each sub-service process;

[0143] The sub-service process monitoring module is used to monitor the node status of the sub-service process. When a readable event or a writable event occurs between the main process and each sub-service process, the main process uses epoll to perform parallel sorting operations.

[0144] The abnormal node processing module is used to remove the node corresponding to the communication connection from the valid node list through the main process when there is an abnormality in the communication connection between the main process and the sub-service process, and clean up the task information of the sub-service process recording the node, and then select a node with the least tasks to re-run the sub-service process task;

[0145] The sub-job redo module is used to record the number of times each sub-job process has been run in the main process. When a sub-job fails after being run three times repeatedly, the parallel sorting job is terminated.

[0146] The parallel fault node sorting task migration device provided in this embodiment only uses the division of the above functional modules as an example when performing data processing. In actual applications, the above functions can be assigned to different functional modules as needed.

[0147] Based on the same inventive concept, Figure 3 As shown, this embodiment also provides a computer device, including: at least a processor and a memory, the memory is used to store at least one computer program, and the at least one computer program is loaded by the processor and executes the parallel fault node sorting task migration method of Example 1.

[0148] Based on the same inventive concept, Figure 4 As shown, this embodiment further provides a computer-readable storage medium, the storage medium is used to store at least one computer program, and the at least one computer program is used to execute the parallel fault node sorting task migration method of embodiment 1.

[0149] Finally, it should be noted that a person skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods.

[0150] Furthermore, it should be appreciated that the computer-readable storage medium (eg, memory) herein may be either volatile memory or nonvolatile memory, or may include both volatile memory and nonvolatile memory.

[0151] It will also be appreciated by those skilled in the art that various exemplary logic blocks, modules, circuits and algorithm steps described in conjunction with the disclosure herein can be implemented as electronic hardware, computer software or a combination of the two. In order to clearly illustrate this interchangeability of hardware and software, a general description has been given to the functions of various schematic components, blocks, modules, circuits and steps. Whether this function is implemented as software or hardware depends on specific applications and the design constraints imposed on the entire system. Those skilled in the art can implement the function in various ways for each specific application, but this implementation decision should not be interpreted as causing a departure from the disclosed scope of the embodiments of the present invention.

[0152] It should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention is described in detail with reference to the above embodiments, those skilled in the art can still modify the technical solutions described in the above embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A parallel sorting fault node task migration method, characterized in that: The following steps are involved: S1. Initialize valid nodes to form a valid node list. This step specifically includes: S1.

1. Parse the node file provided by the multi-node parallel sorting job in the GeoEast system, remove duplicate nodes, and form a node list container AllHosts; S1.2, initialize the invalid node container FailHosts to empty; S1.3, the main process creates a child process for each node according to the node list container AllHosts, starts the background service process on each child process, and adds the node that fails to start the background service process to the invalid node container FailHosts; S1.4, remove the nodes in the invalid node container FailHosts from the node list container AllHosts to form a valid node list; S2. According to the valid node list, the main process communicates with each background service process, and each background service process creates and starts each sub-service process; the main process sets a keep-alive mechanism for each sub-service process; Step S2 specifically includes: S2.

1. According to the valid node list, the main process communicates with each background service process, specifies the type of the sub-process, and each background service process creates and starts each sub-service process; S2.2, save the communication connection descriptor socket between the main process and each background service process into the communication connection descriptor container Sockets; S2.3, record the communication connection descriptor socket and the node information of the sub-service process into the node information container MHosts; S2.4, enable the keep-alive mechanism and set the time for the communication connection between the main process and each background service process in the communication connection descriptor container Sockets; S3, monitor the node status of the sub-service process, when the main process and each sub-service process communicate and connect to each readable event or writable event occurs, the main process uses epoll to perform parallel sorting operations; S4. When there is an abnormality in the communication connection between the main process and the sub-service process, the main process removes the node corresponding to the communication connection from the valid node list, cleans up the task information of the sub-service process recording the node, and then selects a node with the least tasks to re-run the sub-service process task; Step S4 specifically includes: S4.

1. When there is an abnormality in the communication connection between the main process and the sub-service process, remove the abnormal communication connection from epoll; S4.

2. According to the background service process corresponding to the sub-service process, query the communication connection descriptor container Sockets, obtain the communication connection descriptor socket corresponding to the abnormal communication connection, and close the abnormal communication connection; S4.3, query the node information container MHosts, obtain the node name where the abnormal communication connection is located, remove the node from the valid node list, if the number of nodes in the valid node list after removal is 0, then step S4 ends and returns failure, otherwise execute step S4.4; S4.

4. The main process cleans up the task information of the sub-service process of the node recorded; S4.5, select a node for re-running the sub-service process task corresponding to the abnormal communication connection, first count the number of sub-service processes running on each node in the valid node list, then sort them, and select a node ReplaceNode with the least tasks; S4.

6. Create a new child process for the node ReplaceNode. Use ssh in the new child process to start the background service process. Start the new background service process on the new child process. Create and start a new child service process through the new background service process. Then create a communication connection between the main process and the new child service process, set a keep-alive mechanism, and set the service type of the child process. S4.7, add the writable event connection of the communication connection between the main process and the new sub-service process to epoll. If it is in the rough selection task stage of the parallel sorting operation, execute step S4.8, otherwise, step S4 ends and returns success; S4.

8. Recreate the parameter files of each sub-job, update the information of each rough selection service node recorded in each sub-job parameter file, and update the data mode output in each sub-job parameter file to the overwrite mode; the sub-job is the job corresponding to the sub-job process started by the main process later; S5. The number of times each sub-job has been run is recorded in the main process. When a sub-job fails after being run three times repeatedly, the parallel sorting job is terminated.

2. The parallel sorting fault node task migration method according to claim 1 is characterized in that: Step S1.3 specifically includes: S1.3.1, the main process checks whether the current node is in the invalid node container FailHosts according to the node list container AllHosts. If not, execute step S1.3.

2. If yes, execute step S1.3.

5. S1.3.

2. Create a child process for the current node; S1.3.

3. Use ssh in the child process to start the background service process, and set the startup timeout to 120 seconds. If the startup of the background service process times out, execute step S1.3.4, otherwise execute step S1.3.5; S1.3.

4. The node corresponding to the child process that times out when starting the background service process is considered a failed node and is added to the invalid node container FailHosts. S1.3.

5. If all nodes have been processed, this step ends; otherwise, process the next node and execute step S1.3.

1.

3. The parallel sorting fault node task migration method according to claim 1 is characterized in that: Step S3 specifically includes: S3.

1. Create epoll using the total number of child processes as a parameter; S3.

2. Add the writable events for the communication connection between the main process and each sub-service process to epoll; S3.3, wait for the main process to communicate with each sub-service process. The readable event or the writable event occurs. If the writable event occurs, execute step S3.

4. If the readable event occurs, execute step S3.

5. S3.4, send subtask information for the corresponding main process and sub-service process communication connection. If the sending is successful, add the readable event of the current communication connection to epoll and execute step S3.

6. If the sending fails abnormally, execute step S4; if step S4 fails to execute, execute step S3.7, otherwise, execute step S3.6; S3.5, receive information from the corresponding main process and the sub-service process communication connection. If the reception is successful, the task executed by the sub-service process is completed, and the communication connection is removed from epoll, and step S3.6 is executed. If the reception fails abnormally, step S4 is executed; if step S4 fails to execute, step S3.7 is executed, otherwise, step S3.6 is executed; S3.6, if the communication connection between the main process and each sub-service process is completed, this stage ends, otherwise, execute step S3.3; S3.7, Parallel sorting operation fails.

4. The parallel sorting fault node task migration method according to claim 1 is characterized in that: Step S5 specifically includes: S5.

1. The main process evenly distributes the sub-jobs to each node in the valid node list according to the number of sub-jobs. S5.

2. The main process creates a subprocess for each subjob, and the subjob is started in the subprocess using ssh; S5.3, initialize the number of times each sub-job runs to 1, create a sub-job status queue JobsRunning, and add each sub-job to the sub-job status queue JobsRunning; S5.

4. Start the sub-job process through the main process and monitor the status of each sub-job process; specifically, the following steps are included: S5.4.1, loop query the status of the sub-job process corresponding to each sub-job, if the current sub-job process is running, query the sub-job process corresponding to the next sub-job, if the current sub-job process ends normally, execute step S5.4.2, if the current sub-job process exits abnormally, execute step S5.4.3; S5.4.

2. Remove the sub-job corresponding to the sub-job process from the sub-job running queue JobsRunning, stop monitoring the sub-job, and execute step S5.4.1; S5.4.

3. Query the number of times the sub-job has been run. If it has been run three times, the parallel sorting job has failed. Otherwise, execute step S5.4.

4. S5.4.

4. Modify the job parameter file corresponding to the sub-job process, and update the data mode output in the job parameter file to the overwrite mode; S5.4.

5. Remove the node of the sub-job corresponding to the sub-job process from the valid node list. If the number of nodes in the valid node list is 0, the parallel sorting operation fails. Otherwise, execute step S5.4.

6. S5.4.

6. Query a node with the least number of running sub-job processes from the valid node list; S5.4.

7. The main process creates a sub-process for the sub-job corresponding to the sub-job process. The sub-job is started in the sub-process using ssh mode to monitor the sub-job process. S5.4.

8. Update the number of times the sub-job runs; S5.4.9, check the number of elements in the sub-job running queue JobsRunning. If it is 0, it means that all sub-jobs are completed and step S5 ends. Otherwise, execute step S5.4.10; S5.4.

10. Set the waiting sleep time to 3 seconds. If the time is up, execute step S5.4.

1.

5. A parallel sorting fault node task migration device, characterized in that: The device is used to implement the parallel sorting fault node task migration method described in any one of claims 1 to 4, comprising: The valid node list module is used to initialize valid nodes and form a valid node list; The sub-service process keep-alive mechanism module is used to establish a communication connection between the main process and each background service process based on the valid node list, and each background service process creates and starts each sub-service process; the main process sets a keep-alive mechanism for each sub-service process; The sub-service process monitoring module is used to monitor the node status of the sub-service process. When a readable event or a writable event occurs between the main process and each sub-service process, the main process uses epoll to perform parallel sorting operations. The abnormal node processing module is used to remove the node corresponding to the communication connection from the valid node list through the main process when there is an abnormality in the communication connection between the main process and the sub-service process, and clean up the task information of the sub-service process recording the node, and then select a node with the least tasks to re-run the sub-service process task; The sub-job redo module is used to record the number of times each sub-job process has been run in the main process. When a sub-job fails after being run three times repeatedly, the parallel sorting job is terminated.

6. A computer device, characterized in that: The computer device includes a processor and a memory, the memory is used to store at least one computer program, and the at least one computer program is loaded by the processor and executes the parallel fault node sorting task migration method described in any one of claims 1 to 4.

7. A storage medium, characterized in that: The storage medium is used to store at least one computer program, and the at least one computer program is used to execute the parallel fault node sorting task migration method described in any one of claims 1 to 4.

Images