Task processing method and device

Abstract

The present invention provides a task processing method and device, the method comprising: a collaborative management node in a self-organizing network group stores the execution process and execution result of the first computing node in the self-organizing network group in the process of executing a target task in a storage node in the self-organizing network group; if a crash occurs in the process of the first computing node executing the target task, a second computing node is reselected from the self-organizing network group; the execution process and execution result of the most recent execution of the target task by the first computing node in the storage node are sent to the second computing node, so that the second computing node can continue to execute the target task according to the execution result. The present invention realizes real-time synchronization of the execution process and execution result of the first computing node when it crashes to the second computing node, so that the second computing node can continue to execute the target task from the execution process of the breakpoint-resume task, effectively improving the efficiency of task execution.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a task processing method and device. Background Art

[0002] The distributed computing framework is mainly used to decompose a complex computing task into multiple smaller parts and assign them to different computing units for processing. Compared with traditional single-machine computing, distributed computing saves computing time and greatly improves computing efficiency.

[0003] The existing distributed computing framework is mainly based on the MapReduce concept, and commonly used frameworks include hadoop and spark. The Hadoop computing framework is a powerful open source distributed computing framework under the MapReduce development model. It is mainly composed of the computing framework MapReduce and the storage framework HDFS (Hadoop Distribution FileSystem, a distributed file system on general hardware) used to store computing data. It uses a master-slave mode to process computing tasks. The master node is responsible for the division and scheduling of specific tasks and operation monitoring, and the slave node is responsible for the execution of specific tasks, and performs computing processes such as data acquisition, output and execution results of its own tasks.

[0004] When executing computing tasks, the existing distributed computing architecture usually uses multiple computing nodes that serve as the primary and backup nodes to execute computing tasks. When the original primary computing node crashes while executing a task, the backup computing node needs to be switched to the new primary computing node. At this time, the initiating node will resend the task to the new primary computing node. The new primary computing node needs to execute the task from the beginning, which reduces the efficiency of task processing. Summary of the invention

[0005] The present invention provides a task processing method and device to solve the defect in the prior art that after the main computing node crashes, a new main computing node needs to execute the task from the beginning, which reduces the efficiency of task processing, and improves the task processing efficiency.

[0006] The present invention provides a task processing method, comprising:

[0007] The collaborative management node in the self-organizing network group stores the execution process and execution result of the first computing node in the self-organizing network group in the process of executing the target task in the storage node in the self-organizing network group;

[0008] If a downtime occurs during the execution of the target task by the first computing node, reselecting a second computing node from the self-organizing network group;

[0009] The execution process and execution result of the target task most recently executed by the first computing node in the storage node are sent to the second computing node, so that the second computing node can continue to execute the target task according to the execution result and the execution process.

[0010] According to a task processing method provided by the present invention, the reselecting a second computing node from the self-organizing network group includes:

[0011] Obtaining the computing efficiency of each node according to the remaining number of CPU cores, remaining memory space, communication bandwidth and disk space of each node in the self-organizing network cluster;

[0012] Selecting other nodes with the highest computing efficiency except the first computing node from the self-organizing network group as the second computing node.

[0013] According to a task processing method provided by the present invention, the reselecting a second computing node from the self-organizing network group includes:

[0014] Receiving a request for reselecting a computing node sent by an initiating node in the self-organizing network group, and sending a confirmation message to the first computing node according to the request for reselecting a computing node;

[0015] If no feedback message of the confirmation message sent by the first computing node is received, reselecting the second computing node from the self-organizing network group;

[0016] Wherein, after sending the detection message to the first computing node, the initiating node sends the request to reselect a computing node if no feedback message of the detection message sent by the first computing node is received.

[0017] According to a task processing method provided by the present invention, after sending the execution process and execution result of the target task most recently executed by the first computing node in the storage node to the second computing node, the method comprises:

[0018] If the storage node is down, reselect a new storage node from the self-organizing network group;

[0019] The execution process and execution result of the second computing node are stored in the new storage node.

[0020] According to a task processing method provided by the present invention, when the collaborative management node is down, the collaborative management node is updated to another node with the smallest ID in the self-organizing network group except the collaborative management node.

[0021] According to a task processing method provided by the present invention, before sending the execution process and execution result of the target task most recently executed by the first computing node in the storage node to the second computing node, the method further includes:

[0022] The IP address of the second computing node is sent to the initiating node in the self-organizing network group, so that the initiating node can send the target task to the second computing node to which the IP address belongs.

[0023] According to a task processing method provided by the present invention, the self-organizing network grouping is obtained by grouping the self-organizing network by a network grouping algorithm;

[0024] The network clustering algorithm is a minimum ID clustering algorithm or a maximum connectivity clustering algorithm.

[0025] The present invention also provides a task processing device, comprising:

[0026] A storage module, used for the collaborative management node in the self-organizing network group to store the execution process and execution result of the first computing node in the self-organizing network group in the process of executing the target task in the storage node in the self-organizing network group;

[0027] A selection module, configured to reselect a second computing node from the self-organizing network group if a downtime occurs during the execution of the target task by the first computing node;

[0028] The sending module is used to send the execution process and execution result of the first computing node in the storage node's most recent execution of the target task to the second computing node, so that the second computing node can continue to execute the target task according to the execution result and the execution process.

[0029] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of any of the above-mentioned task processing methods when executing the program.

[0030] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the task processing methods described above are implemented.

[0031] The task processing method and device provided by the present invention can quickly locate the execution process of the breakpoint-resume task of the first computing node when it is down, based on the execution process of the first computing node's most recent execution of the target task, and synchronize the execution process and execution result of the first computing node when it is down to the second computing node in real time, so that the second computing node can continue to execute the target task from the execution process of the breakpoint-resume task, effectively improving the efficiency of task execution. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0033] Figure 1 It is a flowchart of the task processing method provided by the present invention;

[0034] Figure 2 It is one of the structural schematic diagrams of the self-organizing network grouping in the task processing method provided by the present invention;

[0035] Figure 3 This is the second structural diagram of the self-organizing network grouping in the task processing method provided by the present invention;

[0036] Figure 4 This is the third structural diagram of the self-organizing network grouping in the task processing method provided by the present invention;

[0037] Figure 5 is a structural schematic diagram of a task processing device provided by the present invention;

[0038] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] Combine the following Figure 1The task processing method of the present invention is described, comprising: step 101, a collaborative management node in a self-organizing network group stores the execution process and execution result of a first computing node in the self-organizing network group in the process of executing a target task in a storage node in the self-organizing network group;

[0041] Optionally, the target task is a task that requires calculation.

[0042] A self-organizing network is a network that combines mobile communications and computer networks. The information exchange of the network adopts the packet switching mechanism in computer networks. The user terminals are portable terminals that can be moved. Each user terminal in the self-organizing network has both the functions of a router and a host.

[0043] When the distributed computing framework in the self-organizing network communication mode conducts self-organizing networking on nodes, considering the characteristics of unstable communication and connection of nodes in the self-organizing network within a limited range, the neighboring nodes are divided into a self-organizing network cluster, so that the computing tasks are distributed and the computing status is synchronized within the cluster area, weakening the impact of network communication on computing.

[0044] Optionally, a network grouping algorithm is used to divide the self-organizing network into a plurality of self-organizing network groups. This embodiment is not limited to the number of self-organizing network groups.

[0045] like Figure 2 As shown, the self-organizing network includes three areas, each area is a self-organizing network cluster, and each self-organizing network cluster includes a cluster head node (black node) and other non-cluster head nodes.

[0046] The nodes in each self-organizing network cluster are divided into three categories: collaborative management nodes, storage nodes and computing nodes. The collaborative management nodes are generally the cluster head nodes, and the non-cluster head nodes can be used as computing nodes or storage nodes.

[0047] The collaborative management node is the cluster leader node in the region, which is mainly responsible for node management and computing process scheduling. Storage nodes and computing nodes are the remaining nodes except the cluster leader node. Storage nodes are mainly used to store the status of computing processes on other nodes and the data generated during the computing process; computing nodes are mainly used to execute tasks.

[0048] like Figure 3 and Figure 4 As shown, the steps for task processing in each organization network cluster include:

[0049] Step 1: The initiating node in the self-organizing network cluster sends a computing task execution request to the collaborative management node; the node management module in the collaborative management node selects the first computing node and the storage node according to the computing task execution request, and feeds back the selection result to the initiating node. The selection result includes the IP (Internet Protocol) address of the first computing node and the storage node.

[0050] Step 2: The task distribution module in the initiating node sends the target task to the first computing node according to the information fed back by the collaborative management node;

[0051] Step 3: The real-time computing module in the first computing node executes the target task, and at the same time, the state synchronization module in the first computing node synchronizes the execution process and execution results of the first computing node in the process of executing the target task to the storage node, so that the execution process and execution results of the first computing node can be viewed at any time.

[0052] Step 4: After the first computing node completes the target task, it returns the computing result to the initiating node, so that the initiating node can process the computing result and then feed it back to the terminal.

[0053] The number of first computing nodes and storage nodes may be reasonably selected according to the configuration information of the target task.

[0054] It should be noted that, when there are multiple first computing nodes, each first computing node can execute tasks in parallel.

[0055] Step 102, if the first computing node crashes during the process of executing the target task, reselecting a second computing node from the self-organizing network group;

[0056] The first computing node is monitored in real time during the process of the first computing node executing the target task, and whether the first computing node is down is determined according to the monitoring result.

[0057] Among them, the status of the first computing node can be monitored in real time using the collaborative management node, or the status of the first computing node can be monitored by other nodes except the collaborative management node, and the monitoring results can be sent to the collaborative management node in real time. This embodiment does not make specific limitations on this.

[0058] If the first computing node crashes during the execution of the target task, the target task cannot be continued. It is necessary to reselect a second computing node from the self-organizing network group to replace the first computing node and continue to execute the target task through the second computing node.

[0059] If the first computing node does not crash during the execution of the target task, the first computing node continues to execute the target task until the target task is completed and returns the execution result to the initiating node, so that the initiating node can process the calculation result and then feed it back to the terminal.

[0060] This embodiment monitors the status of the first computing node in real time to promptly switch to the second computing node when the first computing node fails.

[0061] Step 103, sending the execution process and execution result of the most recent execution of the target task by the first computing node in the storage node to the second computing node, so that the second computing node can continue to execute the target task according to the execution result and the execution process.

[0062] Optionally, the execution process and execution result of the target task most recently executed by the first computing node are the execution process and execution result when the first computing node is down.

[0063] By querying the execution process and execution results stored by the first computing node during the execution of the target task in the storage node, the process of executing the target task and the execution result of the first computing node when it is down can be obtained and synchronized to the second computing node.

[0064] The second computing node can restore the computing task of the first computing node when it is down according to the execution process and the execution result, and then the execution process of the first computing node continues to execute the target task.

[0065] When the first computing node crashes, the execution process of the breakpoint-resume task can be accurately located through the execution process of the first computing node stored in the storage node, and the target task can be switched to the second computing node to continue executing the target task, so that the execution process of the breakpoint-resume task of the first computing node and the second node is synchronized, thereby effectively avoiding the second computing node from executing the target task from the beginning, resulting in a large number of duplicate execution processes of the second computing node and the second computing node, and low execution efficiency.

[0066] For example, the execution process of the first computing node is 80% when it is down. In this embodiment, the second computing node synchronizes the execution process of the first computing node, and the second computing node needs to complete the remaining 20% ​​of the computing tasks. In the prior art, when the backup computing node switches to the main computing node, the task needs to be re-executed, that is, 100% of the computing tasks need to be completed.

[0067] This embodiment can quickly locate the execution process of the breakpoint-resume task of the first computing node when it is down based on the execution process of the most recent execution of the target task by the first computing node, and synchronize the execution process and execution results of the first computing node when it is down to the second computing node in real time, so that the second computing node can continue to execute the target task from the execution process of the breakpoint-resume task, effectively improving the efficiency of task execution.

[0068] Based on the above embodiments, the reselection of the second computing node from the self-organizing network grouping in this implementation includes: obtaining the computing efficiency of each node based on the remaining CPU (Central Processing Unit) cores, remaining memory space, communication bandwidth and disk space of each node in the self-organizing network grouping; and selecting the other node with the highest computing efficiency except the first computing node from the self-organizing network grouping as the second computing node.

[0069] Optionally, the collaborative management node uses the node management module to collect real-time statistics on indicators such as the remaining number of CPU cores, remaining memory space, communication bandwidth and disk space, and network connection status of each node in the self-organizing network cluster, and uses the above indicators to specify which nodes are responsible for computing tasks and which nodes are responsible for storage tasks.

[0070] When selecting a computing node, you can choose based on the node's computing efficiency; when selecting a storage node, you can choose based on the node's storage capacity.

[0071] Optionally, the calculation efficiency of each node is calculated as:

[0072] S = w1*a+w2*b+w3*c+w4*d;

[0073] Wherein, S represents the computing efficiency, w1, w2, w3 and w4 are weight coefficients, wherein w1+w2+w3+w4=1; a, b, c and d represent the remaining number of CPU cores, remaining memory space, communication bandwidth and disk space respectively. Wherein, the value of w1 is greater than w2, w3 and w4, and the specific values ​​of w1, w2, w3 and w4 can be set according to actual needs, and this embodiment does not specifically limit this.

[0074] After the computing efficiency of each node is obtained, other nodes with the highest computing efficiency except the first computing node are selected from the self-organizing network cluster according to the computing efficiency as the second computing node.

[0075] This embodiment comprehensively considers various indicators of each node and selects the node with the highest computing efficiency as the computing node, which effectively improves the task processing efficiency. In addition, the communication bandwidth is considered to effectively avoid the problem that the computing node is affected by the communication bandwidth and the computing performance is reduced.

[0076] On the basis of the above-mentioned embodiments, the reselection of the second computing node from the self-organizing network group described in this implementation includes: receiving a computing node reselection request sent by an initiating node in the self-organizing network group, and sending a confirmation message to the first computing node according to the computing node reselection request; if a feedback message of the confirmation message sent by the first computing node is not received, reselecting the second computing node from the self-organizing network group; wherein, after the initiating node sends a detection message to the first computing node, if it does not receive a feedback message of the detection message sent by the first computing node, it sends the computing node reselection request.

[0077] Optionally, during the process of task processing by the first computing node, the initiating node monitors the first computing node in real time through the node monitoring module, and when it detects that the connection status of the first computing node is abnormal, it sends a request to reselect the computing node to the collaborative management node, so that the collaborative management node can select the second computing node in time when the first computing node fails.

[0078] Optionally, the monitoring method is to periodically send a detection message to the first computing node, and determine whether the connection between the first computing node and the initiating node is valid according to whether a feedback message from the first computing node is received.

[0079] The detection message may be a heartbeat packet, which is a self-defined command word that regularly notifies the client and the server of their status and is sent at a certain time interval.

[0080] Similarly, the initiating node may also monitor other nodes in the self-organizing network cluster that are connected to the initiating node, such as storage nodes, according to the above monitoring method.

[0081] The following takes the first computing node as an example to describe the steps of monitoring whether the first computing node is abnormal:

[0082] Step 1: The initiating node periodically sends a detection message to the first computing node. When the initiating node receives a feedback message of the detection message sent by the first computing node, it indicates that the connection between the first computing node and the initiating node is valid; when the initiating node does not receive a feedback message of the detection message sent by the first computing node, it indicates that the connection between the first computing node and the initiating node is invalid. At this time, the initiating node sends a request to reallocate the computing node to the collaborative management node;

[0083] Step 2: The collaborative management node receives the computing node reallocation request sent by the initiating node, and sends a confirmation message to the first computing node according to the computing node reallocation request; wherein the confirmation message may be a predefined datagram, which is not specifically limited in this embodiment;

[0084] Step 3: The collaborative management node confirms whether it has received a feedback message of the confirmation message sent by the first computing node; if not, it indicates that the first computing node is down, and a second computing node is reselected from the self-organizing network group; if received, it indicates that the first computing node is not really down, but the connection between the first computing node and the initiating node is disconnected. The collaborative management node sends a reconnection instruction to the initiating node so that the initiating node can reconnect with the first computing node.

[0085] This embodiment sets up a complete disaster recovery mechanism in the self-organizing network, monitors the status of the first computing node in real time to accurately determine whether the first computing node is down. When the first computing node is down, the second computing node is selected in time and the tasks of the first computing node are synchronized to the second computing node.

[0086] Based on the above embodiments, in this implementation, after sending the execution process and execution result of the most recent execution of the target task by the first computing node in the storage node to the second computing node, it includes: if the storage node crashes, reselecting a new storage node from the self-organizing network group; storing the execution process and execution result of the second computing node in the new storage node.

[0087] Optionally, during the task processing by the second computing node, the storage node may be monitored by using the method of monitoring whether the first computing node is abnormal.

[0088] When the collaborative management module detects that a storage node is down, it can also reselect a new storage node from the self-organizing network cluster based on indicators such as the remaining number of CPU cores, remaining memory space, communication bandwidth and disk space, and network connection status.

[0089] When selecting a storage node, the remaining number of CPU cores, remaining memory space, communication bandwidth, and disk space may also be weighted and added, and the weights corresponding to the remaining memory space and disk space are set to be greater than the weights corresponding to the remaining number of CPU cores and communication bandwidth. The node with the largest weighted addition result is selected as the new storage node.

[0090] The collaborative management node synchronizes the execution process and execution result of the second computing node to the new storage node. At the same time, the initiating node continues to monitor the new storage node according to the above monitoring method.

[0091] When other computing nodes are processing tasks, the above method can also be used to monitor the storage node and switch to a new storage node when the storage node fails.

[0092] This embodiment monitors the status of the storage node in real time to promptly switch to a new storage node when a storage node fails, thereby ensuring that the execution process and execution results of each computing node can be effectively and completely stored.

[0093] On the basis of the above embodiments, in this implementation, when the collaborative management node goes down, the collaborative management node is updated to other nodes with the smallest ID (Identity Document) in the self-organizing network group except the collaborative management node.

[0094] Optionally, each node in the self-organizing network cluster has a unique ID. When the collaborative management node goes down, other nodes with the smallest ID other than the collaborative management node can be selected as new collaborative management nodes according to the ID of each node in the self-organizing network cluster to be responsible for node management.

[0095] In this embodiment, a complete disaster recovery mechanism is set up in the self-organizing network grouping. Even if the network topology changes due to changes in the collaborative management nodes, the connection between the computing nodes and / or storage nodes and the initiating nodes is disconnected, or a downtime occurs, the disaster recovery mechanism can be used to ensure that the computing tasks continue to be executed, thereby achieving high computing reliability and adapting well to complex situations such as structural changes, limited bandwidth, and unstable connections that self-organizing network communications often face.

[0096] Based on the above embodiments, the present embodiment further includes, before sending the execution process and execution result of the most recent execution of the target task by the first computing node in the storage node to the second computing node, sending the IP address of the second computing node to the initiating node in the self-organizing network cluster, so that the initiating node can send the target task to the second computing node to which the IP address belongs.

[0097] Optionally, each node in the ad hoc network cluster has a unique IP address.

[0098] After the collaborative management node reselects the second computing node, the IP address of the second computing node is sent to the initiating node.

[0099] After receiving the IP address, the initiating node establishes a connection with the second computing node to which the IP address belongs, so as to send the target task to the second computing node and monitor the second computing node.

[0100] Similarly, after the collaborative management node reselects a new storage node, the IP address of the new storage node may also be sent to the initiating node so that the initiating node can establish a connection with the storage node to which the IP address belongs.

[0101] On the basis of the above embodiments, the self-organizing network grouping in this embodiment is obtained by grouping the self-organizing network using a network grouping algorithm; wherein the network grouping algorithm is a minimum ID grouping algorithm or a maximum connectivity grouping algorithm.

[0102] Optionally, the network grouping algorithm may be a minimum ID grouping algorithm or a maximum connectivity grouping algorithm, etc., which is not specifically limited in this embodiment.

[0103] Among them, the minimum ID network clustering algorithm clusters the nodes in the self-organizing network. The minimum ID network clustering algorithm refers to the self-organizing network networking that assigns a different ID to each node, each node broadcasts its own ID, and each node determines the clustering state based on the ID numbers of all the nodes it hears. The minimum ID clustering algorithm refers to the node exchanging the node IDs heard with the adjacent nodes, and selecting the node with the smallest ID as its cluster leader. This clustering algorithm has a small amount of calculation, is easy to implement, and has a fast algorithm convergence speed.

[0104] This embodiment fully considers the characteristics of self-organizing network communication, such as narrow bandwidth and weak connection. The self-organizing network is grouped, and computing tasks are performed in the grouped area. At the same time, when selecting computing nodes and storage nodes, they are selected in the area based on the network communication quality and node status, effectively reducing the impact of network communication on computing.

[0105] The task processing device provided by the present invention is described below. The task processing device described below and the task processing method described above can be referenced to each other.

[0106] like Figure 5 As shown, this embodiment provides a task processing device, which includes a storage module 501, a selection module 502 and a sending module 503, wherein:

[0107] Storage module 501, used for the collaborative management node in the self-organizing network group to store the execution process and execution result of the first computing node in the self-organizing network group in the process of executing the target task in the storage node in the self-organizing network group;

[0108] Optionally, the self-organizing network is a network that combines mobile communications and computer networks. The information exchange of the network adopts the packet switching mechanism in the computer network. The user terminal is a portable terminal that can be moved. Each user terminal in the self-organizing network has both the functions of a router and a host.

[0109] When the distributed computing framework in the self-organizing network communication mode conducts self-organizing networking on nodes, considering the characteristics of unstable communication and connection of nodes in the self-organizing network within a limited range, the neighboring nodes are divided into a self-organizing network cluster, so that the computing tasks are distributed and the computing status is synchronized within the cluster area, weakening the impact of network communication on computing.

[0110] Optionally, a network grouping algorithm is used to divide the self-organizing network into a plurality of self-organizing network groups. This embodiment is not limited to the number of self-organizing network groups.

[0111] like Figure 2 As shown, the self-organizing network includes three areas, each area is a self-organizing network cluster, and each self-organizing network cluster includes a cluster head node (black node) and other non-cluster head nodes.

[0112] The nodes in each self-organizing network cluster are divided into three categories: collaborative management nodes, storage nodes and computing nodes. The collaborative management nodes are generally the cluster head nodes, and the non-cluster head nodes can be used as computing nodes or storage nodes.

[0113] The collaborative management node is the cluster leader node in the region, which is mainly responsible for node management and computing process scheduling. Storage nodes and computing nodes are the remaining nodes except the cluster leader node. Storage nodes are mainly used to store the status of computing processes on other nodes and the data generated during the computing process; computing nodes are mainly used to execute tasks.

[0114] like Figure 3 and Figure 4 As shown, the steps for task processing in each organization network cluster include:

[0115] Step 1: The initiating node in the self-organizing network cluster sends a computing task execution request to the collaborative management node; the node management module in the collaborative management node selects the first computing node and the storage node according to the computing task execution request, and feeds back the selection result to the initiating node. The selection result includes the IP addresses of the first computing node and the storage node.

[0116] Step 2: The task distribution module in the initiating node sends the target task to the first computing node according to the information fed back by the collaborative management node;

[0117] Step 3: The real-time computing module in the first computing node executes the target task, and at the same time, the state synchronization module in the first computing node synchronizes the execution process and execution results of the first computing node in the process of executing the target task to the storage node, so that the execution process and execution results of the first computing node can be viewed at any time.

[0118] Step 4: After the first computing node completes the target task, it returns the computing result to the initiating node, so that the initiating node can process the computing result and then feed it back to the terminal.

[0119] The number of first computing nodes and storage nodes may be reasonably selected according to the configuration information of the target task.

[0120] It should be noted that, when there are multiple first computing nodes, each first computing node can execute tasks in parallel.

[0121] The selection module 502 is used to reselect a second computing node from the self-organizing network group if a crash occurs during the execution of the target task by the first computing node;

[0122] The first computing node is monitored in real time during the process of the first computing node executing the target task, and whether the first computing node is down is determined according to the monitoring result.

[0123] Among them, the status of the first computing node can be monitored in real time using the collaborative management node, or the status of the first computing node can be monitored by other nodes except the collaborative management node, and the monitoring results can be sent to the collaborative management node in real time. This embodiment does not make specific limitations on this.

[0124] If the first computing node crashes during the execution of the target task, the target task cannot be continued. It is necessary to reselect a second computing node from the self-organizing network group to replace the first computing node and continue to execute the target task through the second computing node.

[0125] If the first computing node does not crash during the execution of the target task, the first computing node continues to execute the target task until the target task is completed and returns the execution result to the initiating node, so that the initiating node can process the calculation result and then feed it back to the terminal.

[0126] This embodiment monitors the status of the first computing node in real time to promptly switch to the second computing node when the first computing node fails.

[0127] The sending module 503 is used to send the execution process and execution result of the first computing node in the storage node's most recent execution of the target task to the second computing node, so that the second computing node can continue to execute the target task according to the execution result and the execution process.

[0128] Optionally, the execution process and execution result of the target task most recently executed by the first computing node are the execution process and execution result when the first computing node is down.

[0129] By querying the execution process and execution results stored by the first computing node during the execution of the target task in the storage node, the process of executing the target task and the execution result of the first computing node when it is down can be obtained and synchronized to the second computing node.

[0130] The second computing node can restore the computing task of the first computing node when it is down according to the execution process and the execution result, and then the execution process of the first computing node continues to execute the target task.

[0131] When the first computing node crashes, the execution process of the breakpoint-resume task can be accurately located through the execution process of the first computing node stored in the storage node, and the target task can be switched to the second computing node to continue executing the target task, so that the execution process of the breakpoint-resume task of the first computing node and the second node is synchronized, thereby effectively avoiding the second computing node from executing the target task from the beginning, resulting in a large number of duplicate execution processes of the second computing node and the second computing node, and low execution efficiency.

[0132] This embodiment can quickly locate the execution process of the breakpoint-resume task of the first computing node when it is down based on the execution process of the most recent execution of the target task by the first computing node, and synchronize the execution process and execution results of the first computing node when it is down to the second computing node in real time, so that the second computing node can continue to execute the target task from the execution process of the breakpoint-resume task, effectively improving the efficiency of task execution.

[0133] Based on the above embodiments, the selection module in this implementation is specifically used to: obtain the computing efficiency of each node according to the remaining CPU cores, remaining memory space, communication bandwidth and disk space of each node in the self-organizing network group; select the other nodes with the highest computing efficiency except the first computing node from the self-organizing network group as the second computing node.

[0134] On the basis of the above-mentioned embodiment, the selection module in this implementation is also used for: receiving a request for reselecting a computing node sent by an initiating node in the self-organizing network group, and sending a confirmation message to the first computing node according to the request for reselecting the computing node; if a feedback message of the confirmation message sent by the first computing node is not received, reselecting the second computing node from the self-organizing network group; wherein, after the initiating node sends a detection message to the first computing node, if it does not receive a feedback message of the detection message sent by the first computing node, it sends the request for reselecting the computing node.

[0135] Based on the above embodiments, the storage module in this implementation is specifically used for: if the storage node crashes, reselecting a new storage node from the self-organizing network group; storing the execution process and execution results of the second computing node in the new storage node.

[0136] On the basis of the above embodiments, in this implementation, when the collaborative management node is down, the collaborative management node is updated to another node with the smallest ID in the self-organizing network group except the collaborative management node.

[0137] Based on the above embodiments, the present implementation also includes a distribution module specifically used to: send the IP address of the second computing node to the initiating node in the self-organizing network group, so that the initiating node can send the target task to the second computing node to which the IP address belongs.

[0138] On the basis of the above embodiments, the self-organizing network grouping in this embodiment is obtained by grouping the self-organizing network using a network grouping algorithm; wherein the network grouping algorithm is a minimum ID grouping algorithm or a maximum connectivity grouping algorithm.

[0139] Figure 6 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 6 As shown, the electronic device may include: a processor 601, a communication interface 602, a memory 603 and a communication bus 604, wherein the processor 601, the communication interface 602 and the memory 603 communicate with each other through the communication bus 604. The processor 601 may call the logic instructions in the memory 603 to execute the task processing method, which includes: the collaborative management node in the self-organizing network group stores the execution process and execution result of the first computing node in the self-organizing network group in the process of executing the target task in the storage node in the self-organizing network group; if the first computing node crashes during the process of executing the target task, reselect the second computing node from the self-organizing network group; send the execution process and execution result of the first computing node in the storage node that executed the target task the most recently to the second computing node, so that the second computing node can continue to execute the target task according to the execution result and the execution process.

[0140] In addition, the logic instructions in the above-mentioned memory 603 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0141] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the task processing methods provided by the above-mentioned methods, and the method includes: the collaborative management node in the self-organizing network group stores the execution process and execution result of the first computing node in the self-organizing network group in the process of executing the target task in the storage node in the self-organizing network group; if a crash occurs during the process of the first computing node executing the target task, a second computing node is reselected from the self-organizing network group; the execution process and execution result of the most recent execution of the target task by the first computing node in the storage node are sent to the second computing node, so that the second computing node can continue to execute the target task according to the execution result and the execution process.

[0142] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the above-mentioned task processing methods, the methods comprising: the collaborative management node in the self-organizing network group stores the execution process and execution result of the first computing node in the self-organizing network group in the process of executing the target task in the storage node in the self-organizing network group; if a crash occurs during the process of the first computing node executing the target task, reselect a second computing node from the self-organizing network group; the execution process and execution result of the most recent execution of the target task by the first computing node in the storage node are sent to the second computing node, so that the second computing node can continue to execute the target task according to the execution result and the execution process.

[0143] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0144] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0145] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A task processing method, characterized in that: include: The collaborative management node in the self-organizing network group stores the execution process and execution result of the first computing node in the self-organizing network group in the process of executing the target task in the storage node in the self-organizing network group; If a downtime occurs during the execution of the target task by the first computing node, reselecting a second computing node from the self-organizing network group; Sending the execution process and execution result of the target task most recently executed by the first computing node in the storage node to the second computing node, so that the second computing node can continue to execute the target task according to the execution result and the execution process; The reselecting a second computing node from the self-organizing network group includes: Obtaining the computing efficiency of each node according to the remaining number of CPU cores, remaining memory space, communication bandwidth and disk space of each node in the self-organizing network cluster; Selecting other nodes with the highest computing efficiency except the first computing node from the self-organizing network group as the second computing node; The reselecting a second computing node from the self-organizing network group includes: Receiving a request for reselecting a computing node sent by an initiating node in the self-organizing network group, and sending a confirmation message to the first computing node according to the request for reselecting a computing node; If no feedback message of the confirmation message sent by the first computing node is received, reselecting the second computing node from the self-organizing network group; Wherein, after sending the detection message to the first computing node, the initiating node sends the request to reselect a computing node if the initiating node does not receive a feedback message of the detection message sent by the first computing node.

2. The task processing method according to claim 1, characterized in that: After sending the execution process and the execution result of the target task most recently executed by the first computing node in the storage node to the second computing node, the method further comprises: If the storage node is down, reselect a new storage node from the self-organizing network group; The execution process and execution result of the second computing node are stored in the new storage node.

3. The task processing method according to claim 1, characterized in that: When the collaborative management node is down, the collaborative management node is updated to another node with the smallest ID in the self-organizing network group except the collaborative management node.

4. The task processing method according to claim 1, characterized in that: Before sending the execution process and the execution result of the target task most recently executed by the first computing node in the storage node to the second computing node, the method further includes: The IP address of the second computing node is sent to the initiating node in the self-organizing network group, so that the initiating node can send the target task to the second computing node to which the IP address belongs.

5. The task processing method according to claim 1, characterized in that: The self-organizing network grouping is performed by grouping the self-organizing network using a network grouping algorithm; The network clustering algorithm is a minimum ID clustering algorithm or a maximum connectivity clustering algorithm.

6. A task processing device, characterized in that: include: A storage module, used for the collaborative management node in the self-organizing network group to store the execution process and execution result of the first computing node in the self-organizing network group in the process of executing the target task in the storage node in the self-organizing network group; A selection module, configured to reselect a second computing node from the self-organizing network group if a downtime occurs during the execution of the target task by the first computing node; A sending module, used for sending the execution process and execution result of the target task most recently executed by the first computing node in the storage node to the second computing node, so that the second computing node can continue to execute the target task according to the execution result and the execution process; The selection module is specifically used for: Obtaining the computing efficiency of each node according to the remaining number of CPU cores, remaining memory space, communication bandwidth and disk space of each node in the self-organizing network cluster; Selecting other nodes with the highest computing efficiency except the first computing node from the self-organizing network group as the second computing node; The selection module is further used for: Receiving a request for reselecting a computing node sent by an initiating node in the self-organizing network group, and sending a confirmation message to the first computing node according to the request for reselecting a computing node; If no feedback message of the confirmation message sent by the first computing node is received, reselecting the second computing node from the self-organizing network group; Wherein, after sending the detection message to the first computing node, the initiating node sends the request to reselect a computing node if the initiating node does not receive a feedback message of the detection message sent by the first computing node.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the task processing method according to any one of claims 1 to 5 are implemented.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the task processing method according to any one of claims 1 to 5 are implemented.

Images