Task processing method, device and system and node server

Abstract

The invention discloses a task processing method, device and system and a node server, and relates to the technical field of network information, the method comprises the following steps: when the node server receives node configuration information, searching corresponding node label information from the node configuration information; when the node label information is task processing information, receiving a to-be-executed task sent by other node servers; and then the task to be executed is processed, and a task processing result is transmitted to a distributed storage server ETCD. The node server defines the task role of the node server according to the searched node label information, and when the node label information is the task processing information, the to-be-executed task is obtained and processed, so that when the timed task is triggered, the task role of the to-be-executed task is defined; according to the node configuration information, a response is quickly made and the timed tasks to be processed are executed, so that the task processing problem in a distributed scene is effectively solved.

Description

Technical Field

[0001] The present invention relates to the field of network information technology, and in particular to a task processing method, device, system and node server. Background Art

[0002] In the Linux operating system, it is often necessary to execute some scheduled tasks. The existing task systems basically use the Cron server that comes with the Linux system to execute these scheduled tasks.

[0003] This method of using the Cron server to process tasks does not have the ability to perform distributed tasks and cannot solve the task processing problem in distributed scenarios.

[0004] The above content is only used to assist in understanding the technical solution of the present invention and does not constitute an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of the present invention is to provide a task processing method, device, system and node server, aiming to solve the technical problem that existing task processing methods cannot solve task processing in distributed scenarios.

[0006] To achieve the above object, the present invention provides a task processing method, which includes the following steps:

[0007] When the node server receives the node configuration information, it searches for the corresponding node label information from the node configuration information;

[0008] When the node label information is task processing information, receiving tasks to be executed sent by other node servers;

[0009] The tasks to be executed are processed and the task processing results are transmitted to the distributed storage server ETCD.

[0010] Optionally, when the node label information is task processing information, the step of receiving tasks to be executed sent by other node servers includes:

[0011] When the node label information is task processing information, extracting server extension definition information from the task processing information;

[0012] Performing server parameter configuration according to the server extension definition information;

[0013] When the server parameter configuration is completed, tasks to be executed sent by other node servers are received.

[0014] Optionally, the step of processing the task to be executed and transmitting the task processing result to the distributed storage server ETCD includes:

[0015] Processing the tasks to be executed and obtaining the task execution results;

[0016] Detecting whether there is a target task result that failed to execute in the task execution results;

[0017] When the target task result is detected to exist, the target task result is saved in the distributed storage server ETCD.

[0018] Optionally, when the node label information is task processing information, after the step of receiving tasks to be executed sent by other node servers, the task processing method further includes:

[0019] The tasks to be executed are processed, and the task processing results are stored in the form of logs in the structured storage software.

[0020] Optionally, after the step of searching the node configuration information for corresponding node label information when the node server receives the node configuration information, the task processing method further includes:

[0021] When the node label information is task assignment information, reading the task identifier included in the task assignment information;

[0022] Read the corresponding task to be executed in the distributed storage server ETCD according to the task identifier;

[0023] Allocate the tasks to be executed to other node servers.

[0024] Optionally, the step of allocating the tasks to be executed to other node servers includes:

[0025] Determining the association relationship between the tasks to be executed according to the task identifiers;

[0026] The tasks to be executed are distributed to other node servers according to the association relationship.

[0027] Optionally, the association relationship includes a parent-child relationship and / or a mutually exclusive relationship;

[0028] The step of allocating the tasks to be executed to other node servers according to the association relationship includes:

[0029] When there are tasks with the parent-child relationship among the tasks to be executed, allocating the tasks with the parent-child relationship to the same node server;

[0030] and / or,

[0031] When there are tasks with the mutually exclusive relationship among the tasks to be executed, tasks corresponding to the tasks with the mutually exclusive relationship are allocated to different node servers.

[0032] Optionally, after the step of allocating the to-be-executed tasks to other node servers, the task processing method includes:

[0033] Monitor the task execution process of the other node servers in real time to obtain monitoring results;

[0034] Determine whether there is a down server among the other node servers according to the monitoring result;

[0035] When the downtime server exists, unfinished tasks corresponding to the downtime server are obtained, and the unfinished tasks are reallocated.

[0036] Optionally, the step of reallocating the unfinished tasks includes:

[0037] Obtain the executable task type corresponding to each node server and the current task type corresponding to the unfinished task;

[0038] Matching the current task type according to the executable task type to obtain a matching result;

[0039] The unfinished tasks are reallocated according to the matching results.

[0040] Optionally, before the step of searching the node configuration information for corresponding node label information when the node server receives the node configuration information, the task processing method further includes:

[0041] The node server initiates the master node election according to the preset distributed election algorithm and obtains the election results;

[0042] An information acquisition request is generated according to the election result, and the information acquisition request is sent to the distributed storage server ETCD, so that the distributed storage server ETCD feeds back corresponding node configuration information.

[0043] Optionally, the node server initiates a master node election according to a preset distributed election algorithm and obtains the election result, including:

[0044] The node server obtains its corresponding node attribute information;

[0045] The Bully election algorithm is used to initiate the master node election based on the node attribute information and obtain the election results.

[0046] Optionally, the node attribute information includes a transaction identifier;

[0047] The step of initiating a master node election using the Bully election algorithm according to the node attribute information and obtaining the election result includes:

[0048] Obtaining the transaction identifier and determining the server code according to the transaction identifier;

[0049] Find a target node server whose code is greater than the server code, and send an election message to the target node server;

[0050] Obtaining an election result fed back by the target node server based on the election message.

[0051] Optionally, the task processing method further includes:

[0052] Get its own local Internet Protocol address;

[0053] Searching for a resource interface corresponding to the local Internet protocol address in a configured interface set;

[0054] The resource mounted under the local Internet protocol address is obtained according to the resource interface.

[0055] In addition, to achieve the above-mentioned purpose, the present invention further proposes a task processing device, the task processing device comprising:

[0056] An information receiving module is used to search for corresponding node label information from the node configuration information when receiving the node configuration information;

[0057] A task acquisition module, configured to receive tasks to be executed sent by other node servers when the node label information is task processing information;

[0058] The task processing module is used to process the tasks to be executed and transmit the task processing results to the distributed storage server ETCD.

[0059] Optionally, the task acquisition module is also used to extract server extension definition information from the task processing information when the node label information is task processing information; configure server parameters according to the server extension definition information; and receive tasks to be executed sent by other node servers when the server parameter configuration is completed.

[0060] Optionally, the task acquisition module is also used to process the task to be executed and obtain the task execution result; detect whether there is a target task result that failed to execute in the task execution result; when it is detected that the target task result exists, save the target task result to the distributed storage server ETCD.

[0061] Optionally, the task processing device further includes: a task allocation module;

[0062] The task assignment module is used to read the task identifier contained in the task assignment information when the node label information is task assignment information; read the corresponding task to be executed in the distributed storage server ETCD according to the task identifier; and assign the task to be executed to other node servers.

[0063] In addition, to achieve the above-mentioned purpose, the present invention also proposes a node server, which includes: a memory, a processor, and a task processing program stored on the memory and runnable on the processor, and the task processing program is configured to implement the steps of the task processing method described above.

[0064] In addition, to achieve the above-mentioned purpose, the present invention also proposes a task processing system, which includes a plurality of node servers as described above.

[0065] In addition, to achieve the above-mentioned purpose, the present invention further proposes a storage medium, on which a task processing program is stored. When the task processing program is executed by a processor, the steps of the task processing method described above are implemented.

[0066] When receiving node configuration information, the node server in the distributed task processing framework of the present invention searches for corresponding node label information from the node configuration information; then, when the node label information is task processing information, it receives tasks to be executed sent by other node servers; then, it processes the tasks to be executed, and transmits the task processing results to the distributed storage server ETCD. Since the node server in the present invention defines its own task role based on the node label information found, when the node label information is task processing information, it obtains and processes the tasks to be executed, so that when the scheduled tasks are triggered, it can quickly respond according to the node configuration information and execute these scheduled tasks to be processed, effectively solving the task processing problem in the distributed scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 Schematic diagram of a distributed task processing framework according to an embodiment of the present invention;

[0068] Figure 2This is a flowchart of a first embodiment of the task processing method of the present invention;

[0069] Figure 3 This is a flow chart of a second embodiment of the task processing method of the present invention;

[0070] Figure 4 This is a flowchart of a third embodiment of the task processing method of the present invention;

[0071] Figure 5 Schematic diagram of the master node election process in the third embodiment of the task processing method of the present invention;

[0072] Figure 6 This is a structural block diagram of the first embodiment of the task processing device of the present invention.

[0073] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0074] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0075] The embodiment of the present invention provides a task processing method.

[0076] It should be noted that the task processing method of the present invention can be applied to a distributed task processing framework composed of multiple node servers.

[0077] Reference Figure 1 , Figure 1 Schematic diagram of a distributed task processing framework according to an embodiment of the present invention.

[0078] like Figure 1 As shown, the distributed task processing framework includes multiple node servers (1, 2-1...2-N) and a distributed storage server ETCD (3).

[0079] Before processing a task, the node servers included in this distributed task processing framework can first jointly elect a node server 1 (i.e., a task allocation server) to perform task allocation. The remaining node servers (2-1…2-N) will then automatically serve as task execution servers (i.e., task execution servers) to execute or process the corresponding tasks. In this distributed task processing framework, there is typically only one task allocation server (the number can be set according to actual circumstances), but there can be multiple task execution servers.

[0080] Furthermore, in the distributed task processing framework of this embodiment, before each node server performs corresponding node configuration operations based on the received node configuration information, there are no significant differences in the functions that they can implement or possess. However, after receiving the node configuration information, different node servers will modify their own server configuration information based on the specific differences in the node configuration information to implement different server functions. For example, a node server can configure itself as a task allocation server or a task execution server based on the node configuration information.

[0081] Furthermore, in order to facilitate the unified management of node configuration information, the node configuration information described in this embodiment can be pre-stored in the distributed storage server ETCD. When a task needs to be executed, the distributed storage server ETCD can obtain the information request sent by the node server and send these node configuration information to the corresponding node server. Of course, the node server can also read the corresponding node configuration information from the distributed storage server ETCD.

[0082] Reference Figure 2 , Figure 2 Schematic diagram of the flow of the first embodiment of the task processing method of the present invention.

[0083] In this embodiment, the task processing method includes the following steps:

[0084] Step S10: When the node server receives the node configuration information, it searches for the corresponding node label information from the node configuration information;

[0085] It should be noted that the node configuration information may include server configuration parameters and node labels. In this embodiment, the node configuration information may be stored in the distributed storage server ETCD in the form of a configuration file. The node label information may be label information used by the node server to define its own role, and the label information may include task processing information or task allocation information.

[0086] In a specific implementation, when any node server in the distributed task processing framework receives node configuration information, it can read the corresponding node label information from the node configuration information, and then determine whether to perform task processing or task allocation based on the node label information.

[0087] Step S20: When the node label information is task processing information, receiving tasks to be executed sent by other node servers;

[0088] It is understandable that if the node tag information found by the node server is task processing information, it indicates that it will act as a task execution server to process the received pending tasks. However, since there is usually only one task allocation server in the entire distributed task processing framework, the node server will receive pending tasks assigned by other node servers (i.e., the task allocation server).

[0089] It should be understood that since the functions of the task allocation server and the task execution server are not the same, in this embodiment, when the node server detects that the node label information is task processing information, it will also extract the server extension definition information from the task processing information; then configure the server parameters according to the server extension definition information; and when the server parameter configuration is completed, receive the tasks to be executed sent by other node servers.

[0090] It should be noted that the server extension definition information includes the specific configuration method and required configuration parameters used by the node server when configuring itself as a task execution server.

[0091] Step S30: Process the task to be executed and transmit the task processing result to the distributed storage server ETCD.

[0092] In the specific implementation, after the node server obtains the task to be executed, it can start to process the task, and then send the task processing result to the distributed storage server ETCD for storage for subsequent viewing.

[0093] As an implementation method, in this embodiment, when the node server saves the task processing results, it can also store the task processing results in the form of logs, which can be stored in structured storage software (such as PostgreSQL), so as to facilitate subsequent queries on the task execution status and number of times.

[0094] Furthermore, considering that in actual operation, when the node server is processing tasks, there may be situations where task processing fails, and usually in the process of task processing, the task publisher is most concerned about these failed tasks. Therefore, in order to greatly reduce the amount of IO (Input / Output) operations, so that the distributed task processing framework can support infinite loop second-level task execution, the node server in this embodiment can only save the execution results of failed tasks when saving the task processing results. Specifically, the node server can process the task to be executed and obtain the task execution result; then detect in real time whether there is a target task result that failed in the task execution result; when the target task result is detected, the target task result is saved to the distributed storage server ETCD.

[0095] Furthermore, in this embodiment, the node servers that perform task processing may include: Shell node servers and GRPC node servers, and these two node servers can be used to process different tasks. Among them, when performing task processing, the GRPC node server will also obtain its own local Internet Protocol address (virtual IP address); then search for the resource interface corresponding to the local Internet Protocol address in the configured interface set; and then obtain the resources mounted under the local Internet Protocol address based on the resource interface. This method of accessing the back-end resources under the virtual IP address can prevent the single point problem of GRPC requests.

[0096] When the node server in the distributed task processing framework of this embodiment receives the node configuration information, it searches for the corresponding node label information from the node configuration information; then, when the node label information is task processing information, it receives the tasks to be executed sent by other node servers; then, it processes the tasks to be executed, and transmits the task processing results to the distributed storage server ETCD. Since the node server in this embodiment defines its own task role based on the node label information found, when the node label information is task processing information, it obtains the tasks to be executed and processes them, so that when the scheduled tasks are triggered, it can quickly respond according to the node configuration information and execute these scheduled tasks to be processed, effectively solving the task processing problem in the distributed scenario.

[0097] refer to Figure 3 , Figure 3 2 is a flow chart of the second embodiment of the task processing method of the present invention.

[0098] Based on the first embodiment above, after step S10, the task processing method of this embodiment further includes:

[0099] Step S20': when the node label information is task assignment information, reading the task identifier contained in the task assignment information;

[0100] In this embodiment, the task identifier may be a mark or code that can represent the uniqueness of the task or the attributes of the task.

[0101] It should be understood that if the node label information that the node server searches for is task allocation information, it indicates that it will act as a task allocation server to allocate tasks to be executed.

[0102] In a specific implementation, to conserve each node server's operating memory and storage resources, all pending tasks can be pre-stored in the distributed storage server ETCD. Furthermore, to improve the node server's task reading efficiency, a mapping relationship between the pending task's task identifier and the task's storage path / address can be pre-established. Upon receiving the task identifier, the node server can quickly determine the pending task's storage path / address by querying this mapping relationship, thereby improving task reading efficiency.

[0103] Step S30': reading the corresponding task to be executed in the distributed storage server ETCD according to the task identifier;

[0104] In a specific implementation, when the node server obtains the task allocation information, it can first configure itself as a task allocation server according to the received node configuration information, and then search for the corresponding task storage path / storage address in the above mapping relationship according to the task identifier carried in the task allocation information, and then read the corresponding task to be executed from the distributed storage server ETCD according to the task storage path / storage address.

[0105] Of course, if the node server reads multiple task identifiers, it means that there are a large number of tasks to be executed. At this time, the node server can use asynchronous operations to simultaneously call multiple threads to read the tasks to be executed from the distributed storage server ETCD, thereby improving the overall task processing efficiency.

[0106] Step S40': Allocate the tasks to be executed to other node servers.

[0107] It should be noted that the other node servers mentioned in this embodiment refer to task execution servers.

[0108] In a specific implementation, after obtaining the tasks to be executed, the node server can distribute the tasks to be executed to other node servers for processing.

[0109] Considering that there may be interrelated pending tasks in the actual task processing process, when assigning such tasks, the association relationship between tasks (including parent-child relationships and / or mutually exclusive relationships) needs to be considered to prevent mutually exclusive tasks from being assigned to the same node server for processing, or tasks that need to be processed by the same node server (such as parent-child tasks) from being assigned to different node servers. Therefore, in this embodiment, when a node server obtains a task identifier, it can first determine the association relationship between the pending tasks based on the task identifier; and then assign the pending tasks to other node servers based on the association relationship.

[0110] Of course, in order to realize the operation of determining the association relationship between tasks based on task identifiers, the distributed task processing framework of this embodiment can first separately configure the task identifiers of the tasks to be processed with the association relationship when receiving the tasks to be executed. For example, the task identifiers of the father and son tasks are set to fs001, fs002, etc.; the task identifiers of the mutual exclusion tasks are set to me001, me002, etc. (this is only an example and a specific limitation).

[0111] In a specific implementation, if a node server detects that there are tasks with the parent-child relationship among the tasks to be executed, the tasks with the parent-child relationship will be assigned to the same node server; if there are tasks with the mutually exclusive relationship among the tasks to be executed, the tasks corresponding to the tasks with the mutually exclusive relationship will be assigned to different node servers, so that the distributed task processing framework can effectively support parent-child tasks and mutually exclusive tasks.

[0112] Furthermore, in order to ensure the effectiveness of task processing and avoid the inability to execute tasks in a timely manner due to downtime or failure of certain node servers, the node server of this embodiment will also monitor the task execution process of the other node servers in real time to obtain monitoring results; then, based on the monitoring results, it is determined whether there is a downtime server among the other node servers; when there is a downtime server, the unfinished tasks corresponding to the downtime server are obtained and the unfinished tasks are reallocated, thereby improving the disaster recovery capability of the entire distributed task processing framework and automatically drifting the unfinished tasks to non-faulty node servers for processing.

[0113] Furthermore, to prevent unfinished tasks from being assigned to a node server that cannot handle tasks of that type (for example, assigning mutually exclusive tasks to the same node server, or assigning a type A task to a node server that can only execute type B tasks), and to improve the effectiveness of task reallocation, the node server of this embodiment may first obtain the executable task types corresponding to each node server and the current task type corresponding to the unfinished task when reallocating unfinished tasks; then, match the current task type with the executable task type to obtain a matching result; and finally, reallocate the unfinished task based on the matching result.

[0114] In the distributed task processing framework of this embodiment, when the node label information is task allocation information, the node server reads the task identifier contained in the task allocation information; then reads the corresponding task to be executed in the distributed storage server ETCD according to the task identifier; and then allocates the task to be executed to other node servers, which can improve the allocation efficiency of scheduled tasks and effectively adapt to task allocation in distributed task scenarios.

[0115] refer to Figure 4 , Figure 4 2 is a flowchart of the third embodiment of the task processing method of the present invention.

[0116] Based on the above embodiments, in this embodiment, before step S10, the task processing method further includes:

[0117] Step S01: The node server initiates a master node election according to a preset distributed election algorithm and obtains the election result;

[0118] It should be noted that the node servers in the distributed task processing framework of this embodiment can automatically select node servers for task allocation before performing task processing, and then the remaining node servers automatically serve as node servers for task processing.

[0119] It should be understood that the preset distributed election algorithm can be a solution for selecting a master node server in a distributed scenario. This embodiment is mainly used to elect a master node server (task allocation server).

[0120] In a specific implementation, the node server in the distributed task processing framework can initiate a master node election according to a preset distributed election algorithm and obtain the election results.

[0121] As an implementation method, the preset distributed algorithm in this embodiment may be a Bully election algorithm. Specifically, the node server obtains its own corresponding node attribute information; then, based on the node attribute information, uses the Bully election algorithm to initiate a master node election and obtain the election result.

[0122] The node attribute information may be the transaction identifier of the node server, and these transaction identifiers may be codes or numbers in order of size (for example, transaction identifiers: 1, 2, 3...N), which are used to determine the corresponding node server code. The so-called Bully election algorithm, also known as the bully election algorithm, can be used to select the node server with the largest ID (identifier or number) among the surviving processes (normal node servers) as the task allocation server. This embodiment uses the Bully election algorithm to make the election process simple, effective, and easy to implement, thereby improving the efficiency of selecting the master node.

[0123] In actual applications, each node server in the distributed task processing framework of this embodiment can first obtain its own transaction identifier, and then determine the server code based on the transaction identifier; then find the target node server whose code is greater than the server code, and send an election message to the target node server; then obtain the election result fed back by the target node server based on the election message.

[0124] It should be noted that, when the node server in this embodiment elects a master node using the Bully election algorithm, the specific election process may be as follows:

[0125] The node server first determines its own server code based on the read transaction identifier, and then checks whether the server code is the first in the sort order or the code with the largest value;

[0126] If yes, it will directly send a message of election success to the other node servers and become the master node server;

[0127] If not, search for all node servers whose codes are greater than the code of the own server (i.e., the target node servers), then send election messages to these node servers, and simultaneously obtain the election results fed back by these node servers based on the received election messages;

[0128] If the received election result is empty, or if no feedback is received from any node server, it will directly send an election success message to all node servers except itself and become the master node server;

[0129] If the received election result is not empty, or if a server in the target node server feedbacks the message of successful election, stop sending any messages and end the election operation.

[0130] As another implementation, the preset distributed election algorithm in this embodiment may be a Raft distributed election algorithm.

[0131] Reference Figure 5 , Figure 5 Schematic diagram of the master node election process in the third embodiment of the task processing method of the present invention.

[0132] like Figure 5 As shown, in this embodiment, when the node server elects the master node through the Raft distributed election algorithm, the specific election process can be as follows:

[0133] Every node server in the distributed task processing framework participates in the master node election. For example, node server C can send a "Vote for me" request to another node server F (Follower). If the other node server agrees, it replies "OK" to node server C. When (N / 2)+1 node servers F in the entire distributed task processing framework reply "OK" to node server C, node server C successfully becomes the master node server. N is the total number of node servers in the distributed task processing framework.

[0134] Step S02: Generate an information acquisition request according to the election result, and send the information acquisition request to the distributed storage server ETCD, so that the distributed storage server ETCD feeds back corresponding node configuration information.

[0135] It is understandable that after the node server in the distributed task processing framework elects the master node and obtains the election results in accordance with the above step S01, it can determine whether it has been successful in the election. If successful, it means that it will be used as the master node server for task allocation; otherwise, it means that the election has failed and it will be used as a slave node server to process tasks.

[0136] It should be noted that since the functions that can be achieved as a task allocation server and as a task execution server are different, the corresponding node configuration information is also different. In order to save the storage resource space of each node server and facilitate the update and modification of the node configuration information, these node configuration information can be stored in the distributed storage server ETCD in advance, so that the node server can generate an information acquisition request based on the election results after obtaining the election results, so as to obtain the corresponding node configuration information from the distributed storage server ETCD and complete the subsequent server parameter configuration.

[0137] In this embodiment, the node server initiates a master node election according to a preset distributed election algorithm and obtains the election results; then, it generates an information acquisition request based on the election results, and sends the information acquisition request to the distributed storage server ETCD, so that the distributed storage server ETCD can feedback the corresponding node configuration information, effectively select the node server for task allocation, and ensure the smooth progress of task processing.

[0138] In addition, to achieve the above-mentioned purpose, the present invention also proposes a node server, which includes: a memory, a processor, and a task processing program stored on the memory and runnable on the processor, and the task processing program is configured to implement the steps of the task processing method embodiment described above.

[0139] In addition, an embodiment of the present invention further provides a task processing system, which includes a plurality of the node servers described above.

[0140] Furthermore, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, on which a task processing program is stored. When the task processing program is executed by a processor, the steps of the task processing method embodiment described above are implemented.

[0141] In addition, an embodiment of the present invention also provides a task processing device. Figure 6 , Figure 6 This is a structural block diagram of the first embodiment of the task processing device of the present invention.

[0142] like Figure 6 As shown, in the first embodiment of the task processing device of the present invention, the task processing device includes:

[0143] The information receiving module 601 is used to search for corresponding node label information from the node configuration information when receiving the node configuration information;

[0144] The task acquisition module 602 is used to receive tasks to be executed sent by other node servers when the node label information is task processing information;

[0145] The task processing module 603 is used to process the task to be executed and transmit the task processing result to the distributed storage server ETCD.

[0146] When the task processing device of this embodiment receives the node configuration information, it searches for the corresponding node label information from the node configuration information; then, when the node label information is task processing information, it receives the pending tasks sent by other node servers; then, it processes the pending tasks and transmits the task processing results to the distributed storage server ETCD. Since the node server in this embodiment defines its own task role based on the node label information found, when the node label information is task processing information, it obtains the pending tasks and processes them, so that when the scheduled tasks are triggered, it can quickly respond according to the node configuration information and execute these pending scheduled tasks, effectively solving the task processing problem in distributed scenarios.

[0147] Based on the first embodiment of the task processing device of the present invention, a second embodiment of the task processing device of the present invention is proposed.

[0148] In this embodiment, the task acquisition module 602 is also used to extract server extension definition information from the task processing information when the node label information is task processing information; configure server parameters according to the server extension definition information; and receive tasks to be executed sent by other node servers when the server parameter configuration is completed.

[0149] Optionally, the task acquisition module 602 is also used to extract server extension definition information from the task processing information when the node label information is task processing information; configure server parameters according to the server extension definition information; and receive tasks to be executed sent by other node servers when the server parameter configuration is completed.

[0150] Optionally, the task processing module 603 is also used to process the task to be executed and obtain the task execution result; detect whether there is a target task result that failed to execute in the task execution result; when it is detected that there is the target task result, save the target task result to the distributed storage server ETCD.

[0151] Optionally, the task processing module 603 is further configured to process the to-be-executed task and store the task processing result in the form of a log in structured storage software.

[0152] Optionally, the task processing device also includes: a task allocation module; the task allocation module is used to read the task identifier contained in the task allocation information when the node label information is task allocation information; read the corresponding task to be executed in the distributed storage server ETCD according to the task identifier; and allocate the task to be executed to other node servers.

[0153] Optionally, the task allocation module is further configured to determine associations between the tasks to be executed according to the task identifiers; and allocate the tasks to be executed to other node servers according to the associations.

[0154] Optionally, the task allocation module is also used to allocate the tasks with a parent-child relationship to the same node server when there are tasks with a parent-child relationship among the tasks to be executed; and / or, when there are tasks with a mutually exclusive relationship among the tasks to be executed, allocate the tasks corresponding to the tasks with the mutually exclusive relationship to different node servers.

[0155] Optionally, the task allocation module is also used to monitor the task execution process of the other node servers in real time to obtain monitoring results; determine whether there is a down server among the other node servers based on the monitoring results; when there is a down server, obtain the unfinished tasks corresponding to the down server and reallocate the unfinished tasks.

[0156] Optionally, the task allocation module is also used to obtain the executable task type corresponding to each node server and the current task type corresponding to the unfinished task; match the current task type according to the executable task type to obtain a matching result; and reallocate the unfinished task according to the matching result.

[0157] Optionally, the task processing device also includes: an election module; the election module is used to initiate a master node election according to a preset distributed election algorithm and obtain an election result; generate an information acquisition request based on the election result, and send the information acquisition request to the distributed storage server ETCD, so that the distributed storage server ETCD can feedback the corresponding node configuration information.

[0158] Optionally, the election module is further used to obtain node attribute information corresponding to itself; initiate a master node election using the Bully election algorithm based on the node attribute information, and obtain the election result.

[0159] Optionally, the election module is also used to obtain a transaction identifier and determine a server code based on the transaction identifier; search for a target node server whose code is greater than the server code and send an election message to the target node server; and obtain an election result fed back by the target node server based on the election message.

[0160] Optionally, the election module is further configured to obtain its own local Internet Protocol address; search for a resource interface corresponding to the local Internet Protocol address in a configured interface set; and obtain resources mounted under the local Internet Protocol address according to the resource interface.

[0161] Other embodiments or specific implementations of the task processing device of the present invention can refer to the above-mentioned method embodiments and will not be described in detail here.

[0162] The present invention discloses A1, a task processing method, the task processing method comprising:

[0163] When the node server receives the node configuration information, it searches for the corresponding node label information from the node configuration information;

[0164] When the node label information is task processing information, receiving tasks to be executed sent by other node servers;

[0165] The tasks to be executed are processed and the task processing results are transmitted to the distributed storage server ETCD.

[0166] A2. The task processing method as described in A1, wherein when the node label information is task processing information, the step of receiving tasks to be executed sent by other node servers comprises:

[0167] When the node label information is task processing information, extracting server extension definition information from the task processing information;

[0168] Performing server parameter configuration according to the server extension definition information;

[0169] When the server parameter configuration is completed, tasks to be executed sent by other node servers are received.

[0170] A3. The task processing method as described in A1, wherein the step of processing the task to be executed and transmitting the task processing result to the distributed storage server ETCD comprises:

[0171] Processing the tasks to be executed and obtaining the task execution results;

[0172] Detecting whether there is a target task result that failed to execute in the task execution results;

[0173] When the target task result is detected to exist, the target task result is saved in the distributed storage server ETCD.

[0174] A4. The task processing method according to A1, wherein when the node label information is task processing information, after the step of receiving tasks to be executed sent by other node servers, the task processing method further comprises:

[0175] The tasks to be executed are processed, and the task processing results are stored in the form of logs in the structured storage software.

[0176] A5. In the task processing method described in A1, after the node server searches for corresponding node label information from the node configuration information upon receiving the node configuration information, the task processing method further comprises:

[0177] When the node label information is task assignment information, reading the task identifier included in the task assignment information;

[0178] Read the corresponding task to be executed in the distributed storage server ETCD according to the task identifier;

[0179] Allocate the tasks to be executed to other node servers.

[0180] A6. In the task processing method described in A5, the step of allocating the to-be-executed tasks to other node servers comprises:

[0181] Determining the association relationship between the tasks to be executed according to the task identifiers;

[0182] The tasks to be executed are distributed to other node servers according to the association relationship.

[0183] A7. The task processing method according to A6, wherein the association relationship includes a parent-child relationship and / or a mutually exclusive relationship;

[0184] The step of allocating the tasks to be executed to other node servers according to the association relationship includes:

[0185] When there are tasks with the parent-child relationship among the tasks to be executed, allocating the tasks with the parent-child relationship to the same node server;

[0186] and / or,

[0187] When there are tasks with the mutually exclusive relationship among the tasks to be executed, tasks corresponding to the tasks with the mutually exclusive relationship are allocated to different node servers.

[0188] A8. The task processing method according to A5, after the step of allocating the to-be-executed tasks to other node servers, the task processing method includes:

[0189] Monitor the task execution process of the other node servers in real time to obtain monitoring results;

[0190] Determine whether there is a down server among the other node servers according to the monitoring result;

[0191] When the downtime server exists, unfinished tasks corresponding to the downtime server are obtained, and the unfinished tasks are reallocated.

[0192] A9. The task processing method according to A8, wherein the step of reallocating the unfinished tasks comprises:

[0193] Obtain the executable task type corresponding to each node server and the current task type corresponding to the unfinished task;

[0194] Matching the current task type according to the executable task type to obtain a matching result;

[0195] The unfinished tasks are reallocated according to the matching results.

[0196] A10. The task processing method according to any one of A1 to A9, wherein before the step of searching the node configuration information for corresponding node label information upon receiving the node configuration information, the task processing method further comprises:

[0197] The node server initiates the master node election according to the preset distributed election algorithm and obtains the election results;

[0198] An information acquisition request is generated according to the election result, and the information acquisition request is sent to the distributed storage server ETCD, so that the distributed storage server ETCD feeds back corresponding node configuration information.

[0199] 11. In the task processing method described in A10, the node server initiates a master node election according to a preset distributed election algorithm and obtains the election result, including:

[0200] The node server obtains its corresponding node attribute information;

[0201] The Bully election algorithm is used to initiate the master node election based on the node attribute information and obtain the election results.

[0202] 12. The task processing method according to A11, wherein the node attribute information includes a transaction identifier;

[0203] The step of initiating a master node election using the Bully election algorithm according to the node attribute information and obtaining the election result includes:

[0204] Obtaining the transaction identifier and determining the server code according to the transaction identifier;

[0205] Find a target node server whose code is greater than the server code, and send an election message to the target node server;

[0206] Obtaining an election result fed back by the target node server based on the election message.

[0207] 13. The task processing method according to any one of A1 to A12, further comprising:

[0208] Get its own local Internet Protocol address;

[0209] Searching for a resource interface corresponding to the local Internet protocol address in a configured interface set;

[0210] The resource mounted under the local Internet protocol address is obtained according to the resource interface.

[0211] The present invention further discloses B14, a task processing device, comprising:

[0212] An information receiving module is used to search for corresponding node label information from the node configuration information when receiving the node configuration information;

[0213] A task acquisition module, configured to receive tasks to be executed sent by other node servers when the node label information is task processing information;

[0214] The task processing module is used to process the tasks to be executed and transmit the task processing results to the distributed storage server ETCD.

[0215] B15. In the task processing device as described in B14, the task acquisition module is further used to extract server extension definition information from the task processing information when the node label information is task processing information; configure server parameters according to the server extension definition information; and receive tasks to be executed sent by other node servers when the server parameter configuration is completed.

[0216] B16. In the task processing device as described in B14, the task acquisition module is further used to process the task to be executed and obtain the task execution result; detect whether there is a target task result that failed to execute in the task execution result; when the target task result is detected, save the target task result to the distributed storage server ETCD.

[0217] B17. The task processing device as described in B14, further comprising: a task allocation module;

[0218] The task assignment module is used to read the task identifier contained in the task assignment information when the node label information is task assignment information; read the corresponding task to be executed in the distributed storage server ETCD according to the task identifier; and assign the task to be executed to other node servers.

[0219] The present invention also discloses C18, a node server, which includes: a memory, a processor, and a task processing program stored in the memory and executable on the processor, wherein the task processing program is configured to implement the steps of the task processing method described above.

[0220] The present invention also discloses D19, a task processing system, which includes a plurality of node servers as described above.

[0221] The present invention further discloses E20, a storage medium, on which a task processing program is stored. When the task processing program is executed by a processor, the steps of the task processing method described above are implemented.

[0222] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0223] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0224] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0225] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A task processing method, characterized in that: The task processing method includes: When the node server receives the node configuration information, it searches for the corresponding node label information from the node configuration information; When the node label information is task processing information, receiving tasks to be executed sent by other node servers; The tasks to be executed are processed and the task processing results are transmitted to the distributed storage server ETCD.

2. The task processing method according to claim 1, wherein: When the node label information is task processing information, the step of receiving tasks to be executed sent by other node servers includes: When the node label information is task processing information, extracting server extension definition information from the task processing information; Performing server parameter configuration according to the server extension definition information; When the server parameter configuration is completed, tasks to be executed sent by other node servers are received.

3. The task processing method according to claim 1, wherein: The step of processing the task to be executed and transmitting the task processing result to the distributed storage server ETCD includes: Processing the tasks to be executed and obtaining the task execution results; Detecting whether there is a target task result that failed to execute in the task execution results; When the target task result is detected to exist, the target task result is saved in the distributed storage server ETCD.

4. The task processing method according to claim 1, wherein: When the node label information is task processing information, after the step of receiving tasks to be executed sent by other node servers, the task processing method further includes: The tasks to be executed are processed, and the task processing results are stored in the form of logs in the structured storage software.

5. The task processing method according to claim 1, wherein: After the step of searching the node configuration information for corresponding node label information when the node server receives the node configuration information, the task processing method further includes: When the node label information is task assignment information, reading the task identifier included in the task assignment information; Read the corresponding task to be executed in the distributed storage server ETCD according to the task identifier; Allocate the tasks to be executed to other node servers.

6. The task processing method according to any one of claims 1 to 5, characterized in that: Before the step of searching the node configuration information for corresponding node label information when the node server receives the node configuration information, the task processing method further includes: The node server initiates the master node election according to the preset distributed election algorithm and obtains the election results; An information acquisition request is generated according to the election result, and the information acquisition request is sent to the distributed storage server ETCD, so that the distributed storage server ETCD feeds back corresponding node configuration information.

7. A task processing device, characterized in that: The task processing device includes: An information receiving module is used to search for corresponding node label information from the node configuration information when receiving the node configuration information; A task acquisition module, configured to receive tasks to be executed sent by other node servers when the node label information is task processing information; The task processing module is used to process the tasks to be executed and transmit the task processing results to the distributed storage server ETCD.

8. A node server, characterized in that: The node server includes: a memory, a processor, and a task processing program stored in the memory and executable on the processor, wherein the task processing program is configured to implement the steps of the task processing method according to any one of claims 1 to 6.

9. A task processing system, characterized in that: The task processing system includes a plurality of node servers according to claim 8.

10. A storage medium, characterized in that: The storage medium stores a task processing program, which, when executed by a processor, implements the steps of the task processing method according to any one of claims 1 to 6.

Images