High-availability task instance execution method and system

Abstract

The invention relates to the technical field of task management, and discloses a high-availability task instance execution method and system. According to the application, a task management node is determined from a server node of a server side through election, a virtual address is generated by utilizing a physical route of the server side, the virtual address is bound to the task management node so as to receive task configuration information through the virtual address, a corresponding task instance is generated by utilizing the task management node according to the task configuration information, and the task instance is sent to the server side. The task execution node is determined from the server nodes except the task management node, the received task instance is executed through the task execution node, the election strategy for the task management node is achieved through the virtual address, high availability of task management is achieved, the task instance is executed through the task execution node, and the task management efficiency is improved. Server nodes are subjected to responsibility separation, and task resources are reasonably allocated in combination with an election strategy and responsibility separation, so that the task execution efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of task management, and in particular, to a highly available task instance execution method and system. Background Art

[0002] At present, as the core hub of the data processing process, the big data offline scheduling platform mainly undertakes the coordination and execution of tasks such as data analysis, report generation, and machine learning model training in multiple business fields. In order to improve the resource utilization rate of services and save operation and maintenance costs, the existing technical system is usually built based on a distributed scheduling framework, using a common cluster, allowing multiple business departments to share resources, and supporting core functions such as task dependency management, periodic scheduling, multiple task types, and resource isolation to achieve efficient task scheduling.

[0003] However, since the common cluster consists of multiple server nodes, each server node needs to undertake all core functions such as global resource scheduling, task execution, status monitoring, and fault recovery. Integrating multiple functions into one server node will lead to unreasonable task allocation between different server nodes when the number of tasks reaches the peak. The server node with more allocated tasks will cause memory overflow and even task failure, while the server node with fewer allocated tasks will waste computing resources, resulting in a low task processing efficiency of the entire big data offline scheduling platform. Summary of the Invention

[0004] To provide a basic understanding of some aspects of the disclosed embodiments, a simple summary is given below. This summary is not a comprehensive review, nor is it intended to identify key / important constituent elements or delineate the scope of protection of these embodiments, but rather serves as a preamble to the subsequent detailed description.

[0005] In view of the above-mentioned disadvantages of the prior art, the present application provides a highly available task instance execution method and system to improve task processing efficiency.

[0006] The present application provides a highly available task instance execution method, which is applied to the server side. The method includes: the server side includes a server node and a physical route; an election is performed based on the server node to obtain a task management node, and a virtual address is generated according to the physical route and bound to the task management node; if task configuration information is received through the virtual address, the task management node is used to generate a corresponding task instance according to the task configuration information; the task instance is sent to a task execution node through the task management node to trigger the task execution node to execute the received task instance, where the task execution node is determined from server nodes other than the task management node.

[0007] In an embodiment of the present application, an election is performed according to the server nodes to obtain a task management node, including: obtaining the management weights corresponding to the server nodes respectively, and determining each server node as the current node; in response to the current node, determining whether the current node meets the first preset condition, the second preset condition, and the third preset condition, where the first preset condition includes that there is a task management node among the server nodes other than the current node, the second preset condition includes that the current election mode is the preemption module, and the third preset condition includes that the management weight is greater than or equal to the preset weight threshold; if the current node meets the first preset condition and the current node does not meet the second preset condition, then determining the current node as an alternative node; if the current node meets the first preset condition, the second preset condition, and the third preset condition, then determining the current node as the task management node; if the current node meets the first preset condition and the second preset condition, and the current node does not meet the third preset condition, then determining a task management node from the current node and the current task management node according to the management weight; if the current node does not meet the first preset condition and the current node meets the third preset condition, then determining the current node as the task management node; if the current node does not meet the first preset condition and the third preset condition, then determining the current node as an alternative node, and determining a task management node from the alternative nodes according to the management weight.

[0008] In an embodiment of the present application, the method further includes at least one of the following: performing heartbeat monitoring on the task management node, and if it is detected that the task management node is inactivated, then reducing the management weight of the task management node and re-electing the task management node; performing heartbeat monitoring on the task execution node, and if it is detected that the task execution node is inactivated, then re-determining the task execution node corresponding to the task instance.

[0009] In an embodiment of the present application, the task management node generates a corresponding task instance according to the task configuration information in the following manner: pre-setting a task database and a task template in the task management node; determining the task requirements according to the task configuration information, and extracting data from the task database according to the task requirements to obtain task-related data; filling the task template according to the task configuration information and the task-related data to obtain the task instance.

[0010] In an embodiment of the present application, the task execution node executes a task instance in the following manner: An execution logic plugin is pre-set in the task execution node; a plugin configuration file corresponding to the execution logic plugin is generated according to the task instance, where the plugin configuration file carries a data source, a target source, and a structured query statement; the execution logic plugin is started according to the plugin configuration file to execute the task instance, where the task instance is used to extract data to be imported from the data source through the structured query statement and import the data to be imported into the target source.

[0011] In an embodiment of the present application, the method further includes: The server side further includes a data query engine; if a data query request is obtained through the virtual address, the user permission corresponding to the data query request is obtained; the data query request is executed by using the query engine to obtain target data, and sensitive information that does not meet the user permission is filtered from the target data; a viewing function and / or a downloading function corresponding to the target data are provided.

[0012] In an embodiment of the present application, the method further includes: Using the task execution node to generate a task execution result corresponding to the task instance, and sending the task execution result to the task management node; if the task execution result is a task failure, using the task management node to extract the task responsible person and the exception notification rule from the task configuration information, so as to send the task execution result to the task responsible person according to the exception notification rule.

[0013] The present application provides a highly available method for executing a task instance, which is applied to a user terminal. The method includes: obtaining task configuration information; sending the task configuration information to a server side, where the server side includes server nodes and a physical route. The server side is used to perform an election according to the server nodes to obtain a task management node, and generate a virtual address according to the physical route, bind the virtual address to the task management node. If the task configuration information is received through the virtual address, the task management node is used to generate a corresponding task instance according to the task configuration information, and the task instance is sent to the task execution node through the task management node, triggering the task execution node to execute the received task instance, and the task execution node is determined from the server nodes other than the task management node.

[0014] In an embodiment of the present application, obtaining task configuration information includes: obtaining a task configuration template, where the task configuration template includes at least one of a basic attribute field, an alarm attribute field, and an extended attribute field; receiving, through the task configuration template, task configuration information input by a user, where the basic attribute field is used to carry at least one of a task type, a task execution time, a task execution period, a task dependency relationship, and a person in charge of the task, the alarm attribute field is used to carry an exception notification rule, and the extended attribute field is used to carry at least one of a data source, a target source, and a structured query statement.

[0015] The present application provides a highly available task instance execution system, including: a user terminal configured to obtain task configuration information and send the task configuration information to a server side; the server side includes server nodes and a physical route, and the server side is configured to perform an election based on the server nodes to obtain a task management node, generate a virtual address according to the physical route, bind the virtual address to the task management node, if receiving task configuration information through the virtual address, then using the task management node to generate a corresponding task instance according to the task configuration information, sending the task instance to a task execution node through the task management node, and triggering the task execution node to execute the received task instance, and the task execution node is determined from server nodes other than the task management node.

[0016] The beneficial effects of the present application:

[0017] By performing an election to determine a task management node from the server nodes of the server side, generating a virtual address using the physical route of the server side, binding the virtual address to the task management node to receive task configuration information through the virtual address, using the task management node to generate a corresponding task instance according to the task configuration information, and determining a task execution node from server nodes other than the task management node, and triggering the task execution node to execute the received task instance. In this way, not only is an election strategy for the task management node implemented using the virtual address, achieving high availability of task management, but also the server nodes are separated in responsibilities by the task execution node to execute task instances, and the task resources are reasonably allocated by combining the election strategy and responsibility separation, reducing the occurrence probability of problems such as task resource waste and memory overflow, thereby improving the task execution efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a schematic structural diagram of an application environment in an embodiment of the present application

[0019] Figure 2 is a schematic flowchart of a highly available task instance execution method in an embodiment of the present application;

[0020] Figure 3 It is a schematic flowchart of an election method for a task management node in an embodiment of the present application;

[0021] Figure 4 It is a schematic flowchart of a highly available task instance execution method in an embodiment of the present application;

[0022] Figure 5 It is a schematic flowchart of a highly available task instance execution method in an embodiment of the present application;

[0023] Figure 6 It is a schematic structural diagram of a highly available task instance execution system in an embodiment of the present application;

[0024] Figure 7 It is a schematic structural diagram of a system architecture in an embodiment of the present invention. Detailed implementation manners

[0025] The following uses specific specific examples to illustrate the implementation manners of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation manners. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, without conflict, the following embodiments and sub-samples in the embodiments can be combined with each other.

[0026] It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner. Therefore, only the components related to the present invention are shown in the diagrams, rather than being drawn according to the number, shape, and size of the components in actual implementation. The types, quantities, and proportions of the components in actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0027] In the following description, a large number of details are discussed to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0028] The terms "first", "second", etc. in the specification, claims, and above-mentioned drawings of the present application are used to distinguish similar objects and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so as to implement the embodiments of the present application described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion.

[0029] Unless otherwise specified, the term "plural" means two or more.

[0030] In this application, the character " / " indicates that the objects before and after are in an "or" relationship. For example, A / B means: A or B.

[0031] The term "and / or" is an associative relationship describing an object, indicating that there can be three relationships. For example, A and / or B means: A or B, or, the three relationships of A and B.

[0032] Combined with Figure 1 As shown, this application provides an application environment for implementing a highly available task instance execution method. The application environment includes a server side and a user terminal.

[0033] The server side includes server nodes and a physical route. Among them, the server side communicates with the user terminal through a network.

[0034] Combined with Figure 2 As shown, this application provides a highly available task instance execution method, which is applied to the server side. The method includes:

[0035] Step S201: Elect a task management node according to the server nodes, generate a virtual address according to the physical route, and bind the virtual address to the task management node;

[0036] Step S202: If task configuration information is received through the virtual address, use the task management node to generate a corresponding task instance according to the task configuration information;

[0037] Step S203: Send the task instance to the task execution node through the task management node, and trigger the task execution node to execute the received task instance;

[0038] Among them, the task execution node is determined from server nodes other than the task management node.

[0039] By adopting the highly available task instance execution method provided in this application, a task management node is determined from the server nodes on the server side through election, and a virtual address is generated using the physical routing on the server side and bound to the task management node to receive task configuration information through the virtual address. The task management node is used to generate corresponding task instances according to the task configuration information, and a task execution node is determined from the server nodes other than the task management node, and the received task instances are executed through the task execution node. In this way, not only is the election strategy for the task management node implemented using the virtual address, achieving high availability of task management, but also the task instances are executed through the task execution node, separating the responsibilities of the server nodes, reasonably allocating task resources by combining the election strategy and responsibility separation, reducing the occurrence probability of problems such as task resource waste and memory overflow, and thus improving the task execution efficiency.

[0040] Optionally, the method further includes at least one of the following: performing heartbeat monitoring on the task management node, and if the task management node is detected to be inactivated, reducing the management weight of the task management node and re-electing the task management node; performing heartbeat monitoring on the task execution node, and if the task execution node is detected to be inactivated, re-determining the task execution node corresponding to the task instance.

[0041] In some embodiments, the task management node is responsible for managing resource calculation and task instance distribution, taking memory and CPU as resource units, and distributing task resources to different task instances, making the resource allocation more reasonable.

[0042] In some embodiments, the task management node creates task instances according to the task time and task cycle in the task configuration information; whenever the task time is triggered, the task management node will issue task instances to the task execution nodes with sufficient resources in a polling manner, and through communication experiments, use a storage format with higher communication efficiency to carry task instance information.

[0043] In some embodiments, the number of task execution nodes is one or more, and all task execution nodes are used to execute task instances. Among them, the status of the task process is judged through heartbeat monitoring to avoid the inability to complete task instances caused by the inactivation of the task execution node.

[0044] In some embodiments, after the task execution node is started, it communicates with the task management node through a preset network protocol, automatically registers with the task management node, and the task execution node transmits task status, processor resource information, and memory resource information to the task management node.

[0045] In some embodiments, for the derivative task, it is necessary to import data from a data source into a target source; the task execution node obtains the configuration information required for task execution according to the task configuration template corresponding to the derivative task; the task execution node inputs the configuration information as startup parameters into the task execution plugin, and uses the task execution plugin to execute the derivative task.

[0046] In some embodiments, data splicing is performed according to a task instance to obtain a script execution command; the task execution node starts a task execution script to execute a script command language through the task execution script, so as to implement the execution of script languages such as database query languages.

[0047] Optionally, an election is performed based on server nodes to obtain a task management node, including: obtaining the management weights corresponding to each server node respectively, and determining each server node as the current node; in response to the current node, determining whether the current node meets a first preset condition, a second preset condition, and a third preset condition, where the first preset condition includes that there is a task management node among the server nodes other than the current node, the second preset condition includes that the current election mode is a preemption module, and the third preset condition includes that the management weight is greater than or equal to a preset weight threshold; if the current node meets the first preset condition and the current node does not meet the second preset condition, then the current node is determined as an alternative node; if the current node meets the first preset condition, the second preset condition, and the third preset condition, then the current node is determined as the task management node; if the current node meets the first preset condition and the second preset condition, and the current node does not meet the third preset condition, then a task management node is determined from the current node and the current task management node according to the management weight; if the current node does not meet the first preset condition and the current node meets the third preset condition, then the current node is determined as the task management node; if the current node does not meet the first preset condition and the third preset condition, then the current node is determined as an alternative node, and a task management node is determined from the alternative nodes according to the management weight.

[0048] In some embodiments, only one task management node provides services externally at the same time; after the task management node becomes inactivated, a new task management node is re-elected from the alternative server nodes; multiple physical routing devices are virtualized into one virtual router.

[0049] In this way, the task management node is determined through the election strategy, and the entire election process is separated from the business layer, which is imperceptible to users. At the same time, the switching speed is fast, ensuring service continuity.

[0050] Combined Figure 3 As shown, the embodiments of the present disclosure provide a method for electing a task management node, including:

[0051] Step S301, obtaining the current node;

[0052] Step S302: Determine whether there is a task management node among the server nodes other than the current node. If so, jump to Step S303; if not, jump to Step S308;

[0053] Step S303: Determine whether the current election mode is the preemption module. If so, jump to Step S305; if not, jump to Step S304;

[0054] Step S304: Determine the current node as the alternative node.

[0055] Step S305: Determine whether the management weight of the current node is greater than or equal to the preset weight threshold. If so, jump to Step S306; if not, jump to Step S307;

[0056] Step S306: Determine the current node as the task management node.

[0057] Step S307: Determine a task management node from the current node and the current task management node according to the management weight.

[0058] Step S308: Determine whether the management weight of the current node is greater than or equal to the preset weight threshold. If so, jump to Step S306; if not, jump to Step S309;

[0059] Step S309: Determine the current node as the alternative node, and determine a task management node from the alternative nodes according to the management weight.

[0060] Optionally, the task management node generates corresponding task instances according to the task configuration information in the following manner: pre-set a task database and a task template in the task management node; determine the task requirements according to the task configuration information, and extract data from the task database according to the task requirements to obtain task-related data; fill in the task template according to the task configuration information and the task-related data to obtain the task instance.

[0061] In some embodiments, the number of task management nodes is one; the task management node is used to issue task instances according to different priorities, and allocate task execution nodes to preferentially execute core business tasks, wherein the task instances are allocated task resources according to the task type, so as to implement one task type corresponding to one task process in the task execution node.

[0062] Optionally, the task execution node executes the task instance in the following manner: pre-set an execution logic plugin in the task execution node; generate a plugin configuration file corresponding to the execution logic plugin according to the task instance, where the plugin configuration file carries a data source, a target source, and a structured query statement; start the execution logic plugin according to the plugin configuration file to execute the task instance, where the task instance is used to extract the data to be imported from the data source through the structured query statement and import the data to be imported into the target source.

[0063] In some embodiments, the number of task execution nodes is multiple; the task execution nodes are used to receive the task instances sent by the task management node and execute the task instances according to the resource allocation requirements of the task management node, and after the task instances are executed, feedback the task execution results to the task management node.

[0064] In some embodiments, the task logic plugin is deployed in the task execution node, and the task configuration information input by the user is received through the task configuration template of the user terminal; if the task execution node receives the corresponding task instance, the task logic plugin is started to extract the task parameters in the task instance through the task logic plugin, so as to complete the task, where the task logic plugin supports multiple task execution methods, and the plug-in of the task execution logic is realized through the task logic plugin, reducing the coupling degree between functions and adapting to multiple task scenarios.

[0065] In some embodiments, the templatized task parameters are extracted through the task logic plugin, supporting the user to customize functions to formulate task instances, realizing functions such as data import, realizing the templatization of task configuration information, uniformly managing task configuration information, and the user only needs to write logic code according to the task configuration template to meet the user's business needs and reduce the coding cost.

[0066] Optionally, the method further includes: the server side further includes a data query engine; if a data query request is obtained through the virtual address, the user permission corresponding to the data query request is obtained; the data query engine is used to execute the data query request to obtain the target data, and filter out the sensitive information that does not meet the user permission from the target data; provide a viewing function and / or a downloading function corresponding to the target data.

[0067] In some embodiments, the data query engine integrates multiple existing search engines to meet the interactive user query needs. At the same time, it provides a data download function and a data permission management function, and through the data permission management function, permission isolation and download restrictions can be performed on sensitive data.

[0068] Optionally, the method further includes: generating a task execution result corresponding to the task instance by using a task execution node, and sending the task execution result to a task management node; if the task execution result is a task failure, extracting the task responsible person and the exception notification rule from the task configuration information by using the task management node, so as to send the task execution result to the task responsible person according to the exception notification rule.

[0069] In some embodiments, if in a task exception situation, the task execution result is sent to the task responsible person according to the notification method, where the task exception situation includes task execution failure, task timeout, task retry failure, dependent task execution failure, etc., and the notification method includes communication software, email, short message, phone call, etc.

[0070] Combined with Figure 4 As shown, the present application provides a highly available task instance execution method, which is applied to a user terminal. The method includes:

[0071] Step S401, obtaining task configuration information;

[0072] Step S402, sending the task configuration information to the server side;

[0073] Among them, the server side includes a server node and a physical route. The server side is used to perform an election according to the server node to obtain a task management node, generate a virtual address according to the physical route, bind the virtual address to the task management node. If the task configuration information is received through the virtual address, the task management node is used to generate a corresponding task instance according to the task configuration information, and the task instance is sent to the task execution node through the task management node to trigger the task execution node to execute the received task instance. The task execution node is determined from server nodes other than the task management node.

[0074] Optionally, obtaining the task configuration information includes: obtaining a task configuration template, where the task configuration template includes at least one of a basic attribute field, an alarm attribute field, and an extended attribute field; receiving the task configuration information input by the user through the task configuration template, where the basic attribute field is used to carry at least one of a task type, a task execution time, a task execution period, a task dependency relationship, and a task responsible person, the alarm attribute field is used to carry an exception notification rule, and the extended attribute field is used to carry at least one of a data source, a target source, and a structured query statement.

[0075] In this way, receiving the task configuration information input by the user through the task configuration template realizes the templatization of task configuration, and the task configuration is simple, greatly saving the development cost and operation and maintenance cost.

[0076] In some embodiments, the basic attribute field is used to carry a task scheduling time, a task period, a task dependency relationship, a task responsible person, etc.

[0077] In some embodiments, the alarm attribute field is used to carry the notification method in case of task exceptions. Among them, task exceptions include task execution failure, task timeout, task retry failure, dependent task execution failure, etc., and the notification methods include communication software, email, SMS, phone call, etc.

[0078] In some embodiments, the extended attribute field is used to carry supplementary configurations related to the task type. For example, the database query language in the statistical task, the data source in the derivative task, the target source in the derivative task, etc.

[0079] In some embodiments, the basic unit of task scheduling is the task instance; the task configuration information is abstracted into a task configuration I template, and a visual task configuration policy is provided to the user through a web interface, which is simple to operate and has a low learning cost.

[0080] In some embodiments, the historical configuration data of the user is collected, and data is pre-extracted according to the historical input data; the extracted data is integrated to obtain a preset task; after the user inputs the configuration data again, there is no need to import data again, but the data reading step is skipped and the corresponding preset task is executed.

[0081] Combined Figure 5 As shown, the present application provides a highly available task instance execution method, including:

[0082] Step S501, the user terminal obtains a task configuration template;

[0083] Step S502, the user terminal receives the task configuration information input by the user through the task configuration template;

[0084] Step S503, the user terminal sends the task configuration information to the server side;

[0085] Step S504, the server side conducts an election according to the server nodes to obtain a task management node;

[0086] Step S505, the server side generates a virtual address according to the physical route and binds the virtual address to the task management node;

[0087] Step S506, the server side uses the task management node to generate a corresponding task instance according to the task configuration information;

[0088] Step S507, the server side sends the task instance to the task execution node through the task management node, triggering the task execution node to execute the received task instance.

[0089] By adopting the highly available task instance execution method provided by this application, a task management node is determined from the server nodes on the server side through election, and a virtual address is generated using the physical routing on the server side. The virtual address is bound to the task management node to receive task configuration information through the virtual address. The task management node generates corresponding task instances according to the task configuration information, and determines task execution nodes from the server nodes other than the task management node, and the received task instances are executed by the task execution nodes. In this way, not only is the election strategy for the task management node implemented using the virtual address, achieving high availability of task management, but also the task instances are executed by the task execution nodes, separating the responsibilities of the server nodes. Combining the election strategy and responsibility separation, task resources are reasonably allocated, and the occurrence probability of problems such as task resource waste and memory overflow is reduced, thereby improving the task execution efficiency.

[0090] In some embodiments, the user creates task configuration information through a visual web page; the server side creates task instances according to the task configuration information at the task time, and the creation time is the first day of the task cycle. For example, the creation time of a monthly task is 0:00 on the first day of each month, and the creation time of a weekly task is 0:00 on Sunday; at the task start time, the task management node communicates with the task execution nodes according to task priorities, task dependencies, etc., and selects target nodes with sufficient processor resources and memory resources from the task execution nodes in a polling manner, and distributes the task instances to the target nodes; while executing the task instances, the task execution nodes send process heartbeats, task execution status, etc. to the task management node.

[0091] Combined with Figure 6 As shown, this application provides a highly available task instance execution system, including a user terminal 601 and a server side 602.

[0092] The user terminal 601 is configured to obtain task configuration information and send the task configuration information to the server side.

[0093] The server side 602 includes server nodes and a physical routing.

[0094] The server side 602 is configured to conduct an election based on the server nodes to obtain a task management node, generate a virtual address according to the physical routing, bind the virtual address to the task management node. If task configuration information is received through the virtual address, the task management node is used to generate corresponding task instances according to the task configuration information, and the task instances are sent to the task execution nodes through the task management node, triggering the task execution nodes to execute the received task instances, and the task execution nodes are determined from the server nodes other than the task management node.

[0095] By using the highly available task instance execution system provided by this application, a task management node is determined from the server nodes on the server side through election, and a virtual address is generated using the physical routing on the server side and bound to the task management node, so as to receive task configuration information through the virtual address. The task management node is used to generate corresponding task instances according to the task configuration information, and a task execution node is determined from the server nodes other than the task management node, and the received task instances are executed through the task execution node. In this way, not only is the election strategy for the task management node implemented using the virtual address, achieving high availability of task management, but also the task instances are executed by the task execution node, separating the responsibilities of the server nodes, reasonably allocating task resources by combining the election strategy and responsibility separation, reducing the occurrence probability of problems such as task resource waste and memory overflow, and thus improving the task execution efficiency.

[0096] Combined with Figure 7 As shown, this application provides a system architecture for implementing a highly available task instance execution method, including a user terminal 601 and a server side 602.

[0097] The user terminal 601 is configured with a visualization web page.

[0098] The user terminal 601 obtains task configuration information through the visualization web page; and sends the task configuration information and / or a data query request to the server side 602.

[0099] The server side 602 includes a task scheduling module 6021, a task query module 6022, a physical routing 6023, and server nodes 6024. Among them, the server nodes include alternative nodes 60241 and task execution nodes 60242, and among the alternative nodes 6024, there is a task management node 60243.

[0100] The task scheduling module 6021 is configured with a task database.

[0101] The task scheduling module 6021 is used to extract task basic information from the task database according to the received task configuration information, and send the task configuration information and the task basic information to the task management node 60243.

[0102] The task query module 6022 is used to obtain the user permissions corresponding to the data query request; execute the data query request using the query engine to obtain the target data, and filter out sensitive information that does not meet the user permissions from the target data; and provide a viewing function and / or a downloading function corresponding to the target data to the user terminal 601.

[0103] The routing module 6023 is used to generate a virtual address according to the physical routing and bind the virtual address to the task management node 60243.

[0104] The task management node 60243 is determined from the alternative nodes 60241 through an election strategy.

[0105] The task management node 60243 is used to generate a task instance according to the task configuration information and the basic task information, and send the task instance to the task execution node 60242.

[0106] The task execution node 60242 is used to execute the received task instance.

[0107] The above description and the drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process, and other changes. The embodiments only represent possible variations. Unless explicitly required, the individual components and functions are optional, and the order of operations can vary. Parts and sub-samples of some embodiments can be included in or replace parts and sub-samples of other embodiments. Moreover, the terms used in this application are only for describing the embodiments and are not used to limit the claims. As used in the description of the embodiments and the claims, unless the context clearly indicates otherwise, the singular forms "a", "an", and "the" are intended to also include the plural forms. Similarly, as used in this application, the term "and / or" refers to any and all possible combinations including one or more of the associated listed items. Additionally, when used in this application, the term "comprise" and its variants "comprises" and / or "comprising" etc. mean the presence of the stated sub-samples, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other sub-samples, wholes, steps, operations, elements, components, and / or groupings of these. Without further limitation, an element defined by the statement "comprising one..." does not exclude the presence of additional identical elements in the process, method, or device comprising the element. In this document, what each embodiment focuses on can be the differences from other embodiments, and the same or similar parts among the embodiments can be referred to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, the relevant parts can refer to the description of the method part.

[0108] Those skilled in the art will realize that the units and algorithm steps of the examples described in connection with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software may depend on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods for each specific application to implement the described functions, but such implementation should not be considered to exceed the scope of this application. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0109] In the embodiments disclosed herein, the disclosed methods, products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units can be merely a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some sub-samples can be ignored or not executed. Additionally, the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of devices or units can be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to implement this embodiment. Additionally, in this application, the functional units can be integrated in one processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit.

[0110] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to the present application. In this regard, each block in the flowchart or block diagram may represent a module, a segment of code, or a portion thereof, which contains one or more executable instructions for implementing the specified logical function. In some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, or they may sometimes be executed in reverse order, depending on the functions involved. Each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A highly available task instance execution method, characterized in that, Applied to the server side, the method includes: The server side includes server nodes and a physical route; Elect according to the server nodes to obtain a task management node, generate a virtual address according to the physical route, and bind the virtual address to the task management node; If task configuration information is received through the virtual address, use the task management node to generate a corresponding task instance according to the task configuration information; Send the task instance to the task execution node through the task management node, triggering the task execution node to execute the received task instance, where the task execution node is determined from server nodes other than the task management node.

2. The method according to claim 1, characterized in that Elect according to the server nodes to obtain a task management node, including: Obtain the management weights corresponding to each of the server nodes, and determine each of the server nodes as the current node; In response to the current node, determine whether the current node meets the first preset condition, the second preset condition, and the third preset condition, where the first preset condition includes that there is a task management node among server nodes other than the current node, the second preset condition includes that the current election mode is the preemption module, and the third preset condition includes that the management weight is greater than or equal to the preset weight threshold; If the current node meets the first preset condition and the current node does not meet the second preset condition, determine the current node as an alternative node; If the current node meets the first preset condition, the second preset condition, and the third preset condition, determine the current node as the task management node; If the current node meets the first preset condition and the second preset condition, and the current node does not meet the third preset condition, determine a task management node from the current node and the current task management node according to the management weight; If the current node does not meet the first preset condition and the current node meets the third preset condition, determine the current node as the task management node; If the current node does not meet the first preset condition and the third preset condition, determine the current node as an alternative node, and determine a task management node from the alternative nodes according to the management weight.

3. The method according to claim 2, characterized in that, The method further includes at least one of the following: Perform heartbeat monitoring on the task management node, and if it is detected that the task management node is inactivated, reduce the management weight of the task management node and re-elect the task management node; Perform heartbeat monitoring on the task execution node, and if it is detected that the task execution node is inactivated, re-determine the task execution node corresponding to the task instance.

4. The method according to any one of claims 1 to 3, characterized in that, The task management node generates a corresponding task instance according to the task configuration information in the following manner: Pre-set a task database and a task template in the task management node; Determine the task requirements according to the task configuration information, and extract data from the task database according to the task requirements to obtain task-related data; Fill in the task template according to the task configuration information and the task-related data to obtain the task instance.

5. The method according to claim 4, characterized in that, The task execution node executes the task instance in the following manner: Set an execution logic plugin in the task execution node in advance; Generate a plugin configuration file corresponding to the execution logic plugin according to the task instance, where the plugin configuration file carries a data source, a target source, and a structured query statement; Start the execution logic plugin according to the plugin configuration file to execute the task instance, where the task instance is used to extract data to be imported from the data source through a structured query statement and import the data to be imported into the target source.

6. The method according to any one of claims 1 to 3, characterized in that, The method further includes: The server side further includes a data query engine; If a data query request is obtained through the virtual address, obtain the user permission corresponding to the data query request; Use the query engine to execute the data query request to obtain target data, and filter sensitive information that does not meet the user permission from the target data; Provide a viewing function and / or a downloading function corresponding to the target data.

7. The method according to any one of claims 1 to 3, characterized in that The method further includes: Use the task execution node to generate a task execution result corresponding to the task instance and send the task execution result to the task management node; If the task execution result is a task failure, use the task management node to extract the task responsible person and the exception notification rule from the task configuration information to send the task execution result to the task responsible person according to the exception notification rule.

8. A highly available task instance execution method, characterized in that, Applied to a user terminal, the method includes: Obtain task configuration information; Send the task configuration information to the server side, where the server side includes server nodes and a physical route. The server side is used to elect a task management node according to the server nodes, generate a virtual address according to the physical route, bind the virtual address to the task management node. If the task configuration information is received through the virtual address, use the task management node to generate a corresponding task instance according to the task configuration information, send the task instance to the task execution node through the task management node, and trigger the task execution node to execute the received task instance. The task execution node is determined from server nodes other than the task management node.

9. The method according to claim 8, wherein Obtaining task configuration information includes: Obtain a task configuration template, where the task configuration template includes at least one of a basic attribute field, an alarm attribute field, and an extended attribute field; Receive the task configuration information input by the user through the task configuration template, where the basic attribute field is used to carry at least one of a task type, a task execution time, a task execution cycle, a task dependency relationship, and a task responsible person, the alarm attribute field is used to carry an exception notification rule, and the extended attribute field is used to carry at least one of a data source, a target source, and a structured query statement.

10. A highly available task instance execution system, characterized in that, Includes: A user terminal configured to obtain task configuration information and send the task configuration information to the server side; The server side includes server nodes and a physical route. The server side is configured to conduct an election based on the server nodes to obtain a task management node, generate a virtual address according to the physical route, bind the virtual address to the task management node. If task configuration information is received through the virtual address, the task management node is used to generate a corresponding task instance according to the task configuration information, and the task instance is sent to a task execution node through the task management node to trigger the task execution node to execute the received task instance. The task execution node is determined from server nodes other than the task management node.

Images