Task scheduling method, electronic device and storage medium

Abstract

The present invention discloses a task scheduling method, an electronic device and a storage medium. The task scheduling method applied to nodes of a distributed system comprises: based on a first task to be scheduled, sending a first request to a service node in a first cluster; wherein any two service nodes in the first cluster can communicate; the first request is used to request to obtain first information corresponding to the first task; receiving the first information corresponding to the first task returned by the service node based on the first request; wherein the first information includes device information for executing the first task, all service nodes providing a first service related to the first task and task configuration information; when the device information of the node is the same as the device information in the first information, executing the first task based on the task configuration information by calling a first service provided by at least one service node.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a task scheduling method, electronic equipment and storage medium. Background Art

[0002] With the development of computer technology, more and more technologies, such as blockchain, big data, and distributed technologies, are being applied in the financial field. Traditional finance is gradually transforming into financial technology. However, due to the security and real-time requirements of the financial industry, financial technology also puts forward higher requirements for technology. In the field of financial technology, the distributed scheduled task scheduling framework is built based on the service registration center. When scheduling scheduled tasks, the application server in the distributed system searches for the IP address list corresponding to the microservice related to the scheduled task from the service registration center, and calls the corresponding microservice based on the found IP address list to execute the scheduled task.

[0003] When the number of scheduled tasks to be scheduled in a distributed system increases dramatically, and there are multiple levels of calls between microservices related to the scheduled tasks, the call response time of the microservice at the head or middle of the call chain may be too long, or the corresponding microservice may be unavailable. In this way, the application server will occupy more and more system resources when calling the microservice, which may cause the application server to crash. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a task scheduling method, an electronic device and a storage medium to solve the technical problem in the related art that the system may crash during the process of scheduling scheduled tasks in a distributed system.

[0005] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0006] An embodiment of the present invention provides a task scheduling method, which is applied to a node in a distributed system. The method includes:

[0007] Based on the first task to be scheduled, a first request is sent to a service node in a first cluster; wherein any two service nodes in the first cluster can communicate; and the first request is used to request to obtain first information corresponding to the first task;

[0008] Receiving first information corresponding to the first task returned by the service node based on the first request; wherein the first information includes device information for executing the first task, all service nodes providing a first service related to the first task, and task configuration information;

[0009] In a case where the device information of the node is the same as the device information in the first information, the first task is executed based on the task configuration information by invoking a first service provided by at least one service node.

[0010] In the above solution, the step of executing the first task based on the configuration information by calling the first service provided by at least one service node includes:

[0011] Sending a call request for a first service to a first service node;

[0012] If the first service node does not respond to the calling request within a first set time period, or the first service calling fails, the first task is executed based on the configuration information by calling a first service provided by a second service node.

[0013] In the above solution, after receiving the first information corresponding to the first task, the method further includes:

[0014] In case a setting failure is detected, caching first information corresponding to the unexecuted first task in a setting database;

[0015] In the case where the setting failure has been eliminated, first information corresponding to the cached first task is pulled from the setting database; wherein,

[0016] Setting failures include at least one of the following:

[0017] The node cannot be connected to the Internet;

[0018] The service node providing the first service is down;

[0019] The component that the first service depends on is in an abnormal state.

[0020] In the above scheme, the method further includes:

[0021] When the number of service call failures is greater than or equal to a first set threshold, or the first value is greater than or equal to a second set threshold, the circuit breaker mode is turned on and the external call function is disabled; wherein the first value represents the quotient of the number of service call failures and the total number of service calls;

[0022] Cache the first information corresponding to the unexecuted first task and the first task received after the circuit breaker mode is turned on;

[0023] When the duration of the circuit breaker mode is greater than or equal to the second set duration, the external call function is re-enabled, and when the first service is successfully called continuously, the circuit breaker mode is turned off.

[0024] In the above solution, the circuit breaker mode is turned on, including:

[0025] updating a count value of a first timer based on the number of service call failures, and resetting the count value of the first timer at a set time interval;

[0026] When the count value of the first timer is greater than or equal to the first set threshold within a set time interval, the circuit breaker mode is turned on.

[0027] The embodiment of the present invention also provides a task scheduling method, including:

[0028] A first node in a distributed system sends a first request to a service node in a first cluster based on a first task to be scheduled; any two service nodes in the first cluster can communicate; and the first request is used to request to obtain first information corresponding to the first task;

[0029] The service node that receives the first request determines first information corresponding to the first task based on the first request, and returns the first information to the first node; wherein the first information includes device information for executing the first task, all service nodes that provide first services related to the first task, and task configuration information;

[0030] In a case where the device information of the first node is the same as the device information in the first information, the first node executes the first task based on the task configuration information by invoking a first service provided by at least one service node.

[0031] An embodiment of the present invention further provides an electronic device, including:

[0032] A sending unit, configured to send a first request to a service node in a first cluster based on a first task to be scheduled; wherein any two service nodes in the first cluster can communicate; and the first request is used to request to obtain first information corresponding to the first task;

[0033] A receiving unit, configured to receive first information corresponding to the first task returned by the service node based on the first request; wherein the first information includes device information for executing the first task, all service nodes providing a first service related to the first task, and task configuration information;

[0034] A scheduling unit is used to execute the first task based on the task configuration information by calling a first service provided by at least one service node when the device information of the node is the same as the device information in the first information.

[0035] An embodiment of the present invention further provides an electronic device, comprising: a processor and a memory for storing a computer program that can be run on the processor,

[0036] Wherein, the processor is used to execute the steps of the above-mentioned task scheduling method when running the computer program.

[0037] An embodiment of the present invention further provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned task scheduling method are implemented.

[0038] In an embodiment of the present invention, when the device information of the node is the same as the device information in the first information, the first task is executed based on the task configuration information by calling the first service provided by at least one service node. The service node integrates the functions of the service provider and the registration center, and each service node in the first cluster stores the relevant information of all tasks and the services supported by each service node in the first cluster. Since each service node in the first cluster is a peer node, any two service nodes in the first cluster can communicate, and each node in the distributed system can communicate with any service node in the first cluster, and can also decide whether to stop communicating with the service node. When the response time of calling the first service provided by a service node is too long, any node in the distributed system can terminate the call of the first service provided by the service node in time, and re-call the first service provided by another service node to execute the first task. As long as there is an available service node in the first cluster that provides the first service, the execution of the first task can be supported, thereby reducing the occupation of system resources of nodes in the distributed system and reducing the probability of system crashes in nodes in the distributed system. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 A schematic diagram of a task scheduling system provided by an embodiment of the present invention;

[0040] Figure 2 An interactive diagram of a task scheduling method provided by an embodiment of the present invention;

[0041] Figure 3 A schematic diagram of the implementation flow of the task scheduling method provided in an embodiment of the present invention;

[0042] Figure 4 A schematic diagram of the implementation flow of a task scheduling method provided by an application embodiment of the present invention;

[0043] Figure 5 A schematic diagram of a task scheduling method provided by an application embodiment of the present invention;

[0044] Figure 6 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention;

[0045] Figure 7A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In the related technology, an open source distributed scheduling and timing framework, such as elastic-job, is used to schedule scheduled tasks. Elastic-job is built based on the distributed application coordination service zookeeper as the service registration center. Among them, Zookeeper is a cluster composed of a leader and multiple followers. The service provider registers the name of the supported microservice and the IP address of the service provider to the registration center. When the service consumer needs to call microservice A, it searches the registration center for the IP address list of the service provider that supports microservice A, selects a service provider from the IP address list found, and calls the microservice A of the service provider.

[0047] In the related art, it is impossible to control a single node in Zookeeper. When the number of scheduled tasks to be scheduled in a distributed system increases dramatically, and there are multi-level calls between microservices related to the scheduled tasks, the call response time corresponding to the microservice at the head or middle of the call chain may be too long, or the corresponding microservice may be unavailable. For example, the service node providing the corresponding microservice is disconnected from the network, downtime, or the basic component that the microservice depends on in the service node fails. The application server cannot terminate the microservice that takes too long to call and respond. Therefore, the application server's call to the microservice will occupy more and more system resources, which may cause the application server to crash.

[0048] Based on this, in various embodiments of the present invention, a node in a distributed system sends a first request to a service node in a first cluster based on a first task to be scheduled; wherein any two service nodes in the first cluster can communicate; the first request is used to request to obtain first information corresponding to the first task; the first information corresponding to the first task returned by the service node based on the first request is received; wherein the first information includes device information for executing the first task, all service nodes providing a first service related to the first task, and task configuration information; when the device information of the node matches the device information in the first information, the first task is executed based on the configuration information by calling the first service provided by at least one service node. In this solution, each service node in the first cluster is a peer node, which supports control of a single service node. The service node integrates the functions of the service provider and the registration center. Each service node in the first cluster stores relevant information of all tasks and the services supported by each service node in the first cluster. Therefore, when the response time of calling microservice A provided by a service node is too long, any node in the distributed system can promptly terminate the call to the microservice provided by the service node and re-call microservice A provided by another service node to execute the first task. As long as there is an available service node providing the first service in the first cluster, the execution of the first task can be provided, thereby reducing the occupation of system resources of nodes in the distributed system and reducing the probability of system crashes of nodes in the distributed system.

[0049] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0050] Figure 1 A schematic diagram of a task scheduling system provided in an embodiment of the present invention. The task scheduling system includes a distributed system 1 and a first cluster 2. The distributed system 1 includes a plurality of nodes 11 to 1n, and a setting application is installed in the plurality of nodes 11 to 1n, and the setting application is used to execute tasks of the distributed system; the first cluster 2 includes a plurality of service nodes 21 to 2n. The nodes in the distributed system may be application servers, and the service nodes may be Eureka service nodes.

[0051] Any two service nodes in the first cluster can communicate with each other, that is, the service nodes in the first cluster form a peer-to-peer network. Each service node integrates the functions of a service provider and a registration center. The service nodes store relevant information of all tasks and the services supported by each service node in the first cluster. Each node in the distributed system 1 is used to call the services or service instances supported by the service nodes in the first cluster and perform corresponding tasks.

[0052] It should be noted that each service node in the first cluster can support at least one service, and the nodes in the distributed system can subscribe to the services supported by the service nodes included in the first cluster. When the service node detects that any supported service is unavailable for a short period of time, it sets the unavailable service to an offline (down) state, and publishes the state of the unavailable service to the nodes in the distributed system that have subscribed to the unavailable service, so that the relevant nodes can update the working state of the unavailable service to an offline state locally, thereby avoiding the relevant nodes from calling the unavailable service in the service node.

[0053] It should be noted that the nodes in the distributed system detect whether the subscribed service is available based on the heartbeat mechanism. For example, the nodes in the distributed system send heartbeat detection data to the service node corresponding to the subscribed service at a set time interval. When a node in the distributed system receives the heartbeat data replied by the service node within the set time length, it indicates that the service supported by the corresponding service node is valid; when a node in the distributed system does not receive the heartbeat data replied by the service node within the set time length, it indicates that the service supported by the corresponding service node is invalid, and the node in the distributed system can mark the corresponding service as invalid and disconnect the communication connection with the corresponding service node. When the service node in the first cluster does not receive the heartbeat detection data sent by the node in the distributed system within the set time length, it indicates that the corresponding node in the distributed system is invalid, the service node disconnects the communication connection with the node, and deletes the invalid node from the list of nodes that have subscribed to the corresponding service.

[0054] Considering that in actual applications, the distributed system has high requirements for the timeliness of the service node's response to the called service, the set duration can be 30 seconds, and of course it can also be set to a smaller value, which can be set according to the actual situation. In some embodiments, the renewal threshold corresponding to the service supported by the service node can also be reduced to avoid too many nodes in the distributed system calling the same service and reduce the probability of service call failure.

[0055] The implementation process of the task scheduling method is described in detail below in conjunction with the embodiments and drawings.

[0056] Figure 2 The interactive diagram of the task scheduling method provided by the embodiment of the present invention. Figure 2 As shown, the task scheduling method includes:

[0057] Step 201: A first node in a distributed system sends a first request to a service node in a first cluster based on a first task to be scheduled; any two service nodes in the first cluster can communicate; and the first request is used to request first information corresponding to the first task.

[0058] Here, when the first node in the distributed system detects the first task to be scheduled, it sends a first request to at least one service node in the first cluster to request to obtain the first information corresponding to the first task. The first request carries the identifier of the first task, and the identifier of the first task is used for the service node to find the first information corresponding to the first task. The first information of the first task includes the device identifier that executes the first task, the configuration information of the first task, and the node information of the service node that provides the first service related to the first task, such as the IP address of the service node.

[0059] It should be noted that the first task refers to any task. The first node may send a first request to any service node in the first cluster, or may send the first request to at least two service nodes in the first cluster respectively, or, after sending the first request to the first service node in the first cluster, if the first information corresponding to the first request is not received from the first service node within a first set time period, the first request may be sent to the second service node in the first cluster. Both the first service node and the second service node are any service nodes in the first cluster.

[0060] It should be noted that the first task may be issued by other devices or pre-stored in the first node in the distributed system. For example, when the first task is a scheduled task, the first task may be pre-set or configured and stored in the first node.

[0061] Step 202: The service node that receives the first request determines the first information corresponding to the first task based on the first request, and returns the first information to the first node; wherein the first information includes the device information that executes the first task, all service nodes that provide the first service related to the first task, and task configuration information.

[0062] Here, when receiving the first request sent by the first node in the distributed system, the service node in the first cluster determines the first information corresponding to the first task from the stored task information based on the identifier of the first task carried in the first request, and returns the first information corresponding to the first task to the first node that sent the first request. The task identifier and the related information of the task are stored in association in the service node.

[0063] In actual applications, users can register corresponding tasks with the service nodes in the first cluster through the setting interaction interface of the terminal device, set task configuration information, and subscribe to services related to the tasks. The service node associates and stores the identifier of the registered task with the relevant information of the task, and synchronizes the identifier of the registered task and the corresponding relevant information to other service nodes in the first cluster.

[0064] The first information may include the IP address of the service node providing the first service, so that the first node receiving the first information, when confirming that the device information of the first node is the same as the device information in the first information, accesses the corresponding service node based on the IP address of the service node and calls the first service to perform the first task. The task configuration information in the first information may include task parameters and execution conditions, etc., which are specifically configured according to actual conditions. The first service generally refers to any service required to perform the first task. In actual application, the first service may be a microservice.

[0065] It should be noted that each service node in the first cluster is a peer node, and the working mode of the first cluster is a multi-read multi-write mode, that is, it supports multiple service nodes to perform write operations in parallel and supports multiple service nodes to perform read operations in parallel, and the service nodes in the first cluster can synchronize data at set time intervals. In other words, at different times, the task information stored in different service nodes in the first cluster and the services (or service instances) supported by each service node may be different, but it does not affect the nodes in the distributed system calling the corresponding services to perform the first task, which ensures the availability and fault tolerance of the service.

[0066] In some embodiments, the method further comprises:

[0067] The service node disables the self-protection mode when it is successfully started.

[0068] Among them, the service node can be a Eureka service node.

[0069] Considering that the service node supports the self-protection mode, the service node enters the self-protection mode after a network failure occurs in the service node or the service node loses communication connection with multiple nodes in the distributed system in a short period of time. After the service node enters the self-protection mode, it may cause the nodes in the distributed system to request to call the service or service instance that does not exist or is invalid in the service node, or to retry continuously when the call times out. This does not meet the high timeliness requirements of the distributed system. Therefore, the self-protection mode of the service node needs to be turned off here to avoid the above situation and reduce the resource consumption of nodes in the distributed system.

[0070] It should be noted that after entering the self-protection mode, the service node protects the information in the service registry and no longer deletes the data in the service registry. When the network failure is restored, the service node that has entered the self-protection mode automatically exits the self-protection mode.

[0071] In order to improve the high availability of the first cluster and the timeliness of the called service, in some embodiments, the method further includes:

[0072] The service node performs a corresponding setting operation through a thread of a setting type; wherein each setting operation corresponds to a thread; and the setting operation includes at least one of the following:

[0073] Heartbeat detection, used to detect whether the service is available;

[0074] Service synchronization, used to synchronize services supported by service nodes in the first cluster;

[0075] State synchronization is used to synchronize the working status of services supported by service nodes.

[0076] Here, the service node configures corresponding threads for different types of setting operations, so that different threads are used to execute different setting operations, which can avoid congestion caused by excessive resource competition. Each setting operation corresponds to a thread of a setting type, and different types of setting operations correspond to different threads.

[0077] In actual application, the service node can perform heartbeat detection through the heartbeat detection thread; for example, the first heartbeat data sent by the node in the distributed system is received through the heartbeat detection thread, and the second heartbeat data is replied based on the received first heartbeat data, and the time interval between every two received first heartbeat data is determined.

[0078] The service node can perform service synchronization operations through the service change notification thread. For example, when the service supported by the service node changes, the changed service is synchronized to other service nodes in the first cluster through the service change notification thread, thereby synchronizing the available services and invalid services included in the services currently supported by the service node to other service nodes in the first cluster, and the service change includes the invalidation of the supported service.

[0079] The service node can perform state synchronization operations through the state change notification thread. For example, when the working state of the service supported by the service node changes, the working state of the service supported by the service node is synchronized to the nodes subscribed to the corresponding service in the distributed system and to other service nodes in the first cluster through the state change notification thread. The working state change includes the supported service changing from an online state to an offline state.

[0080] It should be noted that the above setting operations are merely examples, and different setting operations may be configured according to actual conditions.

[0081] Step 203: When the device information of the first node is the same as the device information in the first information, the first node executes the first task based on the configuration information by invoking a first service provided by at least one service node.

[0082] Here, the first node in the distributed system, upon receiving the first information returned by the service node in the first cluster, determines whether the device information of the first node is the same as the device information in the first information. When the device information of the first node is the same as the device information in the first information, the first node is characterized as the execution device of the first task, and the first task is executed by the first node; the first node determines all service nodes providing the first service based on the first information, selects any one service node from all determined service nodes, and accesses the service node, thereby calling the first service provided by the service node; when the first service is successfully called, the first task is executed through the first service based on the task configuration information contained in the first information; when the first service fails to be called, the first service provided by the service node is stopped from being called, and the first service provided by another service node is called from all service nodes determined based on the first information, so as to execute the first task through the first service based on the task configuration information contained in the first information.

[0083] In actual applications, the first node may also determine one service node from all determined service nodes based on the load balancing principle, and call the first service provided by the service node, so as to execute the first task through the first service.

[0084] It should be noted that, when the device information of the first node is different from the device information in the first information, it indicates that the first node is not an execution device of the first task, and the first node does not need to execute the first task.

[0085] In the embodiment of the present invention, since each service node in the first cluster is a peer node, when any node in the distributed system calls a service supported by any service node in the first cluster, the node in the distributed system will promptly terminate the call to the first service provided by the service node and re-call the first service provided by other service nodes when detecting that the call time is too long or the call fails. Therefore, as long as there is an available service node providing the first service in the first cluster, the execution of the first task can be supported, which can not only ensure the availability of the service, but also improve the fault tolerance of the service to the first cluster. At the same time, it can also reduce the resource occupation of the nodes in the distributed system and reduce the probability of system crashes of the nodes in the distributed system.

[0086] Figure 3 The following is a schematic diagram of the implementation process of the task scheduling method provided by the embodiment of the present invention, wherein the execution subject of the process is any node in the distributed system, and the node can be an electronic device such as a terminal or a server. Figure 3 As shown, the task scheduling method includes:

[0087] Step 301: Based on a first task to be scheduled, a first request is sent to a service node in a first cluster; any two service nodes in the first cluster can communicate; and the first request is used to request first information corresponding to the first task.

[0088] Among them, step 301 is the same as step 201. Please refer to the relevant description of step 201 for the implementation process, which will not be repeated here.

[0089] Step 302: Receive first information corresponding to the first task returned by the service node based on the first request; wherein the first information includes device information for executing the first task, all service nodes providing first services related to the first task, and task configuration information.

[0090] Here, the first information may include an IP address of a service node that provides a first service related to the first task.

[0091] Step 303: When the device information of the node is the same as the device information in the first information, the first task is executed based on the task configuration information by calling a first service provided by at least one service node.

[0092] Among them, step 303 is the same as step 203. Please refer to the relevant description of step 203 for the implementation process, which will not be repeated here.

[0093] In an embodiment of the present invention, when the device information of the node is the same as the device information in the first information, the first task is executed based on the task configuration information by calling the first service provided by at least one service node. The service node integrates the functions of the service provider and the registration center, and each service node in the first cluster stores the relevant information of all tasks and the services supported by each service node in the first cluster. Since each service node in the first cluster is a peer node, any two service nodes in the first cluster can communicate, and each node in the distributed system can communicate with any service node in the first cluster, and can also decide whether to stop communicating with the service node. When the response time of calling the first service provided by a service node is too long, any node in the distributed system can stop calling the first service provided by the service node, and re-call the first service provided by another service node to execute the first task. As long as there is an available service node in the first cluster that provides the first service, the execution of the first task can be supported, thereby reducing the occupation of system resources of nodes in the distributed system and reducing the probability of system crashes in nodes in the distributed system.

[0094] Considering that in actual applications, there may be a situation where the first service call fails or the response time is too long, in order to avoid waiting too long for obtaining the first service and improve the timeliness of the service call, in some embodiments, the first task is performed based on the configuration information by calling the first service provided by at least one service node, including:

[0095] Sending a call request for a first service to a first service node;

[0096] If the first service node does not respond to the calling request within a first set time period, or the first service calling fails, the first task is executed based on the configuration information by calling a first service provided by a second service node.

[0097] Here, the first service node and the second service node are any service nodes among all service nodes provided by the first service and included in the first information.

[0098] When a node in the distributed system confirms that the device information of the node is the same as the device information in the first information, the node determines the first service node from all service nodes providing the first service based on the information of all service nodes included in the first information, and sends a call request for the first service to the first service node to request to call the first service.

[0099] After sending the call request to the first service node, the node in the distributed system detects whether the first service node responds to the call request. If the first service node does not respond to the call request within the first set time, or the first service call fails, the node terminates the call of the first service provided by the first service node and calls the first service provided by the second service node. If the first service provided by the second service node is successfully called, the first task is executed based on the configuration information of the first task through the first service provided by the second service node. For example, when the configuration information includes task parameters and execution conditions, the first task is executed according to the task parameters in the configuration information when the execution conditions included in the configuration information are currently met. If the first service fails to call again, another second service node is re-determined from all service nodes providing the first service, and the first service provided by the re-determined second service node is called according to the above method.

[0100] It should be noted that, when the prompt information returned by the service node based on the call request is received, it indicates that the first service call has failed. The prompt information may be an error code.

[0101] Considering that the service node may fail to call the first service due to network failure, downtime, or failure of a basic component on which the first service depends, in order to enhance the stability and fault tolerance of the distributed system, in some embodiments, after receiving the first information corresponding to the first task, the method further includes:

[0102] In case a setting failure is detected, caching first information corresponding to the unexecuted first task in a setting database;

[0103] In the case where the setting failure has been eliminated, first information corresponding to the cached first task is pulled from the setting database; wherein,

[0104] Setting failures include at least one of the following:

[0105] The node cannot be connected to the Internet;

[0106] The service node providing the first service is down;

[0107] The component that the first service depends on is in an abnormal state.

[0108] Here, after receiving the first information corresponding to the first task, it is detected whether there is a setting fault. If there is a setting fault, the first information corresponding to the first task that has not been successfully executed is cached to the setting database to prevent data loss; if it is detected that the corresponding setting fault has been eliminated, the first information corresponding to the cached first task is pulled from the setting database, so that when the device information of the node is the same as the device information in the pulled first information, the first task is executed based on the task configuration information contained in the first information by calling the first service provided by at least one service node.

[0109] In actual application, nodes in the distributed system can determine whether there is a setting failure based on the prompt information fed back by the service node. For example, when the received prompt information is the first error code, it indicates that the service node providing the first service is down; when the received prompt information is the second error code, it indicates that the component that the first service depends on is in an abnormal state. The abnormal state can be an abnormality caused by occasional network jitter, or it can be a failure of the basic component that the first service depends on.

[0110] It should be noted that, in actual applications, when the first task fails to execute or times out, it indicates that a fault currently exists. In some embodiments, when the first task fails to execute or times out, the first information corresponding to the first task is cached.

[0111] In some embodiments, when a setting failure is detected, the setting data may be cached to back up the setting data to avoid loss of important data.

[0112] In order to avoid wasting resources by continuously initiating a call request for the first service after the first service is called multiple times and no response is obtained from the service node, and to improve the stability and disaster tolerance and fault tolerance capabilities of the distributed system, in some embodiments, during the process of calling the first service, the method further includes:

[0113] When the number of service call failures is greater than or equal to a first set threshold, or the first value is greater than or equal to a second set threshold, the circuit breaker mode is turned on and the external call function is disabled; wherein the first value represents the quotient of the number of service call failures and the total number of service calls;

[0114] Cache the first information corresponding to the unexecuted first task and the first task received after the circuit breaker mode is turned on;

[0115] When the working time of the circuit breaker mode is greater than or equal to the second set time, the external call function is re-enabled, and when the first service is successfully called continuously, the circuit breaker mode is turned off.

[0116] Here, in the process of calling the service supported by the service node to perform the corresponding task, the nodes in the distributed system count the number of service call failures and the total number of service calls. When the number of service call failures is greater than or equal to the first set threshold, or the quotient of the number of service call failures and the total number of service calls is greater than or equal to the second set threshold, it indicates that one or more service nodes in the first cluster have an abnormal condition, and the service node may have crashed. At this time, the circuit breaker mode is turned on, and the external call function is prohibited, thereby refusing the nodes in the distributed system to initiate external call requests. Therefore, turning on the circuit breaker mode can effectively prevent the nodes in the distributed system from constantly retrying to call the faulty service, thereby effectively preventing resource waste, and ensuring the timeliness of the service response by rejecting external call requests instead of waiting for the service response to time out.

[0117] When the circuit breaker mode is turned on, the first information corresponding to the unexecuted first task and the first task received after the circuit breaker mode is turned on are cached; when the duration of the circuit breaker mode is greater than or equal to the second set duration, the external call function is re-enabled to re-call the corresponding first service based on the cached first information corresponding to the first task, so as to execute the first task through the first service; when the first service is called successfully continuously, the circuit breaker mode is turned off. When the first service is not called successfully continuously, the circuit breaker mode is kept on and the duration of the circuit breaker mode is reset.

[0118] In actual application, the circuit breaker mode can be turned on by turning on a circuit breaker set in a node of the distributed system, and the circuit breaker is implemented by software; the circuit breaker mode can also be turned on by calling a setting component, and the setting component has a circuit breaker function.

[0119] In actual application, the circuit breaker mode has three states: open state, half-open state and closed state. Under normal circumstances, the circuit breaker mode is closed, and when the above situation occurs, the circuit breaker mode is turned on; when the circuit breaker mode is turned on, the external call function is disabled, and the nodes in the distributed system cache the first information corresponding to the unexecuted first task and the first task that failed to execute; after the second set time, it switches from the open state to the half-open state, re-enables the external call function, and re-calls the first service. If the first service is successfully called at least twice in a row, it switches from the half-open state to the closed state; otherwise, it switches from the half-open state to the open state, and the second set time is re-timed.

[0120] Considering that a sharp increase in the number of service call failures in a short period of time may cause a system crash of a node in a distributed system, in order to more effectively prevent the system crash, in some embodiments, the circuit breaker mode is turned on, including:

[0121] updating a count value of a first timer based on the number of service call failures, and resetting the count value of the first timer at a set time interval;

[0122] When the count value of the first timer is greater than or equal to the first set threshold within a set time interval, the circuit breaker mode is turned on.

[0123] Here, each time a service call failure is detected, the count value of the first timer is incremented by 1, and the count value of the first timer is reset at a set time interval; when the count value of the first timer is greater than or equal to the first set threshold within the set time interval, it indicates that the number of service call failures in a short period of time has increased dramatically, and most or all of the service nodes in the first cluster may fail. At this time, the circuit breaker mode is turned on.

[0124] Figure 4 A schematic diagram of the implementation flow of the task scheduling method provided by the application embodiment of the present invention is shown in FIG. Figure 4 As shown, the task scheduling method includes:

[0125] Step 401: Based on a first task to be scheduled, a first request is sent to a service node in a first cluster; any two service nodes in the first cluster can communicate; and the first request is used to request first information corresponding to the first task.

[0126] Step 402: Receive first information corresponding to the first task returned by the service node based on the first request; wherein the first information includes device information for executing the first task, all service nodes providing first services related to the first task, and task configuration information.

[0127] Step 403: When the device information of the node is the same as the device information in the first information, a call request for the first service is sent to the first service node.

[0128] The first service node is any service node among all service nodes indicated by the first information.

[0129] Step 404: When the first service node does not respond to the calling request within a first set time period, or the first service calling fails, the first task is executed based on the configuration information by calling a first service provided by a second service node.

[0130] The second service node is any service node among all service nodes indicated by the first information, and the second service node is different from the first service node.

[0131] Step 405: Update the count value of the first timer based on the number of service call failures, and reset the count value of the first timer at a set time interval.

[0132] Step 406: When the count value of the first timer is greater than or equal to the first set threshold within a set time interval, the circuit breaker mode is turned on and the external call function is disabled.

[0133] Step 407: Cache the first information corresponding to the unexecuted first task and the first task received after the circuit breaker mode is turned on.

[0134] Step 408: When the duration of the circuit breaker mode is greater than or equal to the second set duration, the external call function is re-enabled, and when the first service is successfully called continuously, the circuit breaker mode is turned off.

[0135] The following example illustrates the implementation process of the task scheduling method:

[0136] like Figure 5As shown, at least one application for executing distributed tasks is installed in node A in the distributed system, and a circuit breaker implemented by software is provided in node A, and node A can receive user requests; the first service node and the second service node can both receive service registration requests sent through a set application programming interface (API, Application Programming Interface), and the service registration request is used to register the corresponding service or register the distributed task to the registration center of the service node. The set application programming interface can be a representational state transfer (REST, Representational State Transfer) API. The first service node and the second service node synchronize the relevant information of their corresponding registered services and distributed tasks with each other.

[0137] When node A detects the first task to be scheduled and confirms that both the first service node and the second service node can provide the first service required to execute the first task, node A sends a first call request to the first service node to request to call the first service provided by the first service node; when node A confirms that the first service node cannot respond to the first call request, it indicates that the first service call has failed, stops calling the first service provided by the first service node, and sends a second call request to the second service node to request to call the first service provided by the second service node. When the first service provided by the second service node is successfully called, the first task is executed through the first service. The first task is a distributed task.

[0138] Node A counts the number of service call failures, updates the count value of the first timer based on the number of service call failures, and resets the count value of the first timer at a set time interval; when the count value of the first timer within the set time interval is greater than or equal to a first set threshold, the circuit breaker in node A is turned on, and the external call function is disabled, and the relevant information of all unexecuted first tasks and the first tasks received after the circuit breaker is turned on are cached to a database; when the duration of the circuit breaker mode is greater than or equal to a second set duration, the external call function is re-enabled, the cached data is read from the database to resume calling the first service to execute the first task, and when the first service is called successfully continuously, the circuit breaker is turned off.

[0139] It should be noted that, in the case where node A can execute other distributed tasks in addition to the first task, node A can also cache relevant information of other tasks to the database after opening the circuit breaker.

[0140] Nodes in a distributed system

[0141] After re-enabling the external call function, the circuit breaker mode is turned off by counting the number of consecutive successful calls to the first service.

[0142] It should be noted that, in extreme cases, all calls to the first service may fail. In this case, the first information corresponding to the first task may be cached, and the first service may be called again after a period of time.

[0143] It should be noted that when a network failure or downtime occurs at a service node or a basic component on which the first service depends fails, the first service call will fail. The first service may be a microservice.

[0144] To implement the method of the embodiment of the present invention, the embodiment of the present invention also provides an electronic device, such as Figure 6 As shown, the electronic device includes:

[0145] A sending unit 61 is configured to send a first request to a service node in a first cluster based on a first task to be scheduled; wherein any two service nodes in the first cluster can communicate; and the first request is used to request to obtain first information corresponding to the first task;

[0146] A receiving unit 62 is configured to receive first information corresponding to the first task returned by the service node based on the first request; wherein the first information includes device information for executing the first task, all service nodes providing a first service related to the first task, and task configuration information;

[0147] The scheduling unit 63 is configured to execute the first task based on the task configuration information by calling a first service provided by at least one service node when the device information of the node is the same as the device information in the first information.

[0148] In some embodiments,

[0149] The sending unit 61 is further used to: send a call request for the first service to the first service node;

[0150] The scheduling unit 63 is specifically used to: when the first service node does not respond to the calling request within a first set time period, or the first service calling fails, execute the first task based on the configuration information by calling the first service provided by the second service node.

[0151] In some embodiments, the electronic device further comprises:

[0152] A first cache unit, configured to cache first information corresponding to an unexecuted first task to a setting database when a setting failure is detected;

[0153] A reading unit is used to pull the first information corresponding to the cached first task from the setting database when the setting fault has been eliminated; wherein,

[0154] Setting failures include at least one of the following:

[0155] The node cannot be connected to the Internet;

[0156] The service node providing the first service is down;

[0157] The component that the first service depends on is in an abnormal state.

[0158] In actual application, XX unit-XX unit can be implemented by a processor in XX device, such as a central processing unit (CPU), a digital signal processor (DSP), a microcontroller unit (MCU) or a field-programmable gate array (FPGA). Of course, the processor needs to run the program stored in the memory to implement the functions of the above program modules.

[0159] In some embodiments, the electronic device further comprises:

[0160] A first processing unit, configured to enable a circuit breaker mode and disable an external call function when the number of service call failures is greater than or equal to a first set threshold, or when the first value is greater than or equal to a second set threshold; wherein the first value represents the quotient of the number of service call failures and the total number of service calls;

[0161] A second cache unit, used for caching first information corresponding to an unexecuted first task and a first task received after the circuit breaker mode is turned on;

[0162] The second processing unit is used to re-enable the external call function when the duration of the circuit breaker mode is greater than or equal to the second set duration, and to close the circuit breaker mode when the first service is called successfully continuously.

[0163] In some embodiments, the first processing unit is specifically configured to:

[0164] updating a count value of a first timer based on the number of service call failures, and resetting the count value of the first timer at a set time interval;

[0165] When the count value of the first timer is greater than or equal to the first set threshold within a set time interval, the circuit breaker mode is turned on.

[0166] In actual application, the above-mentioned units can be implemented by a processor in an electronic device, such as a central processing unit (CPU), a digital signal processor (DSP), a microcontroller unit (MCU) or a programmable gate array (FPGA), or can be implemented jointly by a processor and a communication interface in an electronic device.

[0167] It should be noted that: when the electronic device provided in the above embodiment performs task scheduling, only the division of the above program modules is used as an example. In actual applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device is divided into different program modules to complete all or part of the processing described above. In addition, the electronic device provided in the above embodiment and the task scheduling method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0168] Based on the hardware implementation of the above program modules and in order to implement the method of the embodiment of the present invention, the embodiment of the present invention further provides an electronic device. Figure 7 FIG. 1 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Figure 7 As shown, the electronic device 7 includes:

[0169] Communication interface 71, capable of exchanging information with other devices such as network devices;

[0170] The processor 72 is connected to the communication interface 71 to implement information exchange with other devices and is used to execute the task scheduling method provided by one or more technical solutions applied to nodes in a distributed system when running a computer program. The computer program is stored in the memory 73.

[0171] Of course, in actual application, the various components in the electronic device 7 are coupled together through the bus system 74. It can be understood that the bus system 74 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 74 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, Figure 7 Various buses are labeled as bus system 74 .

[0172] The memory 73 in the embodiment of the present invention is used to store various types of data to support the operation of the electronic device 7. Examples of such data include: any computer program used to operate on the electronic device 7.

[0173] It can be understood that the memory 73 can be a volatile memory or a non-volatile memory, and can also include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a disk memory or a tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory 73 described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0174] The method disclosed in the above embodiment of the present invention can be applied to the processor 72, or implemented by the processor 72. The processor 72 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 72 or the instruction in the form of software. The above processor 72 can be a general-purpose processor, a DSP, or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 72 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiment of the present invention. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor can be combined to execute. The software module can be located in a storage medium, which is located in the memory 73. The processor 72 reads the program in the memory 73 and completes the steps of the above method in combination with its hardware.

[0175] Optionally, when the processor 72 executes the program, it implements the corresponding processes implemented by the terminal in each method of the embodiment of the present invention, which will not be described here for the sake of brevity.

[0176] In an exemplary embodiment, the present invention also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, for example, including a first memory 73 storing a computer program, and the computer program can be executed by a processor 72 of the terminal to complete the steps of the aforementioned method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface storage, optical disk, or CD-ROM.

[0177] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0178] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0179] In addition, all functional units in the embodiments of the present invention may be integrated into one processing module, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0180] A person skilled in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, etc. Various media that can store program codes.

[0181] It should be noted that the technical solutions described in the embodiments of the present invention can be arbitrarily combined without conflict.

[0182] It should be noted that the term "and / or" in the embodiments of the present invention is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent the three situations that A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" in this article represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0183] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A task scheduling method, characterized in that: Applied to a node in a distributed system, the method comprises: Based on the first task to be scheduled, a first request is sent to a service node in a first cluster; wherein any two service nodes in the first cluster can communicate; and the first request is used to request to obtain first information corresponding to the first task; Receiving first information corresponding to the first task returned by the service node based on the first request; wherein the first information includes device information for executing the first task, all service nodes providing a first service related to the first task, and task configuration information; When the device information of the node is the same as the device information in the first information, executing the first task based on the task configuration information by calling a first service provided by at least one service node; The service node integrates the functions of a service provider and a registration center, and each service node in the first cluster stores relevant information of all tasks and services supported by each service node in the first cluster; When any node in the distributed system fails to respond within a first set time period after calling a first service provided by a service node, it stops calling the first service provided by the service node and calls the first service provided by another service node again to perform the first task.

2. The method according to claim 1, characterized in that The performing the first task based on the configuration information by invoking the first service provided by at least one service node includes: Sending a call request for a first service to a first service node; When the first service call fails, the first task is executed based on the configuration information by calling the first service provided by the second service node.

3. The method according to claim 1 or 2, characterized in that: After receiving the first information corresponding to the first task, the method further includes: In case a setting failure is detected, caching first information corresponding to the unexecuted first task in a setting database; In the case where the setting failure has been eliminated, first information corresponding to the cached first task is pulled from the setting database; wherein, Setting failures include at least one of the following: The node cannot be connected to the Internet; The service node providing the first service is down; The component that the first service depends on is in an abnormal state.

4. The method according to claim 1 or 2, characterized in that: The method further comprises: When the number of service call failures is greater than or equal to a first set threshold, or the first value is greater than or equal to a second set threshold, the circuit breaker mode is turned on and the external call function is disabled; wherein the first value represents the quotient of the number of service call failures and the total number of service calls; Cache the first information corresponding to the unexecuted first task and the first task received after the circuit breaker mode is turned on; When the duration of the circuit breaker mode is greater than or equal to the second set duration, the external call function is re-enabled, and when the first service is successfully called continuously, the circuit breaker mode is turned off.

5. The method according to claim 4, characterized in that The circuit breaker mode is turned on, including: updating a count value of a first timer based on the number of service call failures, and resetting the count value of the first timer at a set time interval; When the count value of the first timer is greater than or equal to the first set threshold within a set time interval, the circuit breaker mode is turned on.

6. A task scheduling method, characterized in that: include: A first node in a distributed system sends a first request to a service node in a first cluster based on a first task to be scheduled; any two service nodes in the first cluster can communicate; and the first request is used to request to obtain first information corresponding to the first task; The service node that receives the first request determines first information corresponding to the first task based on the first request, and returns the first information to the first node; wherein the first information includes device information for executing the first task, all service nodes that provide first services related to the first task, and task configuration information; In a case where the device information of the first node is the same as the device information in the first information, the first node performs the first task based on the task configuration information by invoking a first service provided by at least one service node; The service node integrates the functions of a service provider and a registration center, and each service node in the first cluster stores relevant information of all tasks and services supported by each service node in the first cluster; When any node in the distributed system fails to respond within a first set time period after calling a first service provided by a service node, it stops calling the first service provided by the service node and calls the first service provided by another service node again to perform the first task.

7. The method according to claim 6, characterized in that The method further comprises: The service node disables the self-protection mode when it is successfully started.

8. The method according to claim 6, characterized in that The method further comprises: The service node performs a corresponding setting operation through a thread of a setting type; wherein each setting operation corresponds to a thread; and the setting operation includes at least one of the following: Heartbeat detection, used to detect whether the service is available; Service synchronization, used to synchronize services supported by service nodes in the first cluster; State synchronization is used to synchronize the working status of services supported by service nodes.

9. An electronic device, characterized in that: include: A sending unit, configured to send a first request to a service node in a first cluster based on a first task to be scheduled; wherein any two service nodes in the first cluster can communicate; and the first request is used to request to obtain first information corresponding to the first task; A receiving unit, configured to receive first information corresponding to the first task returned by the service node based on the first request; wherein the first information includes device information for executing the first task, all service nodes providing a first service related to the first task, and task configuration information; a scheduling unit, configured to execute the first task based on the task configuration information by calling a first service provided by at least one service node when the device information of the node is the same as the device information in the first information; The service node integrates the functions of a service provider and a registration center, and each service node in the first cluster stores relevant information of all tasks and services supported by each service node in the first cluster; When any node in the distributed system fails to respond within a first set time period after calling a first service provided by a service node, it stops calling the first service provided by the service node and calls the first service provided by another service node again to perform the first task.

10. An electronic device, characterized in that: include: A processor and a memory for storing a computer program that can be run on the processor, wherein the processor executes the steps of the task scheduling method according to any one of claims 1 to 5 when running the computer program.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the task scheduling method according to any one of claims 1 to 5 are implemented.

Images