Cross-network distributed task scheduling method and system

Abstract

The present invention provides a cross-network distributed task scheduling method and system. First, a key generated when creating an application and a communication address of a task scheduling system in an intranet cluster are sent to each task node corresponding to the application, and then a long connection between the task node and the task scheduling system is established based on the key and the communication address. Then, the task scheduling system sends a task instruction ciphertext to the task node through the long connection, and the task node decrypts and matches the instruction ciphertext through the key to form an execution command, and executes the task according to the execution command, thereby solving the technical problem that in a specific network environment, due to a network isolation strategy, the task scheduling system and the task node cannot communicate bidirectionally, and the task scheduling system cannot send the task execution instruction to the task node through a remote call, thereby realizing cross-network partition task scheduling, reducing resource waste, ensuring that the task is executed on the most suitable node, and avoiding unnecessary repeated execution.

Description

Technical Field

[0001] The present invention relates to the field of distributed cluster technology, and more specifically to a distributed task scheduling method and system across networks. Background Art

[0002] In a distributed system, the task scheduling system can automatically trigger the execution of scheduled tasks without manual intervention, ensuring that tasks are executed accurately according to the scheduled time, and executing tasks such as data backup and report generation at a specific time. In order to efficiently utilize task nodes, the task scheduling system can more effectively utilize the resources in the cluster through reasonable scheduling. The current general solution is that the task node needs to actively register with the task scheduling system. After receiving the registration request, the task scheduling system maintains the communication address of the task node in the registry. When it is necessary to issue a scheduled task, the task scheduling system selects a suitable task node from the registry according to the load balancing strategy, and initiates a remote call through the communication address to notify the task node to execute the scheduled task. If the task fails to execute, the task scheduling system will automatically fail over, select another node from the registry, and re-issue the task. To achieve this process, at least two communications need to be completed: the task node initiates a registration request to the task scheduling system, and the task scheduling system sends a task execution instruction to the task node. Therefore, it is required that the network between the task scheduling system and the task node can communicate in both directions.

[0003] However, in a large-scale distributed cluster, network isolation is usually implemented to divide different partitions, such as the intranet and the extranet. The intranet usually has a higher security level, does not allow direct external access, and does not allow access to services in the extranet. The extranet can access the intranet through authorization. This network isolation strategy is highly secure, but it increases the complexity of task scheduling. In another case, when IP addresses are allocated within a partition, even if the task scheduling system obtains the communication addresses of task nodes in other partitions, these addresses cannot be directly used for cross-partition remote calls due to the network isolation strategy that limits direct communication between different partitions, and effective data exchange and task collaboration cannot be performed. At present, in order to assign tasks to task nodes in each partition, a common solution is to deploy a task scheduling system independently in each partition, and the task scheduling system distributes tasks within the partition. This leads to redundant deployment and resource waste, and for tasks that only need to be executed once, this solution causes the task to be executed repeatedly in each partition.

[0004] Therefore, there is an urgent need for a distributed task scheduling method and system across networks that can overcome network isolation, break through different partitions to perform task scheduling, realize two-way communication between the task scheduling system and the task nodes, send task execution instructions to the task nodes through remote calls, realize task scheduling across network partitions, reduce resource waste, ensure that tasks are executed on the most appropriate nodes, and avoid unnecessary repeated execution. Summary of the invention

[0005] In view of the above problems, the purpose of the present invention is to provide a cross-network distributed task scheduling method and system to solve the technical problems of redundant deployment, resource waste and repeated task execution in the distributed task scheduling method in the prior art.

[0006] The present invention provides a cross-network distributed task scheduling method, which includes:

[0007] Send the key generated when creating the application and the communication address of the preset task scheduling system to each task node corresponding to the application;

[0008] Establishing a long connection between the task node and the task scheduling system based on the key and the communication address;

[0009] The task scheduling system sends the task instruction ciphertext to the task node through the long connection, and the task node decrypts and matches the instruction ciphertext through the key to form an execution command, and executes the task according to the execution command.

[0010] Optionally, the process of sending the key generated when creating the application and the communication address of the preset task scheduling system to each task node corresponding to the application includes:

[0011] Creating an application, establishing application information of the application, and generating a key corresponding to the application;

[0012] Determine each task node corresponding to the application, and configure the key, the application information, and a communication address of a preset task scheduling system into a configuration file of the task node.

[0013] Optionally, the process of establishing a long connection between the task node and the task scheduling system based on the key and the communication address includes:

[0014] The task node forms a node information string about its own node, and signs the node information string based on the key in the configuration file to form signed node information;

[0015] Enable the task node to initiate a persistent connection registration request to the task scheduling system through the communication address, and transmit the signature node information to the task scheduling system through the protocol header of the persistent connection registration request;

[0016] The task scheduling system is instructed to perform authentication processing based on the information character string and the signature. If the authentication is successful, a long connection between the task node and the task scheduling system is established.

[0017] Optionally, the task scheduling system performs authentication processing based on the information string and the signature, including:

[0018] The task scheduling system receives a long connection registration request sent by the task node;

[0019] Read the current time in the protocol header of the persistent connection registration request, and determine whether the persistent connection registration request is within the validity period according to the current time; wherein,

[0020] If it is not within the validity period, the authentication fails and the connection is disconnected;

[0021] If it is within the validity period, the application name in the protocol header is read, and a query is made in the preset application information table whether the application name exists. If not, the authentication fails and the connection is disconnected. If it exists, the signature node information is parsed, a signature calculation is performed based on the node information character string in the signature node information to generate a judgment signature, and the judgment signature is matched with the signature in the signature node information. If the match is successful, the authentication is successful.

[0022] Optionally, after establishing a long connection between the task node and the task scheduling system based on the key and the communication address, the method further includes:

[0023] The task scheduling system stores the persistent connection session corresponding to the persistent connection into a local session pool, and stores the node information of the task node into a registration table.

[0024] Optionally, after establishing a long connection between the task node and the task scheduling system based on the key and the communication address, the method further includes:

[0025] The task node is made to monitor the connection status of the long connection in real time, wherein:

[0026] If the connection state of the persistent connection is disconnected, the task node sends a persistent connection request to the task scheduling system again.

[0027] Optionally, after establishing a long connection between the task node and the task scheduling system based on the key and the communication address, the method further includes:

[0028] Enable the task node to start a first background daemon thread, and use the first background daemon thread to periodically send a heartbeat packet to the task scheduling system through a long connection;

[0029] Among them, after receiving the heartbeat packet, the task scheduling system actively updates the active time of the task node in the registry, and the task scheduling system starts a second background daemon thread, which periodically checks the registry through the second background daemon thread, and removes the task nodes whose active time has expired from the registry.

[0030] Optionally, the process of causing the task scheduling system to send a task instruction ciphertext to the task node through the long connection, and causing the task node to decrypt and match the instruction ciphertext through the key to form an execution command includes:

[0031] The task scheduling system periodically queries a preset scheduled task table to obtain scheduled tasks;

[0032] Obtaining an application ID corresponding to the scheduled task, querying application information corresponding to the application ID to obtain a key corresponding to the application ID;

[0033] Generate a unique identifier, perform field assembly based on the unique identifier to form an instruction string, and use the key to perform AES encryption on the instruction string to form an instruction ciphertext;

[0034] Querying the task node list under the relevant application in the registration table, selecting a task node as a target task node in the task node list according to the load balancing strategy, taking out the node identifier of the target task node, and calling the corresponding long connection session in the local session pool based on the node identifier;

[0035] The task scheduling system sends the instruction ciphertext to the target task node through the long connection session;

[0036] The target task node uses the key in the local configuration file to decrypt the instruction ciphertext to generate an execution command.

[0037] Optionally, the task scheduling system determines and receives a task expression by means of time recognition; wherein the task expression is preset in the scheduled task table

[0038] The present invention also provides a cross-network distributed task scheduling system, which implements the cross-network distributed task scheduling method as described above, and the system includes:

[0039] A task node configuration unit, used to send the key generated when creating the application and the communication address of the preset task scheduling system to each task node corresponding to the application;

[0040] A long connection establishing unit, used to establish a long connection between the task node and the task scheduling system based on the key and the communication address;

[0041] The task execution unit is used to enable the task scheduling system to send the task instruction ciphertext to the task node through the long connection, and enable the task node to decrypt and match the instruction ciphertext through the key to form an execution command, and execute the task according to the execution command.

[0042] It can be seen from the above technical solution that the cross-network distributed task scheduling method and system provided by the present invention first sends the key generated when creating the application and the communication address of the task scheduling system in the intranet cluster to each task node corresponding to the application, and then establishes a long connection between the task node and the task scheduling system based on the key and the communication address, and then enables the task scheduling system to send the task instruction ciphertext to the task node through the long connection, and enables the task node to decrypt and match the instruction ciphertext through the key to form an execution command, and execute the task according to the execution command, thereby solving the technical problem that in a specific network environment, due to the network isolation strategy, the task scheduling system and the task node cannot communicate bidirectionally, and the task scheduling system cannot send the task execution instruction to the task node through remote call, thereby realizing task scheduling across network partitions, reducing resource waste, ensuring that the task is executed on the most suitable node, and avoiding unnecessary repeated execution. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] By referring to the following description in conjunction with the accompanying drawings, and with a more comprehensive understanding of the present invention, other objects and results of the present invention will become more clear and easy to understand. In the accompanying drawings:

[0044] Figure 1 is a flow chart of a distributed task scheduling method across networks according to an embodiment of the present invention;

[0045] Figure 2 A cluster diagram of a distributed task scheduling method across networks according to an embodiment of the present invention;

[0046] Figure 3 A schematic diagram of a process flow of a distributed task scheduling method across networks according to an embodiment of the present invention;

[0047] Figure 4 The figure is a logical block diagram of a distributed task scheduling system across networks according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] Implementing network isolation and dividing the intranet and extranet zones has high security, but increases the complexity of task scheduling. However, since the network isolation strategy restricts direct communication between different partitions, the method of allocating IP addresses for communication within the partition cannot perform effective data exchange and task collaboration because the addresses cannot be directly used for cross-partition remote calls. At present, in order to assign tasks to task nodes in each partition, the common solution is to deploy a task scheduling system independently in each partition, and the task scheduling system distributes tasks within the partition, but this leads to redundant deployment and waste of resources. In addition, for tasks that only need to be executed once, this solution will cause the task to be executed repeatedly in each partition.

[0049] In view of the above problems, the present invention provides a cross-network distributed task scheduling method and system. The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0050] In order to illustrate the cross-network distributed task scheduling method and system provided by the present invention, Figure 1-Figure 4 The embodiments of the present invention are exemplarily indicated.

[0051] The following description of exemplary embodiments is merely illustrative in nature and is not intended to limit the present invention and its application or use. Technologies and devices known to ordinary technicians in the relevant field may not be discussed in detail, but in appropriate cases, the technologies and devices should be considered as part of the specification.

[0052] like Figure 1-Figure 3 As shown in the figure, the present invention provides a distributed task scheduling method across networks, comprising:

[0053] S1: Send the key generated when creating the application and the communication address of the preset task scheduling system to each task node corresponding to the application;

[0054] S2: establishing a long connection between the task node and the task scheduling system based on the key and the communication address;

[0055] S3: The task scheduling system sends the task instruction ciphertext to the task node through the long connection, and the task node decrypts and matches the instruction ciphertext through the key to form an execution command, and executes the task according to the execution command.

[0056] It should be noted that steps S1 to S3 can be applied to the situation where the intranet and the extranet are isolated, that is, the task scheduling system is in the intranet cluster of the distributed cluster; the task node is in the extranet cluster or the intranet cluster of the distributed cluster, thereby overcoming network isolation, breaking through different partitions for task scheduling, and realizing two-way communication between the task scheduling system and the task node; it can also be applied to the situation where "IP addresses are allocated within the partition, even if the task scheduling system obtains the communication addresses of task nodes in other partitions, but due to the network isolation strategy, different partitions cannot communicate directly, these addresses cannot be directly used for cross-partition remote calls, and effective data exchange and task collaboration cannot be performed". Similarly, the task scheduling system is placed in a certain zone; the task node is placed in other partitions that cannot communicate directly, thereby overcoming network isolation, breaking through different partitions for task scheduling, and realizing two-way communication between the task scheduling system and the task node. The task execution instruction is sent to the task node through remote call, and task scheduling across network partitions is realized, reducing resource waste, ensuring that the task is executed on the most appropriate node, and avoiding unnecessary repeated execution.

[0057] Specifically, step S1 is a process of sending the key generated when creating the application and the communication address of the preset task scheduling system to each task node corresponding to the application, wherein the process of sending the key generated when creating the application and the communication address of the preset task scheduling system to each task node corresponding to the application includes:

[0058] S11: creating an application, establishing application information of the application, and generating a key corresponding to the application;

[0059] S12: Determine each task node corresponding to the application, and configure the key, the application information, and a communication address of a preset task scheduling system into a configuration file of the task node.

[0060] More specifically, it should be noted that in the task scheduling system, applications can be created according to demand and application information of the applications can be maintained, and the application information can include: application name, application description, etc. When creating an application, the task scheduling system will automatically generate a key for the application, which is used for signature verification, encryption and decryption in subsequent communications.

[0061] Among them, applications are also called microservices. Each application corresponds to a group of running instances, called task nodes, also called instance nodes. Each task node can execute the scheduled tasks configured under the application. Users need to configure the key issued by the system and the communication address of the task scheduling system into the configuration file of the task node. For example, there is a large distributed system with many microservices (applications): order microservices, payment microservices, logistics microservices, SMS microservices, etc. There are multiple servers in the distributed system for deploying microservices, and microservices run on these servers in the form of java processes. In order to process a large amount of data and user requests and ensure the high availability and scalability of services, microservices will be distributed and deployed on multiple servers. For example, the order microservice is deployed on server1, server2, and server3, the payment microservice is deployed on server2 and server3, the logistics microservice is deployed on server3 and server4, and the SMS microservice is deployed on server1 and server2. Therefore, there are 3 running instances (running nodes) for orders, 2 running instances (running nodes) for payments, 2 running nodes for logistics, and 2 running nodes for SMS.

[0062] For example, orders: there is a scheduled task to check the order status. If a user has not paid for an order within 30 minutes, the order will be automatically closed.

[0063] Logistics: There is a scheduled task to push notifications to users when there are changes in logistics information.

[0064] In the above two examples, each running instance of a microservice (Java process) is a "task node".

[0065] After step S1, the key generated when creating the application and the communication address of the preset task scheduling system are sent to each task node corresponding to the application, step S2 is performed. Step S2 is a process of establishing a long connection between the task node and the task scheduling system based on the key and the communication address; wherein the process of establishing a long connection between the task node and the task scheduling system based on the key and the communication address includes:

[0066] S21: enabling the task node to form a node information string about its own node, and signing the node information string based on the key in the configuration file to form signed node information;

[0067] S22: enabling the task node to initiate a persistent connection registration request to the task scheduling system through the communication address, and transmitting the signature node information to the task scheduling system through the protocol header of the persistent connection registration request;

[0068] S23: Instruct the task scheduling system to perform authentication based on the information string and the signature. If the authentication is successful, a long connection between the task node and the task scheduling system is established.

[0069] The process of the task scheduling system performing authentication processing based on the information string and the signature includes:

[0070] S231: The task scheduling system receives a long connection registration request sent by the task node;

[0071] S232: Read the current time in the protocol header of the persistent connection registration request, and determine whether the persistent connection registration request is within the validity period according to the current time; wherein,

[0072] If it is not within the validity period, the authentication fails and the connection is disconnected;

[0073] If it is within the validity period, the application name in the protocol header is read, and a query is made in the preset application information table whether the application name exists. If not, the authentication fails and the connection is disconnected. If it exists, the signature node information is parsed, a signature calculation is performed based on the node information character string in the signature node information to generate a judgment signature, and the judgment signature is matched with the signature in the signature node information. If the match is successful, the authentication is successful.

[0074] Specifically, in one embodiment, for a task node, during the startup phase, the task node assembles the current node information: application name, node name, node IP, and current time into a string, reads the key configured in the configuration parameters, signs the node information string, and then obtains the communication address of the task scheduling system configured in the configuration file, initiates a long connection registration request, and passes the node information and signature to the task scheduling system through the protocol header.

[0075] For the task scheduling system, after receiving the long connection registration request, the task scheduling system performs authentication. Read the current time in the protocol header to determine if the request has expired. If the request has expired, it will directly return authentication failure and disconnect. If it has not expired, read the application name in the protocol header and check whether the application exists in the application information table. If not, it will return authentication failure and actively disconnect the long connection. If there is application configuration information (the application information here needs to check whether the registration request is forged, so it is necessary to check whether the application exists in the application information table), then calculate the signature of the node information and compare it with the signature in the protocol header. If the comparison is successful, the connection is established successfully.

[0076] In addition, after establishing a long connection between the task node and the task scheduling system based on the key and the communication address, the method further includes:

[0077] The task scheduling system stores the persistent connection session corresponding to the persistent connection into a local session pool, and stores the node information of the task node into a registration table.

[0078] Also includes:

[0079] The task node is made to monitor the connection status of the long connection in real time, wherein:

[0080] If the connection state of the persistent connection is disconnected, the task node sends a persistent connection request to the task scheduling system again.

[0081] Also includes:

[0082] Enable the task node to start a first background daemon thread, and use the first background daemon thread to periodically send a heartbeat packet to the task scheduling system through a long connection;

[0083] Among them, after receiving the heartbeat packet, the task scheduling system actively updates the active time of the task node in the registry, and the task scheduling system starts a second background daemon thread, which periodically checks the registry through the second background daemon thread, and removes the task nodes whose active time has expired from the registry.

[0084] In short, the task scheduling system stores the persistent connection session in the local session pool and stores the node information of the task node in the registry. After the task node successfully establishes the connection, it monitors the persistent connection status change event in real time. When the persistent connection is disconnected, it re-initiates the registration request. It also starts the background daemon thread and regularly sends heartbeat packets to the task scheduling system through the persistent connection. After receiving the heartbeat packet, the task scheduling system updates the active time of the task node in the registry. The task scheduling system starts the background daemon thread, regularly checks the registry, and removes the task nodes whose active time has expired from the registry.

[0085] Then, when there is a task that needs to be executed, step S3 is performed, so that the task scheduling system sends the task instruction ciphertext to the task node through the long connection, and the task node decrypts and matches the instruction ciphertext through the key to form an execution command, and executes the task according to the execution command.

[0086] The process of causing the task scheduling system to send a task instruction ciphertext to the task node through the long connection, and causing the task node to decrypt and match the instruction ciphertext through the key to form an execution command includes:

[0087] S31: The task scheduling system periodically queries the preset scheduled task table to obtain the scheduled task; specifically, the task scheduling system periodically queries the scheduled task table, parses the timing expression defined in the task table, obtains the next execution time of the scheduled task, and determines whether the scheduled task needs to be executed in combination with the current time;

[0088] S32: Obtain the application ID corresponding to the scheduled task, query the application information corresponding to the application ID to obtain the key corresponding to the application ID; specifically, in step S31, after the task scheduling system queries the scheduled task to be executed, it can obtain the application ID in the task table, query the application information table according to the application ID, obtain the key field in the application information table, and then perform Base64 decoding on the key field, and take the last 128 bits as the key;

[0089] S33: Generate a unique identifier, and assemble fields based on the unique identifier to form an instruction string, and use the key to perform AES encryption on the instruction string to form an instruction ciphertext; in a specific embodiment, first generate a unique identifier (uuid), and assemble the task processor HANDLER field and the task parameter PARAM field in the task table into the instruction, and the instruction format is:

[0090] {

[0091] "cmd":"run",

[0092] "id":"Task execution record id${UUID}",

[0093] "handler":"Task processor ${HANDLER}",

[0094] "param":"Task parameter ${PARAM}"

[0095] }

[0096] Then the command string is encrypted with AES using the key to form the command ciphertext;

[0097] S34: querying the task node list under the relevant application in the registration table, selecting a task node as a target task node in the task node list according to the load balancing strategy, taking out the node identifier of the target task node, and calling the corresponding long connection session in the local session pool based on the node identifier;

[0098] S35: The task scheduling system sends the instruction ciphertext to the target task node through the long connection session;

[0099] S36: The target task node uses the key in the local configuration file to decrypt the instruction ciphertext to generate an execution command.

[0100] In a specific embodiment, the task scheduling system identifies the scheduled task table by time. The scheduled task table will be configured with a cron expression for each scheduled task, indicating the time to execute the scheduled task. After identifying the scheduled task table, the task scheduling system first assembles the execution task instructions, then queries the key of the application where the task is located, and uses the key to encrypt the execution task instructions.

[0101] When issuing a scheduled task, the task scheduling system queries the task node list in the registry, selects a suitable node according to the load balancing strategy, takes out the node ID of the node, and takes out the corresponding persistent connection session from the local session pool. The persistent connection session pushes the execution task instruction ciphertext to the task node. The load balancing strategy includes:

[0102] Select the first one: Fixedly select the first one in the node list;

[0103] Select the last one: Fixedly select the last one in the node list;

[0104] Polling: Select in order according to the node list;

[0105] Random: randomly select from the node list;

[0106] HASH: A fixed node in the node list is selected according to the Hash algorithm, and all tasks are evenly hashed on different machines.

[0107] Least frequently used: The least frequently used nodes in the node list come first;

[0108] Least recently used: The node in the node list that has not been used for the longest time is prioritized;

[0109] Broadcast: All nodes in the node list.

[0110] After the task node receives the pushed back execution task instruction, it uses the locally configured key to decrypt the ciphertext, and after decryption, it executes the corresponding scheduled task processing logic according to the specific content of the execution task instruction. After successful execution, the task execution success is sent to the task scheduling system through a long connection.

[0111] After pushing the execution task instruction, the task scheduling system will start the background thread and wait for the task node to return to the execution status. If the long connection is disconnected during the execution, the task execution is considered to have failed, and a new node will be selected in the registry to push the execution task instruction to the new node.

[0112] In order to better illustrate the cross-network distributed task scheduling method in this embodiment, a more specific embodiment is provided:

[0113] The application management function is provided in the task scheduling system, and the application information of the application maintenance includes: application name, application description. The table structure of the application information table is not specifically limited. In this specific embodiment, the application information table of the application information is as follows:

[0114]

[0115] The key field is automatically generated by the task scheduling system when the application is created. The key generation logic is: generate a UUID string, perform SHA-256 algorithm digest calculation on the UUID string to obtain a 256-bit key, and perform Base64 encoding to obtain the key string.

[0116] The task node configures parameters in the configuration file, including: task scheduling system address, application name, and key. Configuration parameter example:

[0117]

[0118] During the startup phase, the task node obtains system parameters, including:

[0119] ●Application name: read configuration parameters job.app

[0120] ●Node name: Get the host name of the task node (hostname)

[0121] Node ip: Get the ip of the task node

[0122] ●Node start time: Task node start time, get the timestamp of current time

[0123] Signature key: Read the configuration parameter job.secret, perform Base64 decoding, and take the first 128 bits

[0124] Encryption key: Read the configuration parameter job.secret, perform Base64 decoding, and take the last 128 bits

[0125] Next, concatenate the node information string in the following format:

[0126] app = application name & host = node name & ip = node ip & date = node startup time

[0127] Use the signature key to sign the node information string using the HmacSHA256 algorithm to obtain the signature string. Get the task scheduling system address configured in the configuration file, initiate a websocket connection, and pass the node information and signature to the task scheduling system via the header. The header example is as follows:

[0128]

[0129] After receiving the websocket connection request, the task scheduling system performs authentication:

[0130] 1) Get the ws-date in the header and compare it with the current time. If it is 10 seconds ago, it is considered an expired request, and the authentication failure is returned and the connection is disconnected;

[0131] 2) Get ws-app in the header, query the application information table according to the application name, and if the application does not exist, return authentication failure and disconnect;

[0132] 3) Based on the application information found, get the APP_SECRET field, perform Base64 decoding, and intercept the first 128 bits as the signature key

[0133] 4) Get the node information in the header and concatenate the node information string. The format is:

[0134] app=${ws-app}&host=${ws-host}&ip=${ws-ip}&date=${ws-date}

[0135] 5) Use the signature key to sign the node information string using the HmacSHA256 algorithm to obtain the signature string

[0136] 6) Compare the signature string with the ws-sign value in the header. If the comparison is successful, the connection is established successfully.

[0137] The task scheduling system generates a UUID, records it as the node ID, stores the node ID and the websocket session in the local session pool as a key-value pair, and stores the node information of the task node in the registry. The registry table structure is as follows:

[0138]

[0139]

[0140] After the websocket connection is successfully established, the task node monitors the onClose event of the websocket session in real time. When the connection is disconnected, the onClose event is triggered. After the task node monitors the onClose event, it re-initiates the registration request.

[0141] After the task node successfully establishes a websocket connection, it starts the background daemon thread and reports a heartbeat every 10 seconds: it sends a ping command to the task scheduling system through the websocket session. After receiving the heartbeat packet, the task scheduling system updates the most recent active time in the registry.

[0142] The task scheduling system starts a background daemon thread and performs a registry check every 30 seconds: query the registry and delete the task nodes whose most recent active time is less than 30 seconds ago from the registry.

[0143] The task scheduling system periodically pulls the task information table to check whether there are scheduled tasks that need to be executed. The structure of the scheduled task table is as follows:

[0144]

[0145] When a scheduled task needs to be executed, assemble the execution task instructions:

[0146]

[0147] Query the application information table according to the application name in the task table, obtain the key, perform Base64 decoding, take the last 128 bits as the encryption key, and perform AES encryption on the execution task instruction.

[0148] When issuing scheduled tasks, the task scheduling system queries the registry, selects the appropriate node according to the load balancing strategy, retrieves the node ID of this node, retrieves the corresponding websocket session from the local session pool, and pushes the ciphertext of the task execution instruction to the task node through the websocket session.

[0149] After receiving the pushed execution task instruction, the task node uses the locally configured encryption key to perform AES decryption on the ciphertext, and calls the corresponding processing class in the task node according to the task processor and task parameters specified in the instruction. After successful execution, the execution status is returned to the task scheduling system through the wesocket session:

[0150]

[0151] After pushing the execution task instruction, the task scheduling system starts the background thread and waits for the task node to return the execution status. If the websocket session is accidentally disconnected during the execution, the task execution is considered to have failed, and the registry is queried again, a new node is selected, and the execution task instruction is pushed to the new node.

[0152] As described above, the cross-network distributed task scheduling method provided by the present invention can solve the problems of redundant deployment and repeated execution of tasks in the task scheduling method in the prior art, realize task scheduling across network partitions, reduce resource waste, ensure that tasks are executed on the most appropriate nodes, and avoid unnecessary repeated execution.

[0153] like Figure 4 As shown, the present invention also provides a cross-network distributed task scheduling system 100, which implements the cross-network distributed task scheduling method as described above, and the system includes:

[0154] The task node configuration unit 101 is used to send the key generated when creating the application and the communication address of the preset task scheduling system to each task node corresponding to the application;

[0155] A long connection establishing unit 102, configured to establish a long connection between the task node and the task scheduling system based on the key and the communication address;

[0156] The task execution unit 103 is used to enable the task scheduling system to send the task instruction ciphertext to the task node through the long connection, and enable the task node to decrypt and match the instruction ciphertext through the key to form an execution command, and execute the task according to the execution command.

[0157] The more specific implementation of the cross-network distributed task scheduling system refers to the above-mentioned embodiment of the cross-network distributed task scheduling method, which will not be described in detail here.

[0158] As described above, the distributed task scheduling method and system across networks proposed by the present invention are described by way of example with reference to the accompanying drawings. However, those skilled in the art should understand that various improvements can be made to the distributed task scheduling method and system across networks proposed by the present invention without departing from the content of the present invention. Therefore, the protection scope of the present invention should be determined by the content of the attached claims.

Claims

1. A distributed task scheduling method across networks, characterized in that: include: Send the key generated when creating the application and the communication address of the preset task scheduling system to each task node corresponding to the application; Establishing a long connection between the task node and the task scheduling system based on the key and the communication address; The task scheduling system sends the task instruction ciphertext to the task node through the long connection, and the task node decrypts and matches the instruction ciphertext through the key to form an execution command, and executes the task according to the execution command.

2. The cross-network distributed task scheduling method according to claim 1, characterized in that: The process of sending the key generated when creating the application and the communication address of the preset task scheduling system to each task node corresponding to the application includes: Creating an application, establishing application information of the application, and generating a key corresponding to the application; Determine each task node corresponding to the application, and configure the key, the application information, and a communication address of a preset task scheduling system into a configuration file of the task node.

3. The cross-network distributed task scheduling method according to claim 1, characterized in that: The process of establishing a long connection between the task node and the task scheduling system based on the key and the communication address includes: The task node forms a node information string about its own node, and signs the node information string based on the key in the configuration file to form signed node information; Enable the task node to initiate a persistent connection registration request to the task scheduling system through the communication address, and transmit the signature node information to the task scheduling system through the protocol header of the persistent connection registration request; The task scheduling system is instructed to perform authentication processing based on the information character string and the signature. If the authentication is successful, a long connection between the task node and the task scheduling system is established.

4. The cross-network distributed task scheduling method according to claim 3, characterized in that: The process of the task scheduling system performing authentication processing based on the information string and the signature includes: The task scheduling system receives a long connection registration request sent by the task node; Read the current time in the protocol header of the persistent connection registration request, and determine whether the persistent connection registration request is within the validity period according to the current time; wherein, If it is not within the validity period, the authentication fails and the connection is disconnected; If it is within the validity period, the application name in the protocol header is read, and a query is made in the preset application information table whether the application name exists. If not, the authentication fails and the connection is disconnected. If it exists, the signature node information is parsed, a signature calculation is performed based on the node information character string in the signature node information to generate a judgment signature, and the judgment signature is matched with the signature in the signature node information. If the match is successful, the authentication is successful.

5. The cross-network distributed task scheduling method according to claim 4, characterized in that: After establishing a long connection between the task node and the task scheduling system based on the key and the communication address, the method further includes: The task scheduling system stores the persistent connection session corresponding to the persistent connection into a local session pool, and stores the node information of the task node into a registration table.

6. The cross-network distributed task scheduling method according to claim 4, characterized in that: After establishing a long connection between the task node and the task scheduling system based on the key and the communication address, the method further includes: The task node is made to monitor the connection status of the long connection in real time, wherein: If the connection state of the persistent connection is disconnected, the task node sends a persistent connection request to the task scheduling system again.

7. The cross-network distributed task scheduling method according to claim 6, characterized in that: After establishing a long connection between the task node and the task scheduling system based on the key and the communication address, the method further includes: Enable the task node to start a first background daemon thread, and use the first background daemon thread to periodically send a heartbeat packet to the task scheduling system through a long connection; Among them, after receiving the heartbeat packet, the task scheduling system actively updates the active time of the task node in the registry, and the task scheduling system starts a second background daemon thread, which periodically checks the registry through the second background daemon thread, and removes the task nodes whose active time has expired from the registry.

8. The cross-network distributed task scheduling method according to claim 7, characterized in that: The process of causing the task scheduling system to send a task instruction ciphertext to the task node through the long connection, and causing the task node to decrypt and match the instruction ciphertext through the key to form an execution command includes: The task scheduling system periodically queries a preset scheduled task table to obtain scheduled tasks; Obtaining an application ID corresponding to the scheduled task, querying application information corresponding to the application ID to obtain a key corresponding to the application ID; Generate a unique identifier, perform field assembly based on the unique identifier to form an instruction string, and use the key to perform AES encryption on the instruction string to form an instruction ciphertext; Querying the task node list under the relevant application in the registration table, selecting a task node as a target task node in the task node list according to the load balancing strategy, taking out the node identifier of the target task node, and calling the corresponding long connection session in the local session pool based on the node identifier; The task scheduling system sends the instruction ciphertext to the target task node through the long connection session; The target task node uses the key in the local configuration file to decrypt the instruction ciphertext to generate an execution command.

9. The cross-network distributed task scheduling method according to claim 8, characterized in that: The task scheduling system is located in an intranet cluster of a distributed cluster; the task node is located in an extranet cluster or an intranet cluster of the distributed cluster.

10. A distributed task scheduling system across networks, characterized in that: To implement the cross-network distributed task scheduling method according to any one of claims 1 to 9, the system includes: A task node configuration unit, used to send the key generated when creating the application and the communication address of the preset task scheduling system to each task node corresponding to the application; A long connection establishing unit, used to establish a long connection between the task node and the task scheduling system based on the key and the communication address; The task execution unit is used to enable the task scheduling system to send the task instruction ciphertext to the task node through the long connection, and enable the task node to decrypt and match the instruction ciphertext through the key to form an execution command, and execute the task according to the execution command.

Images