Fusion information scheduling method and system

Abstract

This invention relates to the field of task scheduling technology and discloses a fusion information scheduling method and system. The method includes: a task scheduler executor polling the network status from a task scheduling center via a network component; determining the network status; when the network status is normal, the task scheduler executor acquires and executes the task schedule; when the network status is abnormal, the task scheduling data at the time of the network abnormality is recorded and stored in a local storage container; the task scheduler executor reads the scheduled task from the storage container through a task engine and executes the scheduled task; when the network returns to normal, the task scheduler executor sends the task scheduling data at the time of the network abnormality to the task scheduling center; resuming network status polling and receiving the latest service component information from the task scheduling center. The technical solution provided by this invention solves the problems of task scheduling failure, abnormal task execution status, and loss of task execution results due to network abnormalities.

Description

Technical Field

[0001] This invention relates to the field of task scheduling technology, and in particular to a fusion information scheduling method and system. Background Technology

[0002] Traditional task scheduling is based on customized development and unified management of business requirements and scenario needs of each system. When the network is interrupted, the tasks being executed cannot obtain database connection information, and the task status and task results cannot be stored in time, resulting in task data loss and untimely data processing. Some projects add built-in task scheduling to the project software to meet personalized needs. When the requirements change, the functionality of the project itself needs to be modified. Summary of the Invention

[0003] The main objective of this invention is to provide a fusion information scheduling method and system, which aims to solve the task scheduling problem when the network is interrupted.

[0004] To achieve the above objectives, the present invention provides a fusion information scheduling method, the method comprising: Step S10: The task scheduler executor polls the network status from the task scheduling center through the network component; Step S20: Determine the network status. If the network status is normal, proceed to step S30; if the network status is abnormal, proceed to step S60. Step S30: The task scheduling executor sends network status information to the task scheduling center; Step S40: After receiving the network status information, the task scheduling center returns the latest service component information to the task scheduling executor; Step S50: The task scheduler executor identifies the scheduling task by using the information conversion service to identify the latest service component information, and executes the scheduling task normally; Step S60: Record the task scheduling data when a network anomaly occurs and store it in a local storage container; Step S70: The task scheduler executor reads the scheduled task from the storage container through the task engine and executes the scheduled task; at the same time, it continuously polls the network status. Step S80: When the network returns to normal, the task scheduler executor reads the scheduling task from the storage container through the task engine and executes the scheduling task; at the same time, it sends the task scheduling data during the network anomaly to the task scheduling center. Step S90: Resume polling of network status and receive the latest service component information from the task scheduling center.

[0005] Further, step S50 includes: Step S510: The task scheduler executor identifies the scheduling task by using the information conversion service to analyze the latest service component information; Step S520: Update and store the identified scheduled tasks in the storage container; Step S530: Read the scheduled task in the storage container through the task engine and execute the scheduled task; Step S540: When the task scheduler sends network status information to the task scheduler center, it sends scheduling task data.

[0006] Furthermore, the fusion information scheduling method also includes a method for starting a task scheduling center, the method for starting a task scheduling center including: Start the task scheduling center and provide service interfaces to the outside world; When registration information is received through the service interface, the registration information is recorded and the external service information of the task scheduler is received. When network status information is received through the service interface, the latest component information is proactively sent out through the registration information.

[0007] Furthermore, the fusion information scheduling method also includes a method for starting a task scheduler executor, the method for starting the task scheduler executor including: Start the task scheduler executor and perform internal component self-checks and obtain available service component information through the service components, and start the task engine at the same time; Start the service components and network components; The network component reads information from the task scheduling center and continuously polls the network status from the task scheduling center. The network component detects the network status. When the network status is normal, the task scheduler executor sends registration information to the task scheduling center via the network to register.

[0008] Furthermore, the service components include any one or more of the following: ES search service components, cache service components, file service components, and built-in library service components.

[0009] Furthermore, the ES search service component is used to perform distributed full-text search on the data in the storage container.

[0010] Furthermore, the caching service component is used to improve data reading efficiency.

[0011] Furthermore, the file service component is used to generate state keys or file keys in an encrypted form to improve the security of the service.

[0012] Furthermore, the built-in library service component is used to store information generated during the scheduling and execution of each task.

[0013] In addition, the present invention also provides a task scheduling system, which includes a task scheduling center and several task scheduling executors. The several task scheduling executors are all connected to the task scheduling center through a network. The task scheduling center is used to customize and manage scheduling tasks, and the task scheduling executors are used to execute the scheduling tasks.

[0014] Furthermore, the task scheduler executor includes a task engine, service components, network components, an information conversion service, and a storage container. The task engine is used to execute scheduling tasks. The service components include any one or more of ES search service components, cache service components, file service components, and built-in library service components. The network component is used to poll the network status. The information conversion service is used to convert the latest service component information into scheduling tasks after identifying the scheduling tasks. The storage container is used to store task scheduling center information, registration information, network status information, and scheduling task execution information.

[0015] The fusion information scheduling method and system provided by this invention polls the network status through the network component of the task scheduler executor. When the network is normal, the task scheduling center receives the network status information and returns the latest service component information to the task scheduler executor. When the network is interrupted, the task scheduler executor executes the scheduling task locally and stores it in a local storage container. When the network is restored, it promptly sends the information back to the task scheduling center. This ensures that task scheduling is not affected by network problems during execution, and solves the problems of task scheduling failure, abnormal task execution status, and loss of task execution results due to network anomalies, thereby improving the success rate of task execution. Furthermore, this invention customizes the task scheduling mode according to business needs based on the service components of the task scheduler executor, reducing the maintenance cost of personalized tasks. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a fusion information scheduling method provided in an embodiment of the present invention; Figure 2 for Figure 1 A flowchart illustrating step S50 in the process; Figure 3 A flowchart illustrating a method for starting a task scheduling center according to an embodiment of the present invention; Figure 4 A flowchart illustrating the startup method of a task scheduling executor according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the structure of a task scheduling system provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the structure of a task scheduling executor provided in an embodiment of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0018] Please see Figure 1 The above is a schematic flowchart of a fusion information scheduling method according to an embodiment of the present invention. The method includes: Step S10: The task scheduler executor polls the network status from the task scheduling center through the network component; Step S20: Determine the network status. If the network status is normal, proceed to step S30; if the network status is abnormal, proceed to step S60. Step S30: The task scheduling executor sends network status information to the task scheduling center; Step S40: After receiving the network status information, the task scheduling center returns the latest service component information to the task scheduling executor; Step S50: The task scheduler executor identifies the scheduling task by using the information conversion service to identify the latest service component information, and executes the scheduling task normally; Step S60: Record the task scheduling data when a network anomaly occurs and store it in a local storage container; Step S70: The task scheduler executor reads the scheduled task from the storage container through the task engine and executes the scheduled task; at the same time, it continuously polls the network status. Step S80: When the network returns to normal, the task scheduler executor reads the scheduling task from the storage container through the task engine and executes the scheduling task; at the same time, it sends the task scheduling data during the network anomaly to the task scheduling center. Step S90: Resume polling of network status and receive the latest service component information from the task scheduling center.

[0019] Please see Figure 2 Step S50 includes: Step S510: The task scheduler executor identifies the scheduling task by using the information conversion service to analyze the latest service component information; Step S520: Update and store the identified scheduled tasks in the storage container; Step S530: Read the scheduled task in the storage container through the task engine and execute the scheduled task; Step S540: When the task scheduler sends network status information to the task scheduler center, it sends scheduling task data.

[0020] Please see Figure 3 The fusion information scheduling method further includes a method for starting a task scheduling center, which includes: Step S11: Start the task scheduling center and provide service interfaces to the outside world; specifically, start the task scheduling center to initialize and provide service interfaces to the outside world, that is, expose the service interface of the task scheduling center to the outside world and put it in a listening state, so that external devices can send information to the task scheduling center based on the service interface. Step S12: When registration information is received through the service interface, the registration information is recorded and the external service information of the task scheduler is received; the registration information is the information sent by the task scheduler when registering with the task scheduling center, and in one embodiment, it includes, for example, the identity information, address information, external service information, and task information of the task scheduler; the external service information is, for example, the port number of the TCP external service; Step S13: When network status information is received through the service interface, the latest component information is actively sent out through the registration information. Specifically, the task scheduler executor polls the network status with the task scheduling center and sends the current network status information. When the task scheduling center receives the network status information, it sends the latest component information on the task scheduling center to the task scheduler executor according to the registration information. The latest component information includes task scheduling data, such as management and control instructions for scheduling tasks.

[0021] Please see Figure 4 The fusion information scheduling method further includes a method for starting a task scheduler executor, the method for starting the task scheduler executor including: Step S21: Start the task scheduler executor and perform internal component self-checks and obtain available service component information through the service components, and start the task engine at the same time; wherein, the task engine is the core of the task scheduler executor for scheduling task processing, and in one specific embodiment, the task engine supports multi-threaded polling scheduling; Step S22: Start the service components and network components; wherein, the service components include any one or more of the following: ES search service component, cache service component, file service component, and built-in library service component; specifically,

[0022] The ES search service component is used to perform distributed full-text search on the data in the storage container, thereby enhancing the response efficiency of the data in the storage container. The caching service component is used to improve data reading efficiency; by configuring the caching service component, multiple caching methods can be provided to load frequently used and infrequently changing data, thereby improving data reading speed. The file service component is used to form state keys or file keys through encryption to improve service security; the file service component encrypts critical information and forms state keys and file keys through different encryption methods to improve service security. The built-in library service component is used to store the information generated during the scheduling and execution of each task; Step S23: The network component reads the task scheduling center information and continuously polls the network status to the task scheduling center; wherein, the task scheduling center information can be pre-set and stored in the storage container on the task scheduling executor, or it can be information from previously connected task scheduling centers stored in the storage container; Step S24: Detect the network status through the network component. When the network status is normal, the task scheduler executor sends registration information to the task scheduling center through the network to register. Specifically, the registration information is the information sent by the task scheduler executor when registering with the task scheduling center. In one embodiment, it may include the task scheduler executor's identity information, address information, external service information, and task information. The external service information may be, for example, the port number of the TCP external service.

[0023] In one specific embodiment, the task scheduler executor polls the network status to the task scheduling center through the network component; the polling of the network status means sending polling data packets to the task scheduling center at regular intervals and waiting for the task scheduling center to respond. If a response is received from the task scheduling center within a preset waiting time, it indicates that the network is normal; otherwise, it indicates that the network is abnormal. For example, it sends a polling data packet every 2 seconds, and if no response data packet is received after 6 seconds, it indicates that the network is abnormal.

[0024] When the network is normal, the task scheduler sends network status information to the task scheduling center; after receiving the network status information, the task scheduling center returns the latest service component information to the task scheduler; the task scheduler identifies the scheduling tasks in the latest service component information through the information conversion service, and updates and stores the identified scheduling tasks in the storage container; the task engine of the task scheduler executes the scheduling tasks according to the task scheduling data in the local storage container, stores the task execution data in the local storage container, and promptly sends the execution information of the scheduling tasks to the task scheduling center by sending network status information.

[0025] When a network anomaly occurs, both the task scheduling center and the task scheduling executor record the task scheduling data at the time of the anomaly and store it in a local storage container. The task scheduling executor reads the scheduled tasks from the storage container through its task engine and executes them. Simultaneously, it continuously polls the network status. Therefore, when a network anomaly occurs, the task scheduling executor does not interrupt the execution of the scheduled tasks. Instead, it first searches the local storage container for the status information and execution anomaly point of the task scheduling. Then, it attempts to recover the task scheduling using its own task engine, reading the status of the task scheduling in the local storage container and restarting the abnormal task scheduling using the task engine. Thus, the task engine of the task scheduling executor continues to read and execute the scheduled tasks in the storage container and stores the executed task scheduling data in the local storage container.

[0026] When the network returns to normal, the task scheduler executor reads the scheduled tasks from the storage container through the task engine and executes the scheduled tasks; at the same time, it resumes polling of the network status and receives the latest service component information from the task scheduling center.

[0027] Please see Figure 5 In addition, the present invention also provides a task scheduling system, which includes a task scheduling center 10 and a plurality of task scheduling executors 20. The plurality of task scheduling executors 20 are all connected to the task scheduling center 10 via a network. The task scheduling center 10 is used to customize and manage scheduling tasks, and the task scheduling executors 20 are used to execute the scheduling tasks.

[0028] Please see Figure 6 The task scheduler executor 20 includes a task engine 21, a service component 22, a network component 23, an information conversion service 24, and a storage container 25. The task engine 21 is used to execute scheduling tasks. The service component 22 includes any one or more of the following: an ES search service component, a cache service component, a file service component, and a built-in library service component. The network component 23 is used to poll the network status. The information conversion service 24 is used to convert the latest service component information into scheduling tasks after identifying the scheduling tasks. The storage container 25 is used to store task scheduling center information, registration information, network status information, and scheduling task execution information.

[0029] Specifically, through the connection between the task scheduling center and the task scheduling executor, the startup of the task scheduling executor is independent of whether the task scheduling center is experiencing a network outage. The task scheduling data generated during the task scheduling process, such as the status, process, and execution results of the scheduled tasks, is stored in a local storage container through the integration of the functions of various components of the task scheduling executor, such as ES search service components, cache service components, file service components, and built-in library service components. When the network component polls and discovers that the task scheduling center is online (i.e., when the network is restored to normal), it actively pushes registration information and obtains the latest component information for updates. During the polling process of the network component, the task scheduling data of the task scheduling is also monitored for network status. If the network is normal, the task scheduling data during the network outage will be returned.

[0030] Compared with existing technologies, the fusion information scheduling method and system provided by this invention polls the network status through the network component of the task scheduler executor. When the network is normal, the task scheduling center receives the network status information and returns the latest service component information to the task scheduler executor. When the network is interrupted, the task scheduler executor executes the scheduling task locally and stores it in a local storage container. When the network is restored, it promptly sends the information back to the task scheduling center. This ensures that task scheduling is not affected by network problems during execution, and solves the problems of task scheduling failure, abnormal task execution status, and loss of task execution results due to network anomalies, thereby improving the success rate of task execution. Furthermore, this invention customizes the task scheduling mode according to business needs based on the service components of the task scheduler executor, reducing the maintenance cost of personalized tasks.

[0031] The above description is merely an embodiment of the present invention. It should be noted that those skilled in the art can make improvements without departing from the inventive concept of the present invention, but these improvements all fall within the protection scope of the present invention.

Claims

1. A fusion information scheduling method, characterized in that, The fusion information scheduling method includes: Step S10: The task scheduler executor completes registration by sending registration information to the task scheduling center through the polling service of the network component and using the session key; Step S20: When registration is complete, the task scheduler executor sends network status information and updates the session key through the polling service; Step S30: Determine if the network status is interrupted; if the network is not interrupted, proceed to step S40; otherwise, proceed to step S80. Step S40: The task scheduling center obtains the latest component information and sends the latest service information to the task scheduling executor using the session key based on the component information; Step S50: After receiving the service information and decrypting it using the session key, the task scheduler processes it through the information conversion service to obtain task scheduling information; Step S60: The task scheduling executor stores the task scheduling information in a local storage container and updates the component information to the task scheduling center through the information conversion service; Step S70: The task scheduler executor obtains task scheduling information from the local storage container and performs task scheduling according to the task scheduling information; Step S80: The task scheduler continuously polls the network status through the network component and executes step S70; Step S90: After the session key is successfully decrypted, the task information is determined by priority algorithm, timestamp algorithm, and MD5 algorithm, and then the upload or download operation is performed.

2. The fusion information scheduling method according to claim 1, characterized in that, Step S10 includes: Step S110: The task scheduler executor sends registration information to the task scheduler center through the polling service of the network component according to the preset network service interface information of the task scheduler center; the registration information includes the network service interface information of the task scheduler executor and a session key randomly generated during each polling, and the session key is recorded in the local cache of the task scheduler executor. Step S120: The task scheduling center receives and records the registration information, and updates the session key to the local storage of the task scheduling center; Step S130: The task scheduling center returns the registration result to the task scheduling executor according to the session key; Step S140: The task scheduler executor receives the registration result and completes the registration.

3. The fusion information scheduling method according to claim 1, characterized in that, Step S50 includes: Step S510: The task scheduler executor receives the service information; Step S520: Use the session key to decrypt and identify the management and control instructions in the service information via the information conversion service; Step S530: Obtain task scheduling information according to the management and control instructions.

4. The fusion information scheduling method according to claim 1, characterized in that, Step S60 includes: Step S610: The task scheduler executor updates and stores the identified task scheduling information in the local storage container; Step S620: The task scheduler executor sends the updated component information back to the task scheduling center through the information conversion service; Step S630: The task scheduling center receives the updated component information to record the changed status and store the data.

5. The fusion information scheduling method according to claim 1, characterized in that, The fusion information scheduling method further includes a task scheduling anomaly recovery step, which includes: searching for the status information and execution anomaly points of the abnormal task scheduling in the local storage container to obtain the task scheduling anomaly information; encrypting the task scheduling anomaly information using a session key and uploading it to the task scheduling center; and the task executor executing the abnormal task scheduling through the task engine.

6. A converged information scheduling system, characterized in that, Used to perform the fusion information scheduling method belonging to any one of claims 1-6.

Images