Configured Platform Scheduling Method, System and Storage Medium

Abstract

The present invention relates to big data processing technology, and discloses a configuration platform scheduling method. The method includes annotating scheduling tasks of a configuration platform, associating and storing the scheduling tasks and corresponding annotation information in the database sqlite, and setting the dependency relationship between scheduling tasks according to the annotation information; performing microservice scanning and registration on the scheduling tasks with annotations; allocating shards to the scheduling tasks to determine the association relationship between the server and the scheduling tasks after shard allocation; determining the execution server and execution method of the scheduling tasks according to the association relationship and the dependency relationship between the scheduling tasks; and executing the scheduling tasks according to the execution server and execution method of the scheduling tasks. The present invention also relates to blockchain technology, and the data is stored in the blockchain. The present invention enables the scheduling system of the configuration platform to have great advantages in terms of performance, stability, usability, and scalability.

Description

Technical Field

[0001] The present invention relates to big data processing, and in particular to a configuration platform scheduling method, system and storage medium. Background Art

[0002] The scheduling system is a core infrastructure in the big data platform. Currently, the scheduling method adopted by the scheduling system is a simple time-dependent scheduling method, such as crontab. However, the data processing process has a long dependency chain, resulting in problems such as unclear dependencies between tasks and difficulty in finding scheduling errors.

[0003] In the prior art, by adopting the elastic-job method, the problem of unclear dependencies between tasks has been improved, and the problem of difficulty in finding errors has been solved; however, there are still the following drawbacks:

[0004] 1) Insufficient support for dependencies between tasks;

[0005] 2) The graphical management interface has incomplete functions.

[0006] Therefore, there is an urgent need for a configuration platform scheduling method with rich scheduling functions. Summary of the Invention

[0007] The present invention provides a configuration platform scheduling method, system and computer-readable storage medium, which mainly solves the problem of insufficient support for dependencies between scheduling tasks of the configuration platform.

[0008] To achieve the above object, the present invention provides a configuration platform scheduling method, which is applied to an electronic device. The method includes: annotating the scheduling tasks of the configuration platform, associating and storing the scheduling tasks and corresponding annotation information in a lightweight database sqlite, and setting the dependency relationship between scheduling tasks according to the annotation information; performing microservice scanning on the annotated scheduling tasks, and registering them in the registration center of the configuration platform according to the type to which the annotations of the scheduling tasks belong; according to the number of servers existing in the registration center, allocating shards to the scheduling tasks based on the elastic-job framework, and determining the association relationship between the servers and the scheduling tasks after shard allocation; determining the execution server and execution method of the scheduling tasks according to the association relationship and the dependency relationship between scheduling tasks; and executing the scheduling tasks according to the execution server and execution method of the scheduling tasks.

[0009] Further, preferably, the annotations of the scheduling tasks include task node classification data, task status attribute data and task version attribute data; wherein, the task status attributes are divided into running, to be run, paused, completed or abnormal; the task node classification includes parent node tasks and child node tasks.

[0010] Further, preferably, the step of associatively storing the scheduling task and the corresponding annotation information in the lightweight database sqlite further includes: using the database listener of the configuration platform to add data points for multiple states to the original data points, and synchronizing all the data points to the database sqlite.

[0011] Further, preferably, the scheduling task and the corresponding annotation information are associatively stored in the lightweight database sqlite by the following method: dynamically adding a corresponding log file to the scheduling task according to the task, and storing the scheduling task in four levels of error, warn, info, and debug according to the log file; wherein the log file rolls over according to two dimensions of time and size.

[0012] Further, preferably, the data of the scheduling task is stored in the blockchain; synchronizing the scheduling task data stored in each lightweight database sqlite to the database of the configuration platform to implement the query of the task status of the scheduling task.

[0013] To achieve the above object, the present invention further provides a configuration platform scheduling system, including a scheduling task annotation unit, a scheduling task registration unit, a scheduling task association unit, and a scheduling task execution unit; the scheduling task annotation unit is used to annotate the scheduling tasks of the configuration platform, associatively store the scheduling tasks and the corresponding annotation information in the lightweight database sqlite, and set the dependency relationship between the scheduling tasks according to the annotation information; the scheduling task registration unit is used to perform microservice scanning on the annotated scheduling tasks and register them in the registration center of the configuration platform according to the type to which the annotations of the scheduling tasks belong; the scheduling task association unit is used to allocate shards to the scheduling tasks according to the elastic-job framework according to the number of servers existing in the registration center, and determine the association relationship between the servers and the scheduling tasks after shard allocation; the scheduling task execution unit is used to determine the execution server and execution method of the scheduling task according to the association relationship and the dependency relationship between the scheduling tasks; and execute the scheduling task according to the execution server and execution method of the scheduling task.

[0014] Further, preferably, the data of the scheduling task is stored in the blockchain; the scheduling task annotation unit includes a scheduling task annotation module and a scheduling task storage module with annotations; the scheduling task annotation module is used to annotate the scheduling tasks of the configuration platform; the annotations of the scheduling tasks include task node classification, task status attributes, and task version attributes; among them, the task status attributes are divided into in operation, pending execution, paused, completed, or abnormal; the task node classification includes parent node tasks and child node tasks; the scheduling task storage module with annotations is used to store the scheduling tasks and the corresponding annotation information in the lightweight database sqlite, and set the dependency relationship between the scheduling tasks according to the annotation information.

[0015] Further, preferably, the scheduling task storage module with annotations includes a storage sub-module, a dependency relationship setting sub-module, and a data buried point synchronization sub-module; the storage sub-module is used to store the scheduling tasks and the corresponding annotation information in the lightweight database sqlite; the dependency relationship setting sub-module is used to set the dependency relationship between the scheduling tasks according to the annotation information; the data buried point synchronization sub-module is used to add multiple status data buried points on the original data buried points by using the database listener of the configuration platform, and synchronize all the data buried points to the database sqlite.

[0016] To achieve the above object, the present invention also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores a program executable by the at least one processor, and the program is executed by the at least one processor so that the at least one processor can execute the configuration platform scheduling method as described above.

[0017] In addition, to achieve the above object, the present invention also provides a computer-readable storage medium storing a computer program, characterized in that when the computer program is executed by a processor, the steps of the above-mentioned configuration platform scheduling method are realized.

[0018] The configuration platform scheduling method, system, electronic device and computer-readable storage medium proposed by the present invention annotate the scheduling tasks of the configuration platform, associate and store the scheduling tasks and corresponding annotation information in the lightweight database sqlite, and set the dependency relationship between the scheduling tasks according to the annotation information; perform microservice scanning on the annotated scheduling tasks, and register them in the registration center of the configuration platform according to the type to which the annotations of the scheduling tasks belong; according to the number of servers existing in the registration center, allocate shards to the scheduling tasks based on the elastic-job framework, and determine the association relationship between the servers and the scheduling tasks after shard allocation; according to the association relationship and the dependency relationship between the scheduling tasks, judge the execution server and execution method of the scheduling tasks; execute the scheduling tasks according to the execution server and execution method of the scheduling tasks. It enriches the functions of the configuration platform; the beneficial effects are as follows:

[0019] 1). The management interface provides functions to start, pause, resume, and terminate existing tasks, and at the same time provides the function of dynamically adding new tasks, which will greatly improve the usability;

[0020] 2). The encapsulated framework will complete the establishment of tasks through annotation, and only need to focus on the data itself, which is very helpful for the extensibility of scheduling tasks;

[0021] 3). By combining the advantages of the distributed data scheduling of elastic-job itself, the scheduling system of the configuration platform will have great advantages in performance, stability, usability, and extensibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a flowchart of a preferred embodiment of the configuration platform scheduling method of the present invention;

[0023] Figure 2 It is a schematic structural diagram of a preferred embodiment of the configuration platform scheduling system of the present invention;

[0024] Figure 3 It is a schematic structural diagram of a preferred embodiment of the electronic device of the present invention;

[0025] The realization, functional features and advantages of the object of the present invention will be further described with reference to the embodiments and the accompanying drawings. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0027] In order to improve the user coding efficiency, the present invention provides a configuration platform scheduling method. Figure 1 Shows the flow of a preferred embodiment of the configuration platform scheduling method of the present invention. Refer toFigure 1 As shown, this method can be executed by a device, which can be implemented by software and / or hardware.

[0028] It should be noted that a configuration platform scheduling method of the present invention, specifically, the configuration platform scheduling method includes steps S110 - S150.

[0029] S110. Annotate the scheduling tasks of the configuration platform, associate and store the scheduling tasks and corresponding annotation information in the lightweight database sqlite, and set the dependency relationship between the scheduling tasks according to the annotation information.

[0030] In a specific embodiment, the annotation of the scheduling task includes task node classification data, task status attribute data, and task version attribute data; among them, the task status attribute is divided into running, to be run, paused, completed, or abnormal; the task node classification includes parent node tasks and child node tasks.

[0031] That is to say, add an annotation to each scheduling task, and the annotation includes node classification, task status, and version number; the task node classification is to add annotations for parent node tasks and child node tasks to increase the association (dependency) relationship between tasks. The task status attribute data and task version attribute data are used to store the historical status and version of the scheduling task through the lightweight database sqlite, and initialize the task status information and version information; in the specific implementation process, Sqlite creates a new table in file mode to store version information.

[0032] It should be noted that when the task server goes online, it will automatically register the server information in the registration center, and when it goes offline, it will automatically update the server status. That is to say, adding two attributes of task status and task version to the task provides a basis for adding the task dependency function for distributed tasks.

[0033] Among them, it should be noted that the initial version number is 0, and it is incremented by 1 after each execution. The status of the task is divided into running, to be run, paused, completed, or abnormal.

[0034] The specific implementation scenario is as follows: including RUNNING(1, "running"), READY(2, "to be run"), STOP(3, "to be run"), OVER(4, "to be run"), ERROR(5, "abnormal"); among them, the status of the task is READY when it is initially registered, RUNNING during the execution process, and OVER after the execution is completed. During the process of these status changes, the status and version number of each task are registered in zookeeper. The initial version number is 0 and it is incremented by 1 after each execution.

[0035] In a specific implementation process, the step of associatively storing the scheduling task and the corresponding annotation information in the lightweight database sqlite further includes using the database listener of the configuration platform to add data points for multiple states to the original data points, and synchronizing all the data points to the database sqlite.

[0036] It should be noted that data point is a method for collecting data from applications such as websites, Apps, or backends. Through data points, the behaviors generated by users in the application can be collected, which can then be used to analyze and optimize the subsequent experience of the product, and can also provide data support for the operation of the product. Data points correspond to task listeners, and for Elastic-job, listeners are divided into local listeners and distributed listeners. Among them, the local listener is only scheduled when the node executes its own shard, and the local listener will execute every time a shard task is scheduled. The local listener is defined by the ElasticJobListener interface. The local listener will execute when the job executes the local shard task. If the job is divided into 6 shards, the listener task will execute 6 times. The distributed listener will execute once when the total task starts to execute and once when the total task ends. The distributed listener is also implemented on the basis of the ordinary listener.

[0037] For each task, the original data points are based on the original database listener of elastic-job. Two attributes, task status and task version, are added. Therefore, it is necessary to add listeners and the corresponding data points of the listeners, so as to realize the corresponding control of each state of the task by the listener.

[0038] The specific implementation process is to rewrite the relevant class and add process status data to the original data points. It not only includes the original start and end states, but becomes 5 states: RUNNING(1, "Running"), READY(2, "Pending"), STOP(3, "Paused"), OVER(4, "Completed"), ERROR(5, "Exception").

[0039] In a specific embodiment, the scheduling task and the corresponding annotation information are associatively stored in the sqlite database by the following method. The corresponding log file is dynamically added to the scheduling task according to the task, and the scheduling task is hierarchically stored in four states: error, warn, info, and debug according to the log file; wherein the log file rolls over according to two dimensions of time and size.

[0040] That is to say, in the specific implementation process, if there are two tasks, A and B, during the operation of A, the status and version of each node will be registered in ZooKeeper. Task B will store the version information of A in the SQLite database within the execution plan. Before executing task B, compare the version information of task A in ZooKeeper with the version information of task B stored in the SQLite database. Only when the two version information is consistent and the status information of task A in ZooKeeper shows OVER can task B be executed normally. Otherwise, task B will be suspended.

[0041] S120. Perform microservice scanning on the scheduled task with annotations, and register it in the registry center of the configuration platform according to the type to which the annotation of the scheduled task belongs.

[0042] The registration here means registering tasks with special annotations.

[0043] Specifically, microservice scans all classes with special annotations to complete the registration. Among them, when registering, initialize the attribute task name, number of shards, sharding parameters, execution plan, and dependency relationship of the annotation. Among the above attributes, the task is filled in according to actual needs.

[0044] Reduce the development difficulty of developers, so that developers only need to focus on the business without having to focus on the framework.

[0045] The implementation scenarios are exemplified as follows:

[0046] When developers create a new scheduled task, add the @Job annotation and initialize the relevant attributes of the annotation at the same time. @Job(jobName="testJob", cron="0 / 10****?", shardingTotalCount=3)

[0047] S130. According to the number of servers existing in the registry center, allocate shards to the scheduled task based on the Elastic-Job framework, and determine the association relationship between the server and the scheduled task after shard allocation.

[0048] It is implemented by the task sharding method. A task is split into n independent task items, and the distributed servers execute their respective allocated shard items in parallel.

[0049] In other words, according to the number of servers existing in the registration space, determine the sharding situation of the task, and run the task in parallel or serially according to the plan. That is, determine the task allocation sharding situation according to the number of instances in the namespace. This strategy depends on the Elastic-Job framework.

[0050] It should be noted that the master node election, server online / offline, and the change of the total number of shards will all update the re-sharding flag. When the timing task is triggered, if re-sharding is required, the master server will perform the sharding. During the sharding process, it will be blocked, and the task can only be executed after the sharding is completed. If the master server goes offline during the sharding process, the master server will be elected first, and then the sharding will be carried out. To maintain the stability during the job operation, only the sharding status will be marked during the operation, and no re-sharding will be performed. Sharding is only likely to occur before the next task trigger. Each sharding will be sorted by the server IP to ensure that the sharding result will not fluctuate greatly. The failover function is implemented to actively grab the unassigned shards after a certain server finishes execution, and actively find an available server to execute the task after a certain server goes offline.

[0051] S140. Determine the execution server and execution method of the scheduling task according to the association relationship and the dependency relationship between the scheduling tasks.

[0052] Specifically, in step S130, a task is split into n independent task items. According to the association relationship between the server and the scheduling task after shard allocation, it is determined which server will specifically execute each independent task item. Therefore, the execution server of the scheduling task after shard allocation can be determined through the association relationship between the server and the scheduling task after shard allocation.

[0053] In addition, according to the dependency relationship between the scheduling tasks, that is, the dependency relationship between the scheduling tasks has been determined according to the task node classification (parent node task annotation and child node task annotation), that is, the dependency relationship between the scheduling tasks is clear, and the association relationship between the scheduling tasks is also determined. Therefore, the execution server and execution method of the scheduling task are confirmed.

[0054] S150. Execute the scheduling task according to the execution server and execution method of the scheduling task.

[0055] In a specific embodiment, the data of the scheduling task is stored in the blockchain; the scheduling task data stored in each lightweight database sqlite is synchronized to the database of the configuration platform, so as to realize the query of the task status of the scheduling task.

[0056] Specifically, the log rolling function is added through logback. It should be noted that log rolling means log update. Among them, logback-core is the basic module for the other two modules; logback-classic is an improved version of log4j. At the same time, it fully implements the slf4j API, enabling you to easily switch to other logging systems such as log4j or JDK14 Logging; logback-access is the access module that integrates with the Servlet container to provide the function of accessing logs through Http. By using logback to add the log rolling function, the performance is stable and the running speed is faster.

[0057] The original task included two states: the original start and end. Now, two attributes, task status and task version, are added, changing to 5 states: RUNNING(1, "Running"), READY(2, "Pending"), STOP(3, "Paused"), OVER(4, "Completed"), ERROR(5, "Exception"); therefore, the log update mode also needs to be changed.

[0058] Specifically, for the task, the corresponding log file is dynamically added according to the task, and the log is implemented to roll over in two dimensions: time and size, and the logs of each task are stored in levels according to error, warn, info, and debug.

[0059] According to the dependency relationship, register and store the task status and version in the sqlite database, and synchronize it to the sqlite of other servers. At the same time, synchronize the status and version to the database. Create a node through the curator client, set the content, and other servers obtain the status information and version information in the node.

[0060] In the entire process, tasks can be added, deleted, paused, and run again through the interface. Each operation of the interface will trigger the above process (According to the dependency relationship, register and store the task status and version in the sqlite database, and synchronize it to the sqlite of other servers. At the same time, synchronize the status and version to the database. Create a node through the curator client, set the content, and other servers obtain the status information and version information in the node).

[0061] In a specific embodiment, there are already two tasks, A and B, in the system. Task C does not have the @Job annotation when the system starts and can be activated at a specific time. At this time, parameters can be passed to the server through the page. According to the parameters, task C can be found, with other parameters, task C is activated, and task C is executed according to the execution plan in the parameters. When task C is activated, the task will be registered according to the parameters, the dependency relationship will be established, and the status and version will be managed during the running process.

[0062] In summary, use ZooKeeper as the registration center and synchronize the status of nodes to the registration center; it is realized that child nodes can listen to the task status of relevant parent nodes through listeners, and determine whether the current child node processes data based on the task status; thus realizing task dependencies and also realizing the monitoring of the graphical interface.

[0063] Complete the operation of ZooKeeper through the ZooKeeper connection pool. This includes adding nodes, writing status data into nodes, querying the status data of a certain node, etc. The executor synchronizes and refreshes the information in ZooKeeper to the database to provide query of historical data. Develop a new management platform through the RESTful method of HTTP requests, and support starting, pausing, running again, and terminating existing tasks.

[0064] Figure 2 This is the structure of the preferred embodiment of the configuration platform scheduling system of the present invention; refer to Figure 2 as shown

[0065] The configuration platform scheduling system 200 includes a scheduling task annotation unit 210, a scheduling task registration unit 220, a scheduling task association unit 230, and a scheduling task execution unit 240.

[0066] The scheduling task annotation unit 210 is used to annotate the scheduling tasks of the configuration platform, associate and store the scheduling tasks and the corresponding annotation information in the lightweight database SQLite, and set the dependency relationship between scheduling tasks according to the annotation information. The scheduling task registration unit 220 is used to perform microservice scanning on the annotated scheduling tasks and register them in the registration center of the configuration platform according to the type to which the annotations of the scheduling tasks belong. The scheduling task association unit 230 is used to allocate shards to the scheduling tasks according to the elastic-job framework based on the number of servers existing in the registration center, and determine the association relationship between the servers and the scheduling tasks after shard allocation. The scheduling task execution unit 240 is used to determine the execution server and execution method of the scheduling task according to the association relationship and the dependency relationship between scheduling tasks; execute the scheduling task according to the execution server and execution method of the scheduling task.

[0067] The data of the scheduling task is stored in the blockchain; the scheduling task annotation unit 210 includes a scheduling task annotation module 211 and a scheduled task storage module 212 with annotations; the scheduling task annotation module 211 is used to annotate the scheduling tasks of the configuration platform; the annotations of the scheduling task include task node classification, task status attributes, and task version attributes; among them, the task status attributes are divided into in operation, to be run, paused, completed, or abnormal; the task node classification includes parent node tasks and child node tasks; the scheduled task storage module 212 with annotations is used to associate and store the scheduling task and the corresponding annotation information in the lightweight database sqlite, and set the dependency relationship between scheduling tasks according to the annotation information.

[0068] In a specific embodiment, it should be emphasized that the scheduled task storage module 212 with annotations includes a storage sub-module 2111, a dependency relationship setting sub-module 2112, and a data buried point synchronization sub-module 2113.

[0069] The storage sub-module 2111 is used to associate and store the scheduling task and the corresponding annotation information in the lightweight database sqlite; the dependency relationship setting sub-module 2112 is used to set the dependency relationship between scheduling tasks according to the annotation information; the data buried point synchronization sub-module 2113 is used to add data buried points of multiple statuses to the original data buried points by using the database listener of the configuration platform, and synchronize all the data buried points to the database sqlite.

[0070] Taking the application scenario of the vehicle positioning system as an example, the server environment is built by using the configuration platform scheduling method based on the image page of the present invention. The vehicle-mounted positioning software of each vehicle starts a timing task and uploads the positioning information to the server every 5s. The server management interface can operate on the timing tasks of each vehicle; the system server starts a timing task and analyzes the location information uploaded by all vehicles every 30s:

[0071] That is to say, the scheduling task here is specifically a positioning task. First, annotate the positioning task, associate and store the positioning task and the corresponding annotation information in the lightweight database sqlite, and set the dependency relationship between the positioning task and the historical tasks of the main server according to the annotation information; among them, the historical tasks in the main server are used as the parent node tasks and the positioning tasks are used as the child node tasks.

[0072] Perform microservice scanning on the scheduled tasks with annotations and register them in the registration center of the server according to the types to which the annotations of the scheduled tasks belong; determine the association relationship between each specific server and the positioning task according to the number of servers existing in the registration center, that is, select the server to execute the positioning task; according to the association relationship and the dependency relationship between the scheduled tasks, determine the historical tasks of the server executing the positioning task and its associated primary server. Once the execution server and its parent node tasks are determined, the execution method of the positioning task is determined; execute the positioning task according to the execution server and the execution method of the positioning task, and then the vehicle-mounted positioning system makes corresponding warnings according to the positioning task according to the specified rules.

[0073] The present invention provides a configuration platform scheduling method, which is applied to an electronic device 3.

[0074] Figure 3 Shows the application environment according to a preferred embodiment of the configuration platform scheduling method of the present invention.

[0075] Refer to Figure 3 As shown, in this embodiment, the electronic device 4 may be a terminal device with computing functions such as a server, a smart phone, a tablet computer, a portable computer, a desktop computer, etc.

[0076] The electronic device 3 includes: a processor 32, a memory 31, a communication bus 33, and a network interface 35.

[0077] The memory 31 includes at least one type of readable storage medium. The at least one type of readable storage medium may be a non-volatile storage medium such as a flash memory, a hard disk, a multimedia card, a card-type memory 31, etc. In some embodiments, the readable storage medium may be the internal storage unit of the electronic device 3, such as the hard disk of the electronic device 3. In other embodiments, the readable storage medium may also be the external memory 31 of the electronic device 3, such as a plug-in hard disk equipped on the electronic device 3, a Smart Media Card (SMC), a Secure Digital (SD) card, a Flash Card, etc.

[0078] In this embodiment, the readable storage medium of the memory 31 is generally used to store the configuration platform scheduling program 30 installed in the electronic device 3, etc. The memory 31 can also be used to temporarily store the data that has been output or will be output.

[0079] In some embodiments, the processor 32 may be a central processing unit (CPU), a microprocessor, or other data processing chips, which are used to run the program code stored in the memory 31 or process data, such as executing the configured platform scheduler 30, etc.

[0080] The communication bus 33 is used to realize the connection and communication between these components.

[0081] The network interface 34 may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), which is usually used to establish a communication connection between the electronic device 3 and other electronic devices.

[0082] Figure 3 Only the electronic device 3 with components 31 - 34 is shown, but it should be understood that it is not required to implement all the shown components, and more or fewer components can be alternatively implemented.

[0083] Optionally, the electronic device 3 may further include a user interface. The user interface may include an input unit such as a keyboard, a voice input device such as a microphone and other devices with voice recognition functions, a voice output device such as a speaker, headphones, etc. Optionally, the user interface may further include a standard wired interface and a wireless interface.

[0084] Optionally, the electronic device 3 may further include a display, which may also be referred to as a display screen or a display unit. In some embodiments, it may be an LED display, a liquid crystal display, a touch liquid crystal display, and an organic light-emitting diode (OLED) toucher, etc. The display is used to display the information processed in the electronic device 3 and to display a visual user interface.

[0085] Optionally, the electronic device 3 may further include a radio frequency (RF) circuit, sensors, an audio circuit, etc., which will not be elaborated here.

[0086] In Figure 3In the illustrated device embodiment, in a memory 31 serving as a computer storage medium, an operating system and a configured platform scheduler 30 may be included; when a processor 32 executes the configured platform scheduler 30 stored in the memory 31, the following steps are implemented: annotating the scheduling tasks of the configured platform, associating and storing the scheduling tasks and corresponding annotation information in a lightweight database sqlite, and setting the dependency relationships between the scheduling tasks according to the annotation information; performing microservice scanning on the annotated scheduling tasks, and registering them in the registration center of the configured platform according to the types to which the annotations of the scheduling tasks belong; according to the number of servers existing in the registration center, allocating shards to the scheduling tasks based on the elastic-job framework, and determining the association relationships between the servers and the scheduling tasks after shard allocation; determining the execution servers and execution methods of the scheduling tasks according to the association relationships and the dependency relationships between the scheduling tasks; and executing the scheduling tasks according to the execution servers and execution methods of the scheduling tasks..

[0087] In other embodiments, the configured platform scheduler 30 may also be divided into one or more modules, and the one or more modules are stored in the memory 31 and executed by the processor 32 to complete the present invention. The modules referred to in the present invention refer to a series of computer program segments capable of completing specific functions. The configured platform scheduler 30 may be divided into a scheduling task annotation unit 210, a scheduling task registration unit 220, a scheduling task association unit 230, and a scheduling task execution unit 240.

[0088] In addition, the present invention also provides a computer-readable storage medium, mainly including a storage data area and a storage program area. Among them, the storage data area may store data created according to the use of blockchain nodes, etc., and the storage program area may store an operating system and application programs required for at least one function. The computer-readable storage medium includes a configured platform scheduler, and when the configured platform scheduler is executed by a processor, it implements operations such as those of the configured platform scheduling method.

[0089] The specific implementation manners of the computer-readable storage medium of the present invention are substantially the same as those of the above-mentioned configured platform scheduling method, system, and electronic device, and will not be elaborated herein.

[0090] Generally speaking, the scheduling method, system, electronic device and computer-readable storage medium of the configuration platform of the present invention annotate the scheduling tasks of the configuration platform, store the annotated scheduling tasks in the lightweight database sqlite, and set the dependency relationships between the scheduling tasks; perform microservice scanning on the annotated scheduling tasks and register them in the registration center of the configuration platform according to the types to which the annotations of the scheduling tasks belong; according to the number of servers existing in the registration center, allocate shards to the scheduling tasks based on the elastic-job framework to determine the association relationship between the servers and the scheduling tasks after shard allocation; according to the association relationship between the servers and the scheduling tasks after shard allocation and the dependency relationships between the scheduling tasks, judge the execution servers and execution methods of the scheduling tasks and then execute the scheduling tasks, enriching the functions of the configuration platform; the management interface provides functions to start, pause, resume, and terminate existing tasks, and at the same time provides the function of dynamically adding new tasks, which will greatly improve the usability; the encapsulated framework will complete the establishment of tasks through annotation, only need to focus on the data itself, which is very helpful for the extensibility of scheduling tasks; by combining the advantages of the distributed data scheduling of elastic-job itself, the scheduling system of the configuration platform will have great advantages in terms of performance, stability, usability, and extensibility.

[0091] The blockchain referred to in the present invention is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, and encryption algorithms. Blockchain, in essence, is a decentralized database, a string of data blocks generated by using cryptographic methods. Each data block contains information about a batch of network transactions, which is used to verify the validity (anti-counterfeiting) of the information and generate the next block. The blockchain can include a blockchain underlying platform, a platform product service layer, an application service layer, etc.

[0092] It should be noted that in this article, the terms "including", "comprising" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such process, device, article or method. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of another identical element in the process, device, article or method including that element.

[0093] The serial numbers of the embodiments of the present invention above are only for description and do not represent the superiority or inferiority of the embodiments. Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium as described above (such as ROM / RAM, magnetic disk, optical disk), and includes several programs to enable a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention.

[0094] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. A configuration platform scheduling method, applied to an electronic device, Characterized in that, The method includes: Annotate the scheduling tasks of the configuration platform, and store the scheduling tasks and the corresponding annotation information in the lightweight database sqlite in an associated manner, and set the dependency relationship between the scheduling tasks according to the annotation information; wherein, use the database listener of the configuration platform to add data points for multiple states on the original data points, and synchronize all the data points to the database sqlite; the states of the data points include RUNNING, READY, STOP, OVER, and ERROR; Perform microservice scanning on the scheduling tasks with annotations, and register them in the registration center of the configuration platform according to the type to which the annotations of the scheduling tasks belong; wherein, the annotations of the scheduling tasks include task node classification data, task status attribute data, and task version attribute data; among them, the task status attributes are divided into running, to be run, paused, completed, or abnormal; the task node classification includes parent node tasks and child node tasks; According to the number of servers existing in the registration center, allocate shards to the scheduling tasks according to the elastic-job framework, and determine the association relationship between the servers and the scheduling tasks after shard allocation; wherein, according to the number of servers existing in the registration space, determine the sharding situation of the tasks, and run the tasks in parallel or serially according to the plan; when a timed task is triggered, if re-sharding is required, the main server is used for sharding, and it is blocked during the sharding process, and the task can be executed only after the sharding is completed; According to the association relationship and the dependency relationship between the scheduling tasks, determine the execution server and execution method of the scheduling tasks; Execute the scheduling tasks according to the execution server and execution method of the scheduling tasks.

2. The configuration platform scheduling method according to claim 1, Characterized in that, The scheduling tasks and the corresponding annotation information are stored in the lightweight database sqlite in an associated manner through the following method. Add corresponding log files to the scheduling tasks dynamically according to their tasks, and store the scheduling tasks in four levels of error, warn, info, and debug according to the log files; wherein the log files roll over according to two dimensions of time and size.

3. The configuration platform scheduling method according to claim 1, Characterized in that, The data of the scheduling tasks is stored in the blockchain; synchronize the scheduling task data stored in each lightweight database sqlite to the database of the configuration platform to realize the query of the task status of the scheduling tasks.

4. A configuration platform scheduling system, Characterized in that, It includes a scheduling task annotation unit, a scheduling task registration unit, a scheduling task association unit, and a scheduling task execution unit; wherein, The scheduling task annotation unit is used to annotate the scheduling tasks of the configuration platform, store the scheduling tasks and the corresponding annotation information in the lightweight database sqlite in an associated manner, and set the dependency relationship between the scheduling tasks according to the annotation information; A scheduling task registration unit, which is used to perform microservice scanning on the scheduling tasks with annotations and register them in the registration center of the configuration platform according to the types to which the annotations of the scheduling tasks belong; wherein, the data of the scheduling tasks is stored in the blockchain; the scheduling task annotation unit includes a scheduling task annotation module and a scheduling task storage module with annotations; the scheduling task annotation module is used to annotate the scheduling tasks of the configuration platform; the annotations of the scheduling tasks include task node classification, task status attributes, and task version attributes; wherein, the task status attributes are divided into running, to be run, paused, completed, or abnormal; the task node classification includes parent node tasks and child node tasks; the scheduling task storage module with annotations is used to store the scheduling tasks and the corresponding annotation information in the lightweight database sqlite and set the dependency relationships between the scheduling tasks according to the annotation information; A scheduling task association unit, which is used to allocate shards to the scheduling tasks according to the elastic-job framework based on the number of servers existing in the registration center, and determine the association relationship between the servers and the scheduling tasks after shard allocation; wherein, according to the number of servers existing in the registration space, determine the sharding situation of the tasks, and run the tasks in parallel or serially according to the plan; when the timing task is triggered, if re-sharding is required, the main server is used for sharding, and it is blocked during the sharding process, and the task can be executed only after the sharding is completed; A scheduling task execution unit, which is used to determine the execution server and execution method of the scheduling tasks according to the association relationship and the dependency relationships between the scheduling tasks; execute the scheduling tasks according to the execution server and execution method of the scheduling tasks; the scheduling task storage module with annotations includes a storage sub-module, a dependency relationship setting sub-module, and a data buried point synchronization sub-module; The storage sub-module is used to store the scheduling tasks and the corresponding annotation information in the lightweight database sqlite; The dependency relationship setting sub-module is used to set the dependency relationships between the scheduling tasks according to the annotation information; The data buried point synchronization sub-module is used to add data buried points with multiple statuses to the original data buried points by using the database listener of the configuration platform and synchronize all the data buried points to the database sqlite; the statuses of the data buried points include RUNNING, READY, STOP, OVER, and ERROR.

5. An electronic device, characterized in that, the electronic device includes: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, the memory stores a program executable by the at least one processor, and the program is executed by the at least one processor so that the at least one processor can execute the configuration platform scheduling method according to any one of claims 1 to 3.

6. A computer-readable storage medium storing a computer program, characterized in that, when the computer program is executed by a processor, it implements the configuration platform scheduling method according to any one of claims 1 to 3.

Images