A task scheduling method, device, server and storage medium
By automatically obtaining and analyzing the dependency information in the database operation script and optimizing the task scheduling sequence, the problem of inefficient task scheduling caused by manual filling in the existing technology is solved, and more flexible and efficient task scheduling is achieved.
Patent Information
- Application Number
- CN202011290362.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-11-18
- Publication Date
- 2025-02-18
- Estimated Expiration
- 2040-11-18
AI Technical Summary
In the prior art, task scheduling relies on manual filling, and there is arbitrary and vacant period, resulting in low task scheduling efficiency.
By obtaining the dependency information in the database operation script, the scheduling information of the pre-task is automatically obtained, and the execution of subsequent tasks is triggered after the pre-task is executed, optimizing the task scheduling sequence.
It improves the flexibility and efficiency of task scheduling, reduces the scheduling window period caused by manual errors, and improves the stability and performance of the overall system.
Smart Images

Figure CN112328382B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a task scheduling method, device, server and storage medium. Background Art
[0002] In the current development model, after the developer has developed the database operation script, he needs to manually identify the dependencies in the database operation script and manually copy these dependencies one by one to the scheduling platform. When the database operation script is less complex and short, these repetitive actions will become complicated and prone to errors, which may lead to deviations in the final running results of the database operation script or even errors.
[0003] At present, the scheduling mechanism of the scheduling platform is mostly manually filled in. This scheduling mode controlled by humans is arbitrary. For example, task A1 depends on upstream tasks B1, B2...Bn. The estimated time for B1 to complete is t1, the estimated time for B2 to complete is t2...Bn to complete is tn. If resources are sufficient, task A1 should be scheduled after max{t1, t2...tn}. However, the possible window period in between will reduce the overall scheduling efficiency of the task. Summary of the invention
[0004] The embodiments of the present application provide a task scheduling method, device, server and storage medium, which make the task scheduling process more flexible and improve the overall scheduling efficiency of tasks.
[0005] In a first aspect, an embodiment of the present application provides a task scheduling method, comprising:
[0006] Obtain the database operation script corresponding to the first task;
[0007] Obtaining dependency information of the first task according to the database operation script; the dependency information is precondition information that the first task depends on when executing;
[0008] Acquire scheduling information of a second task according to the dependency information, where the second task is a predecessor task of the first task;
[0009] After determining according to the scheduling information that the execution of the second task is completed, triggering the database operation script to run to execute the first task.
[0010] Optionally, acquiring the scheduling information of the second task according to the dependency information includes:
[0011] Querying a scheduling log of the second task from a database according to the dependency information, wherein the database includes a scheduling log of each task in a plurality of tasks;
[0012] The scheduling information of the second task is read from the scheduling log.
[0013] Optionally, obtaining dependency information of the first task according to the database operation script includes:
[0014] Identify the first keyword included in the database operation script by a text recognition method;
[0015] A second keyword included in the database operation script is determined according to the first keyword, and dependency information of the first task is obtained according to the second keyword, where the second keyword is located after the first keyword.
[0016] Optionally, the identifying the first keyword included in the database operation script by a text recognition method includes:
[0017] Performing word segmentation processing on the database operation script to obtain a word segmentation result;
[0018] A keyword detection technology is used to identify a first keyword included in the database operation script from the word segmentation result.
[0019] Optionally, the adopting of a keyword detection technology to identify a first keyword included in the database operation script from the word segmentation result includes:
[0020] Using a preset feature word library to determine the location of a target feature word from the word segmentation result, the feature word library includes a plurality of feature words, and the feature word refers to a feature word of a database operation statement used by the database operation script;
[0021] The keyword matching technology is used to identify the first keyword from the text content after the position of the target feature word in the word segmentation result.
[0022] Optionally, obtaining dependency information of the first task according to the second keyword includes:
[0023] determining the second keyword as dependency information of the first task; or,
[0024] A task name of a second task is generated according to the second keyword, and the task name of the second task is determined as dependency information of the first task.
[0025] Optionally, the method further comprises:
[0026] When there are multiple second tasks, determining the second task that is executed last among the multiple second tasks according to the scheduling information of each second task in the multiple second tasks, wherein the scheduling information includes the start execution time and the end execution time;
[0027] updating the start execution time of the first task according to the end execution time of the second task that is executed the latest, to obtain the updated start execution time of the first task;
[0028] The triggering of the database operation script to execute the first task includes:
[0029] When it is detected that the updated start execution time of the first task arrives, the database operation script is triggered to run to execute the first task.
[0030] In a second aspect, an embodiment of the present application provides a task scheduling device, including:
[0031] An acquisition module, used to acquire a database operation script corresponding to a first task, and obtain dependency information of the first task according to the database operation script; the dependency information is precondition information that the first task depends on when executing;
[0032] The acquisition module is further used to acquire scheduling information of a second task according to the dependency information, where the second task is a predecessor task of the first task;
[0033] A scheduling module is used to trigger the database operation script to execute the first task after determining that the execution of the second task is completed according to the scheduling information.
[0034] In a third aspect, an embodiment of the present application provides a server comprising a processor and a memory, wherein the processor and the memory are interconnected, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method described in the first aspect.
[0035] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method described in the first aspect.
[0036] To summarize, the server can obtain the database operation script corresponding to the first task, and obtain the dependency information of the first task based on the database operation script; then, the server can obtain the scheduling information of the second task based on the dependency information, and after determining that the execution of the second task is completed based on the scheduling information, trigger the database operation script to run to execute the first task. Compared with the prior art, which causes the task scheduling process to manually fill in the scheduling time, resulting in the problem of low overall scheduling efficiency of the task due to the occurrence of scheduling windows, the present application adopts this process, which makes the task scheduling process more flexible and can improve the overall scheduling efficiency of the task. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0038] Figure 1 It is a flowchart of a task scheduling method provided in an embodiment of the present application;
[0039] Figure 2 It is a flowchart of another task scheduling method provided in an embodiment of the present application;
[0040] Figure 3 It is a structural diagram of a task scheduling device provided in an embodiment of the present application;
[0041] Figure 4 It is a structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION
[0042] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.
[0043] See also Figure 1 , is a flow chart of a task scheduling method provided in an embodiment of the present application. The method can be applied to a server. The server can be a server or a server cluster. Specifically, the method can include the following steps:
[0044] S101. Obtain a database operation script corresponding to a first task.
[0045] The first task may be any task to be executed. For example, the first task may be a task for operating table A, such as a running task corresponding to table A. The running task corresponding to table A may refer to a task for obtaining table A by means of data extraction and data combination. The database operation script may be a query language script such as a Hibernate Query Language (HQL) script, which is not limited in the embodiments of the present application.
[0046] S102: Obtain dependency information of the first task according to the database operation script.
[0047] Among them, the dependency information can be the precondition information that the first task depends on when executing. The dependency information of the first task can be used to obtain the scheduling log of the second task, and the second task is the predecessor task of the first task. For example, the data of Table A comes from Table B, which means that Table A depends on Table B, and Table B is the predecessor dependency of Table A, that is, the running task corresponding to Table B is the predecessor task of the running task corresponding to Table A, and the running task corresponding to Table A can only be executed after the predecessor task corresponding to Table B is executed. Generally speaking, the data of Table A does not only come from Table B, it will come from multiple tables such as B1, B2...Bn, which means that Table A depends on Tables B1, B2,...Bn, and Tables B1, B2...Bn are the predecessor dependencies of Table A, that is, the running tasks corresponding to Tables B1, B2...Bn are all predecessor tasks of the running tasks corresponding to Table A, and the running tasks corresponding to Tables A can only be executed after the running tasks corresponding to Tables B1, B2...Bn are executed.
[0048] In one embodiment, the process by which the server obtains the dependency information of the first task based on the database operation script may be as follows: the server identifies the first keyword included in the database operation script through a text recognition method; the server determines the second keyword included in the database operation script based on the first keyword, and obtains the dependency information of the first task based on the second keyword, where the second keyword is located after the first keyword. Among them, the first keyword is a keyword used to determine the second keyword. In one embodiment, the first keyword may be the library identification information corresponding to the second task, such as a library name or a library code, and the second keyword may be the table name corresponding to the second task. This means that the database operation script needs to rely on the table mentioned in the database operation script.
[0049] In one embodiment, the process of the server identifying the first keyword included in the database operation script by a text recognition method may be as follows: the server performs word segmentation processing on the database operation script to obtain a word segmentation result, and uses a keyword detection technology to identify the first keyword included in the database operation script from the word segmentation result. This method can improve the efficiency and accuracy of keyword recognition.
[0050] In one embodiment, the server performs word segmentation processing on the database operation script, and the process of obtaining the word segmentation result can be specifically as follows: calling a word segmentation toolkit (such as Jieba toolkit) to perform word segmentation processing on the database operation script to obtain the word segmentation result. In one embodiment, the server adopts keyword detection technology, and the method of identifying the first keyword included in the database operation script from the word segmentation result can be as follows: the server uses a preset feature word library to determine the location of the target feature word from the word segmentation result, and uses keyword matching technology to identify the first keyword from the text content after the location of the target feature word in the word segmentation result. Among them, the feature word library can include multiple feature words, and the feature word refers to the feature word of the database operation statement used by the database operation script. For example, the database operation statement is a structured query language (SQL) statement, and accordingly, the feature word can be where, from, join and some symbols (such as $), etc.
[0051] In an application scenario, assume that the database operation script corresponding to the task of operating table A includes the following content:
[0052] SelectB.name
[0053] ,B.type
[0054] ,C.group
[0055] From cx_XXX_safe.text_table B
[0056] Join
[0057] cx_XXX_safe.text_table1 C
[0058] Alternatively, the database operation script corresponding to the operation table A task includes the following content:
[0059] SelectB.name
[0060] ,B.type
[0061] ,C.group
[0062] From${DmSafeDatabase}.text_tableB
[0063] Join
[0064] cx_XXX_safe.text_table1 C
[0065] In order to identify the first keyword included in the database operation text, the server can perform word segmentation processing on the database operation script to obtain a word segmentation result. Specifically, the server can use a word segmentation toolkit, such as the stammer toolkit, to perform word segmentation processing on each SQL statement in the database operation script to obtain a word segmentation result. Assume that the database operation script corresponding to the task of operating table A includes the following content:
[0066] SelectB.name
[0067] ,B.type
[0068] ,C.group
[0069] From cx_XXX_safe.text_table B
[0070] After obtaining the word segmentation result, the server can use keyword detection technology to identify the first keyword included in the database operation script from the word segmentation result. Specifically, the server can use a preset feature word library to determine the location of the target feature word from the word segmentation result, and use keyword matching technology to identify the first keyword from the text content after the location of the target feature word in the word segmentation result. For example, the server can use a preset feature word library to determine the location of From from the word segmentation result, and use keyword matching technology to identify the library name cx_XXX_safe from cx_XXX_safe.text_table B.
[0071] After identifying the library name, the server can determine the second keyword included in the database operation script according to the first keyword. For example, the server can determine the subsequent table name text_table according to cx_XXX_safe.
[0072] After determining the second keyword, the server may determine the second keyword as the dependency information of the first task; or, the server may generate a task name of the second task according to the second keyword, and determine the task name of the second task as the dependency information of the first task. For example, the server may use text_table as the dependency information of the first task, or generate a task name of the second task according to text_table, and determine the task name of the second task as the dependency information of the first task.
[0073] S103: Obtain scheduling information of a second task according to the dependency information, where the second task is a predecessor task of the first task.
[0074] In one embodiment, the server may obtain the scheduling information of the second task according to the dependency information in the following manner: the server may query the scheduling log of the second task from a database, such as an Oracle database, according to the dependency information, and read the scheduling information of the second task from the scheduling log. The database may include a scheduling log for each of the plurality of tasks. The scheduling log may include the task name and scheduling information of the second task. The scheduling information may include a scheduling time, which may include a start execution time and an end execution time.
[0075] S104: After determining according to the scheduling information that the execution of the second task is completed, trigger the database operation script to run to execute the first task.
[0076] In an embodiment of the present application, when there is only one second task, the server can trigger the database operation script to execute the first task after determining that the second task has been completed according to the scheduling information and that the start execution time of the first task has arrived.
[0077] In one embodiment, the server may determine a first difference between the end execution time of the second task and the start execution time of the first task, and update the start execution time of the first task according to the first difference to obtain an updated start execution time of the first task. Specifically, the process in which the server updates the start execution time of the first task according to the first difference may be: the server advances the start execution time of the first task by a target duration, the target duration being greater than zero and less than the first difference, to obtain an updated start execution time of the first task.
[0078] In one embodiment, during the execution of the second task, if the second task is detected to be interrupted, the server can count the time points of each interruption and each re-execution of the second task; the server generates multiple candidate start execution times for the first task based on the time points of each interruption and each re-execution of the second task; the server selects one of the multiple candidate start execution times as the updated start execution time of the first task.
[0079] In one embodiment, the process of the server generating multiple candidate start execution times for the first task based on the time point of each interruption and the time point of each re-execution of the second task can be as follows: the server determines the delayed execution duration of the second task each time based on the time point of each interruption and the time point of each re-execution of the second task; the server generates multiple candidate start execution times for the first task based on the delayed execution duration of each time of the second task and the start execution time of the first task. Specifically, the server can add the delayed execution duration of each time of the second task to the start execution time of the first task to obtain multiple candidate start execution times. In one embodiment, one of the candidate start execution times can be the latest start execution time among multiple candidate start execution times, or it can also refer to the start execution time calculated according to the delayed running time with the highest frequency among multiple candidate start execution times.
[0080] visible, Figure 1 In the illustrated embodiment, the server can obtain a database operation script corresponding to a first task, and obtain dependency information of the first task based on the database operation script; then, the server can obtain scheduling information of a second task based on the dependency information, and after determining that the execution of the second task has ended based on the scheduling information, trigger the database operation script to run to execute the first task. This process makes the task scheduling process more flexible and can improve the overall scheduling efficiency of the task.
[0081] This application involves blockchain technology. For example, the database operation script corresponding to the first task can be saved in the blockchain.
[0082] See also Figure 2 , is a flow chart of another task scheduling method provided in an embodiment of the present application. The task scheduling method can be applied to the aforementioned server. Specifically, the method may include the following steps:
[0083] S201. Obtain a database operation script corresponding to a first task.
[0084] S202: Obtain dependency information of the first task according to the database operation script.
[0085] S203: Obtain scheduling information of a second task according to the dependency information, where the second task is a predecessor task of the first task.
[0086] Among them, steps S201 to S203 can be found in Figure 1 Steps S101 to S103 of the embodiment are not described in detail in the embodiment of the present application.
[0087] S204: When there are multiple second tasks, determine the second task that is executed latest among the multiple second tasks according to scheduling information of each second task in the multiple second tasks, where the scheduling information includes a start execution time and an end execution time.
[0088] S205: Update the start execution time of the first task according to the end execution time of the second task that is executed latest, to obtain the updated start execution time of the first task.
[0089] In step S204-step S205, when there are multiple second tasks, the server can determine the second task that is executed last among the multiple second tasks according to the scheduling information of each second task in the multiple second tasks, and update the start execution time of the first task according to the end execution time included in the scheduling information of the second task that is executed last, to obtain the updated start execution time of the first task. For example, all the pre-dependencies of Table A include Table B1 and Table B2. The end execution time of the running task corresponding to Table B1 is 06:00, and the end execution time of the running task corresponding to Table B2 is 07:00. Therefore, in an embodiment of the present application, the server can advance the start execution time of the running task corresponding to Table A to 07:01 according to the end execution time of the running task corresponding to Table B2, thereby avoiding the waste of time caused by the running task corresponding to Table A not running for a long time after 07:00.
[0090] In the prior art, since the scheduling time of the scheduling platform is manually set, there will be a phenomenon of time waste. For example, the running tasks corresponding to all the pre-dependencies (tables B1, B2, ... Bn) of table A have been executed, but the scheduled running tasks of table A have not yet been run, and the time of the window period in between is wasted. Therefore, using the embodiment of the present application, the server can obtain the end execution time t of the running task corresponding to the last dependency to be completed among all the pre-dependencies. max , and according to t max To set the best scheduling time for table A. The server can specifically read the end execution time of all the predecessor tasks in table A, and use the end execution time t of the last completed task of all the running tasks max , get the best start running scheduling time for table A, that is, get the best start execution time for the running task corresponding to table A. This compact connection execution scheduling strategy can avoid wasting time and prevent all pre-dependencies from running while table A is still waiting, which wastes the time of the window period in the middle.
[0091] In one embodiment, the server updates the start execution time of the first task according to the end execution time of the second task that is executed latest, and the process of obtaining the updated start execution time of the first task can be as follows: the server determines the second difference between the end execution time of the second task that is executed latest and the start execution time of the first task, and updates the start execution time of the first task according to the second difference, to obtain the updated start execution time of the first task. The second difference can be the same as the first, or different. Specifically, the process of the server updating the start execution time of the first task according to the second difference can be: the server advances the start execution time of the first task by a specified time period, and the specified time period is greater than zero and less than the second difference, to obtain the updated start execution time of the first task.
[0092] S206: After determining according to the scheduling information that the execution of the second task is completed, upon detecting that the updated start execution time of the first task has arrived, triggering the database operation script to run to execute the first task.
[0093] In an embodiment of the present application, after determining that the second task has been completed according to the scheduling information, the server can trigger the database operation script to run to execute the first task. Specifically, after determining that the second task has been completed according to the scheduling information, the server can monitor whether the updated start execution time of the first task has arrived, and when it is detected that the updated start execution time of the first task has arrived, trigger the database operation script to run to execute the first task. This process advances the scheduling time of the first task, thereby improving the overall scheduling efficiency of the task.
[0094] In one embodiment, after extracting the dependency information of the first task, the server can also query whether the preset task dependency information set includes the dependency information; if it does, the server performs a deduplication operation on the information in the task dependency information set; if it does not, the server performs an operation of obtaining the scheduling information of the second task based on the dependency information. With this process, duplicate task dependency information in the task dependency information set can be deleted to obtain more accurate task dependency information.
[0095] visible, Figure 2 In the illustrated embodiment, when there are multiple second tasks, the server can determine the second task that is executed last among the multiple second tasks based on the scheduling information of each second task among the multiple second tasks, and upon detecting that the updated start execution time of the first task has arrived, trigger the database operation script to run to execute the first task. By adopting this process, the subsequent tasks can be scheduled with the optimal scheduling time according to the scheduling time of each predecessor task among the multiple predecessor tasks, thereby improving the overall scheduling efficiency of the tasks.
[0096] See also Figure 3 , is a structural diagram of a task scheduling device provided in an embodiment of the present application. The task scheduling device can be applied to the aforementioned server. The task scheduling device may include:
[0097] The acquisition module 301 is used to acquire a database operation script corresponding to a first task, and obtain dependency information of the first task according to the database operation script; the dependency information is precondition information that the first task depends on when executing.
[0098] The acquisition module 301 is further used to acquire the scheduling information of the second task according to the dependency information, where the second task is a predecessor task of the first task.
[0099] The scheduling module 302 is used to trigger the database operation script to execute the first task after determining that the execution of the second task is completed according to the scheduling information.
[0100] In an optional implementation, the acquisition module 301 acquires the scheduling information of the second task based on the dependency information, specifically by querying the scheduling log of the second task from a database based on the dependency information, the database including the scheduling log of each task in multiple tasks; and reading the scheduling information of the second task from the scheduling log.
[0101] In an optional embodiment, the acquisition module 301 obtains the dependency information of the first task based on the database operation script, specifically by identifying the first keyword included in the database operation script through a text recognition method; determining the second keyword included in the database operation script based on the first keyword, and obtaining the dependency information of the first task based on the second keyword, wherein the second keyword is located after the first keyword.
[0102] In an optional implementation, the acquisition module 301 identifies the first keyword included in the database operation script through a text recognition method, specifically by performing word segmentation processing on the database operation script to obtain a word segmentation result; and uses keyword detection technology to identify the first keyword included in the database operation script from the word segmentation result.
[0103] In an optional embodiment, the acquisition module 301 uses keyword detection technology to identify the first keyword included in the database operation script from the word segmentation result, specifically using a preset feature word library to determine the location of the target feature word from the word segmentation result, the feature word library includes multiple feature words, and the feature word refers to the feature word of the database operation statement used by the database operation script; using keyword matching technology, the first keyword is identified from the text content after the location of the target feature word in the word segmentation result.
[0104] In an optional embodiment, the acquisition module 301 obtains the dependency information of the first task based on the second keyword, specifically determining the second keyword as the dependency information of the first task; or, generates a task name of the second task based on the second keyword, and determines the task name of the second task as the dependency information of the first task.
[0105] In an optional embodiment, the scheduling module 302 is also used to determine the second task that is executed latest among the multiple second tasks based on the scheduling information of each second task in the multiple second tasks, when there are multiple second tasks, and the scheduling information includes a start execution time and an end execution time; update the start execution time of the first task based on the end execution time of the second task that is executed latest, and obtain the updated start execution time of the first task.
[0106] In an optional implementation, the scheduling module 302 triggers the database operation script to execute the first task, specifically, when it is detected that the updated start execution time of the first task has arrived, triggers the database operation script to execute the first task.
[0107] visible, Figure 3 In the illustrated embodiment, the task scheduling device can obtain a database operation script corresponding to a first task, and obtain dependency information of the first task based on the database operation script; then, the task scheduling device can obtain scheduling information of a second task based on the dependency information, and after determining that the execution of the second task has ended based on the scheduling information, trigger the database operation script to run to execute the first task. This process makes the task scheduling process more flexible and can improve the overall scheduling efficiency of the task.
[0108] See also Figure 4 , is a schematic diagram of a server structure provided in an embodiment of the present application. The server described in this embodiment may include: one or more processors 1000 and a memory 2000. The processor 1000 and the memory 2000 may be connected via a bus.
[0109] The processor 1000 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.
[0110] The memory 2000 may be a high-speed RAM memory or a non-volatile memory, such as a disk memory. The memory 2000 is used to store a set of program codes, and the processor 1000 may call the program codes stored in the memory 2000. Specifically:
[0111] Processor 1000 is used to obtain a database operation script corresponding to a first task; obtain dependency information of the first task according to the database operation script; the dependency information is precondition information that the first task depends on when executing; obtain scheduling information of a second task according to the dependency information, the second task being a predecessor task of the first task; after determining that the execution of the second task is completed according to the scheduling information, triggering the database operation script to run to execute the first task.
[0112] In one embodiment, the processor 1000 obtains the scheduling information of the second task based on the dependency information, specifically by querying the scheduling log of the second task from a database based on the dependency information, the database including the scheduling log of each task in a plurality of tasks; and reading the scheduling information of the second task from the scheduling log.
[0113] In one embodiment, the processor 1000 obtains dependency information of the first task based on the database operation script, specifically by identifying a first keyword included in the database operation script through a text recognition method; determining a second keyword included in the database operation script based on the first keyword, and obtaining the dependency information of the first task based on the second keyword, wherein the second keyword is located after the first keyword.
[0114] In one embodiment, the processor 1000 identifies the first keyword included in the database operation script through a text recognition method, specifically by performing word segmentation processing on the database operation script to obtain a word segmentation result; and uses keyword detection technology to identify the first keyword included in the database operation script from the word segmentation result.
[0115] In one embodiment, the processor 1000 uses keyword detection technology to identify the first keyword included in the database operation script from the word segmentation result, specifically using a preset feature word library to determine the location of the target feature word from the word segmentation result, the feature word library includes multiple feature words, and the feature word refers to the feature word of the database operation statement used by the database operation script; using keyword matching technology, the first keyword is identified from the text content after the location of the target feature word in the word segmentation result.
[0116] In one embodiment, the processor 1000 obtains the dependency information of the first task based on the second keyword, specifically determining the second keyword as the dependency information of the first task; or, generates a task name of the second task based on the second keyword, and determines the task name of the second task as the dependency information of the first task.
[0117] In one embodiment, the processor 1000 is also used to determine the second task that is executed last among the multiple second tasks based on the scheduling information of each second task in the multiple second tasks, when there are multiple second tasks, and the scheduling information includes a start execution time and an end execution time; update the start execution time of the first task based on the end execution time of the second task that is executed last, to obtain an updated start execution time of the first task.
[0118] In one embodiment, the processor 1000 triggers the database operation script to execute the first task, specifically, when detecting that the updated start execution time of the first task has arrived, triggers the database operation script to execute the first task.
[0119] In a specific implementation, the processor 1000 described in the embodiment of the present application may execute Figure 1 Example Figure 2 The implementation method described in the embodiment can also execute the implementation method described in the embodiment of the present application, which will not be repeated here.
[0120] Each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of sampling hardware or sampling software functional modules.
[0121] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the computer-readable storage medium can be volatile or non-volatile. For example, the computer storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM). The computer-readable storage medium can mainly include a storage program area and a storage data area, wherein the storage program area can store an operating system, at least one application required for a function, etc.; the storage data area can store data created according to the use of the blockchain node, etc.
[0122] Among them, the blockchain referred to in this application is a new application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.
[0123] What is disclosed above is only a preferred embodiment of the present application, and it certainly cannot be used to limit the scope of rights of the present application. Ordinary technicians in this field can understand that all or part of the processes of implementing the above embodiment and equivalent changes made according to the claims of the present application are still within the scope covered by the present application.
Claims
1. A task scheduling method, characterized in that: include: Obtain the database operation script corresponding to the first task; Obtain dependency information of the first task according to a first keyword included in the database operation script; the dependency information is precondition information that the first task depends on when executing; the first keyword is library identification information corresponding to the second task; the method of obtaining the dependency information is as follows: identify the first keyword included in the database operation script by a text recognition method, determine the second keyword included in the database operation script according to the first keyword, and obtain the dependency information of the first task according to the second keyword, the second keyword is located after the first keyword, and the second keyword is the table name corresponding to the second task; Acquire scheduling information of a second task according to the dependency information, where the second task is a predecessor task of the first task; After determining according to the scheduling information that the execution of the second task is completed, triggering the database operation script to run to execute the first task; Wherein, during the execution of the second task, if the second task is detected to be interrupted, then the time point of each interruption of the second task and the time point of each re-execution are counted; Determine the delayed execution time of each time of the second task according to the time point of each interruption and the time point of each re-execution of the second task; generate multiple candidate start execution times for the first task according to the delayed execution time of each time of the second task and the start execution time of the first task; the multiple candidate start execution times are obtained by adding the delayed execution time of each time of the second task to the start execution time of the first task respectively; One of the candidate execution start times is selected from the multiple candidate execution start times as the updated execution start time of the first task.
2. The method according to claim 1, characterized in that The acquiring the scheduling information of the second task according to the dependency information includes: Querying a scheduling log of the second task from a database according to the dependency information, wherein the database includes a scheduling log of each task in a plurality of tasks; The scheduling information of the second task is read from the scheduling log.
3. The method according to claim 1, characterized in that: The step of identifying the first keyword included in the database operation script by a text recognition method includes: Performing word segmentation processing on the database operation script to obtain a word segmentation result; A keyword detection technology is used to identify a first keyword included in the database operation script from the word segmentation result.
4. The method according to claim 3, characterized in that: The keyword detection technology is used to identify the first keyword included in the database operation script from the word segmentation result, including: Using a preset feature word library to determine the location of a target feature word from the word segmentation result, the feature word library includes a plurality of feature words, and the feature word refers to a feature word of a database operation statement used by the database operation script; The keyword matching technology is used to identify the first keyword from the text content after the position of the target feature word in the word segmentation result.
5. The method according to any one of claims 1 to 4, characterized in that: The obtaining the dependency information of the first task according to the second keyword includes: determining the second keyword as dependency information of the first task; or, A task name of a second task is generated according to the second keyword, and the task name of the second task is determined as dependency information of the first task.
6. The method according to claim 1, characterized in that The method further comprises: When there are multiple second tasks, determining the second task that is executed last among the multiple second tasks according to the scheduling information of each second task in the multiple second tasks, wherein the scheduling information includes the start execution time and the end execution time; updating the start execution time of the first task according to the end execution time of the second task that is executed the latest, to obtain the updated start execution time of the first task; The triggering the database operation script to run to execute the first task includes: When it is detected that the updated start execution time of the first task arrives, the database operation script is triggered to run to execute the first task.
7. A task scheduling device, characterized in that: include: An acquisition module, used to acquire a database operation script corresponding to a first task, and obtain dependency information of the first task according to a first keyword included in the database operation script; The dependency information is the precondition information that the first task depends on when executing; the first keyword is the library identification information corresponding to the second task; the method of obtaining the dependency information is as follows: identifying the first keyword included in the database operation script by a text recognition method, determining the second keyword included in the database operation script according to the first keyword, and obtaining the dependency information of the first task according to the second keyword, the second keyword is located after the first keyword, and the second keyword is the table name corresponding to the second task; The acquisition module is further used to acquire scheduling information of a second task according to the dependency information, where the second task is a predecessor task of the first task; A scheduling module, configured to trigger the database operation script to execute the first task after determining according to the scheduling information that the execution of the second task is completed; The scheduling module is further configured to count the time point of each interruption of the second task and the time point of each re-execution of the second task if an interruption of the second task is detected during the execution of the second task; Determining a delayed execution duration of each second task according to a time point of each interruption and a time point of each re-execution of the second task; Based on the delayed execution duration of each time of the second task and the start execution time of the first task, multiple candidate start execution times are generated for the first task; one of the candidate start execution times is selected from the multiple candidate start execution times as the updated start execution time of the first task; the multiple candidate start execution times are obtained by adding the delayed execution duration of each time of the second task to the start execution time of the first task.
8. A server, characterized in that: The method comprises a processor and a memory, wherein the processor and the memory are connected to each other, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method according to any one of claims 1 to 6.
Citation Information
Patent Citations
ETL (Extraction-Transformation-Loading) dispatching system and method for error-correction restarting and automatic-judgment starting
CN103034554A
Data management method and device
CN111475504A