Aging optimization method and device

Abstract

This invention relates to the field of big data batch processing technology, and discloses a timeliness optimization method and apparatus. The method includes acquiring scheduling task data from a scheduling platform; performing dependency processing on the scheduling task data to obtain task dependency data for each scheduling task; sequentially obtaining the full-link task relationship of each target task from the task dependency data based on a target task list; and filtering the target tasks based on the full-link task relationship and the target task list, using the historical runtime of the scheduling tasks and a preset timeliness baseline, to obtain a list of target tasks to be optimized. This application solves the technical problem in the prior art of optimizing the timeliness of big data batch processing in terms of ease of batch and speed, and achieves the technical effect of simple, fast, and efficient optimization of the timeliness of big data batch processing.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of big data batch processing technology, and in particular to a timeliness optimization method and apparatus. Background Technology

[0002] In existing data platforms, big data is typically processed and stored in layers. A series of tasks are deployed through an automated scheduling platform for processing. Typically, the running logs and scheduling dependency DAG graphs of a task can be viewed on the scheduling platform or asset management platform.

[0003] However, the financial industry has high requirements for data timeliness in some scenarios. DAG graphs are usually limited to viewing the upstream and downstream relationships of tasks and are difficult to use directly for timeliness optimization analysis in a batch and quickly. When data processing personnel locate timeliness issues, they usually need to manually extract information from the original scheduling configuration information and operation logs for analysis, which is quite difficult. Therefore, there is a lack of effective and easy-to-use rules and tools to support timeliness optimization. Summary of the Invention

[0004] This invention provides a timeliness optimization method and apparatus, which solves the technical problem in the prior art of optimizing the timeliness of big data batch processing, which is difficult to perform batch and fast timeliness optimization.

[0005] In a first aspect, embodiments of the present invention provide a timeliness optimization method, the timeliness optimization method comprising:

[0006] Obtain scheduling task data from the scheduling platform, wherein the scheduling task data includes the configuration data of the scheduling platform when performing task scheduling and the operation log data of the scheduling platform;

[0007] The scheduling task data is processed to obtain task dependency data for each scheduling task, wherein the task dependency data represents the logical order in which each scheduling task occurs.

[0008] Based on the target task list, the full-link task relationship of each target task is obtained sequentially from the task dependency relationship data. The full-link task relationship includes the topic, level, subsequent tasks and preceding tasks of the target task. The level of the last target task run in the target task list is recorded as level 0.

[0009] Based on the full-link task relationship and the target task list, the target tasks are filtered using the historical runtime efficiency of the scheduled tasks and the preset time baseline to obtain a target task list to be optimized.

[0010] Secondly, embodiments of the present invention also provide a timeliness optimization device, the timeliness optimization device comprising:

[0011] The data acquisition unit is used to acquire scheduling task data in the scheduling platform, wherein the scheduling task data is the configuration data of the scheduling platform when performing task scheduling and the operation log data of the scheduling platform;

[0012] The data processing unit is used to perform dependency processing on the scheduling task data to obtain task dependency data for each scheduling task, wherein the task dependency data represents the logical order in which each scheduling task occurs.

[0013] The task acquisition unit is used to sequentially acquire the full-link task relationship of each target task from the task dependency relationship data based on the target task list. The full-link task relationship includes the topic, level, subsequent tasks and preceding tasks of the target task. The level of the last target task running in the target task list is recorded as level 0.

[0014] The task determination unit is used to filter the target tasks based on the full-link task relationship and the target task list, using the historical runtime efficiency of the scheduled tasks and a preset time baseline, to obtain the target task list to be optimized.

[0015] Thirdly, embodiments of the present invention also provide a timeliness optimization device, the timeliness optimization device comprising:

[0016] One or more processors;

[0017] Storage device for storing one or more programs;

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement any of the time optimization methods as described in the first aspect of the embodiments of the present invention.

[0019] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the time-efficiency optimization methods as described in the first aspect of the present invention.

[0020] Fifthly, embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements any of the time optimization methods described in the first aspect of the present invention.

[0021] This invention discloses a timeliness optimization method and apparatus. The method includes acquiring scheduling task data from a scheduling platform; performing dependency processing on the scheduling task data to obtain task dependency data for each scheduling task; sequentially obtaining the full-link task relationship of each target task from the task dependency data based on a target task list; and filtering target tasks based on the full-link task relationship and the target task list, using historical runtime efficiency of scheduling tasks and a preset timeliness baseline to obtain a list of target tasks to be optimized. This application obtains the logical order of events between the acquired scheduling task data through dependency processing, then uses the user-input target task list to obtain the full-link task relationship representing the upstream and downstream tasks of the target tasks from the task dependency data, and finally uses the full-link task relationship to filter target tasks and find the target tasks that need timeliness optimization. This application solves the technical problem of difficulty in batch and rapid timeliness optimization when optimizing the timeliness of big data batch processing in the prior art, achieving a simple, fast, and efficient technical effect for optimizing the timeliness of big data batch processing. Attached Figure Description

[0022] Figure 1 This is a flowchart of a timeliness optimization method provided in an embodiment of the present invention;

[0023] Figure 2 This is a flowchart of another timeliness optimization method provided in an embodiment of the present invention;

[0024] Figure 3 This is a flowchart of another timeliness optimization method provided by an embodiment of the present invention;

[0025] Figure 4 This is a flowchart of another timeliness optimization method provided by an embodiment of the present invention;

[0026] Figure 5 This is a structural diagram of a time-efficiency optimization device provided in an embodiment of the present invention;

[0027] Figure 6 This is a schematic diagram of a time-optimization device provided in an embodiment of the present invention. Detailed Implementation

[0028] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0029] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this invention are used to distinguish different objects, not to limit a specific order. The various embodiments of this invention described below can be performed individually or in combination with each other; the embodiments of this invention do not impose specific limitations in this regard.

[0030] Figure 1 This is a flowchart illustrating a timeliness optimization method provided in an embodiment of the present invention. This timeliness optimization method is applicable to all scenarios requiring timeliness optimization in big data batch processing. The method can be executed by a timeliness optimization device, which can be implemented in hardware and / or software and is generally integrated into a server. The acquisition, storage, use, and processing of data in this application comply with relevant national laws and regulations.

[0031] like Figure 1 As shown, this timeliness optimization method specifically includes the following steps:

[0032] S101, Obtain scheduling task data from the scheduling platform, wherein the scheduling task data consists of the configuration data of the scheduling platform when performing task scheduling and the running log data of the scheduling platform.

[0033] Specifically, the scheduling task data obtained from the scheduling platform mainly includes the following: (1) basic information of the scheduling task, including but not limited to task name, task number, system to which it belongs, scheduling frequency, scheduling start time, number of retry attempts, priority, execution script, execution command, etc.; (2) input and output event information of the scheduling task, including but not limited to task name, task number, event type, event name, etc.; (3) running instance information of the scheduling task, including but not limited to task name, task number, scheduling date, start time, end time, running status, running duration, number of runs, etc.

[0034] It should be noted that the obtained scheduling task data does not include the specific detailed parameters of each scheduling task during its execution; only the data of each scheduling task during the scheduling process is obtained.

[0035] S102, perform dependency processing on the scheduling task data to obtain the task dependency data of each scheduling task, wherein the task dependency data represents the logical order in which each scheduling task occurs.

[0036] Specifically, the scheduling task data obtained from the scheduling platform consists of the input and output events of each scheduling task. The data format is <task name, event type, event name>, where the event type is [output event, input event]. For example, (1) Task name = JOB_A; event type = output event; event name = EVENT_JOB_A_OK. (2) Task name = JOB_B; event type = input event; event name = EVENT_JOB_A_OK.

[0037] The scheduling task data is processed according to preset processing rules to obtain the task dependency data of each scheduling task. The data format is <predecessor task, successor task, event name>. For example, the specific rule is: if the output event of JOB_A is equal to the input event of JOB_B, then JOB_A is the predecessor task of JOB_B, and JOB_B is the successor task of JOB_A. After processing by the preset processing rules, the task dependency data obtained is: predecessor task = JOB_A; successor task = JOB_B; event name = EVENT_JOB_A_OK.

[0038] S103. Based on the target task list, obtain the full-link task relationship of each target task from the task dependency relationship data in sequence. The full-link task relationship includes the topic, level, subsequent tasks and preceding tasks of the target task. The level of the last target task to be run in the target task list is recorded as level 0.

[0039] Specifically, the target task list consists of target tasks pre-input by the user to be analyzed. It's important to note that this list represents a single type of target task, potentially a subset of millions of target tasks. After obtaining the scheduling task dependency data, further processing is required to generate the complete upstream task relationships for each target task. Specifically, the user-input target task list is retrieved, and the last executed target task is identified as the starting point for tracing the source, designated as level 0. Then, a recursive execution method is used. For each level i task, the task dependency data is used to match "subsequent tasks" to find all "previous tasks." If the preceding task of the task is empty, the process ends; otherwise, it is recorded as the (i+1)th level task, and a full-link task relationship data is obtained, in the format: <target task, level, subsequent task, preceding task>, for example: (1) target task = JOB_T, level = 1, subsequent task = JOB_T, preceding task = JOB_B; (2) target task = JOB_T, level = 2, subsequent task = JOB_B, preceding task = JOB_A.

[0040] S104. Based on the full-link task relationship and the target task list, the target tasks are filtered using the historical runtime efficiency of the scheduled tasks and the preset time baseline to obtain the target task list to be optimized.

[0041] Specifically, after obtaining the full-link task relationship and the target task list, the target tasks are filtered using the historical runtime of the scheduled tasks and the preset time baseline. The target tasks whose estimated runtime exceeds the historical runtime of the scheduled tasks or the preset time baseline are selected as target tasks to be optimized. Then, a list of target tasks to be optimized is generated and displayed to the user to indicate that there are target tasks in the target task list whose runtime can be optimized.

[0042] This application processes the acquired scheduling task data to determine the logical order of events between various scheduling tasks. Then, using a user-input target task list, it extracts the full-link task relationships representing the upstream and downstream tasks of each target task from the task dependency data. Finally, it uses these full-link task relationships to filter target tasks and identify those requiring timeliness optimization. This application solves the technical problem in existing technologies regarding the difficulty and speed of optimizing the timeliness of large-scale batch processing, achieving a simple, fast, and efficient technical effect for optimizing the timeliness of large-scale batch processing.

[0043] Based on the above technical solutions, Figure 2 This is a flowchart of another timeliness optimization method provided by an embodiment of the present invention, such as... Figure 2 As shown, after S101 obtains the scheduling task data from the scheduling platform, this timeliness optimization method also includes:

[0044] S201, receive and store the target task parameters input by the user, wherein the target task parameters include: a list of target tasks, the scheduling time to be analyzed, the expected timeframe for the completion of all target tasks, and a preset timeframe baseline.

[0045] Specifically, users can input target task parameters through human-computer interaction units such as keyboards, mice, and touch screens. These parameters include a list of target tasks to be analyzed, the scheduling time to be analyzed (e.g., the specific time range or time period to be analyzed), the user's expected completion time for the target task analysis, and a preset time baseline. The preset time baseline includes the source table to data time baseline, the single task runtime time baseline, and the critical path runtime time baseline.

[0046] After receiving the target task parameters input by the user, these parameters are stored and recorded. When the user inputs target task parameters again, the previously entered parameters can be used as default values ​​for the user to choose from. It should be noted that the expected timeframe for completing all target tasks can be a default value set based on expert experience, or a value customized by the user as needed; there are no restrictions here.

[0047] Optionally, the methods for calculating the historical runtime of scheduled tasks include:

[0048] Based on the full-link task relationship, the statistical indicators of the completion time of each target task within the preset time are calculated, as well as the running time of completing all target tasks within the preset time are calculated. The statistical indicators include minimum, mean, median, standard deviation, achievement rate, process capability index, and sigma.

[0049] Specifically, the preset time can be defined as 10 days, or it can be customized according to user needs. The Process Capability Index (CPK) is a convenient way to represent the level of process quality; its essential function is to reflect the process pass rate. Sigma is a unit in statistics that represents the standard deviation from the mean. The calculation of the running time to complete all target tasks within the preset time is the average running time of all target tasks within the preset time.

[0050] Based on the above technical solutions, Figure 3 This is a flowchart of another timeliness optimization method provided by an embodiment of the present invention, such as... Figure 3 As shown, S104 specifically includes:

[0051] S301 filters source table to data tasks and critical paths of target tasks based on the full-link task relationship and the target task list.

[0052] Specifically, to more accurately filter out the target tasks that need optimization and make runtime optimization more precise, the target task list can be filtered in different parts. The first is the source table to data task, which refers to the process of the upstream system unloading data into files and then sending it to the file buffer of the data platform. The file buffer can be regarded as the database. The second is to find the critical path of the target tasks and determine whether there are any tasks that need optimization on the target tasks on the critical path.

[0053] S302, using the historical runtime efficiency of the scheduling task and the preset time baseline, the source table to data task, the task in the critical path of the target task, and all target tasks in the target task list are filtered to obtain the target task list to be optimized.

[0054] Specifically, based on the historical runtime of the scheduled tasks and the preset time baseline, the target tasks are filtered from three parts: tasks from the source table to the target task, tasks in the critical path of the target task, and all target tasks in the target task list. Tasks that exceed the historical runtime of their respective scheduled tasks or the preset time baseline are selected as target tasks to be optimized, and a list of target tasks to be optimized is generated.

[0055] Optionally, S301, based on the end-to-end task relationship and the target task list, the source table to data task and the target task critical path in the target task are filtered, including:

[0056] S1 takes the most source task in each task path in the full-link task relationship as the source table to data task, where the source table to data task refers to the task in the target task that transmits source table data to the database.

[0057] Specifically, the source table serves as the starting point of the data computation path. When the runtime objective (i.e., the desired outcome) of the target task at the end is determined, the runtime efficiency of the source table determines the available time margin for processing and computation. Based on the target task and its upstream end-to-end task relationships, the most fundamental task is filtered out, which is the source table data transmission task (i.e., the aforementioned source table to data task), and its completion time is recorded as the source table to data time.

[0058] S2, calculate the total running time of all target tasks on each task path in the full-link task relationship, and take the task path with the largest total running time as the critical path of the target task.

[0059] Specifically, when the execution time from the source table to the data is determined, the final completion time of the target task is affected by the execution time and waiting time of the entire task chain. Waiting time is influenced by factors such as scheduling dependencies and resource utilization, and can vary significantly across different scheduling dates. Ideally, if resources are sufficient and the system is free of anomalies, the waiting time of the critical path can approach zero. Execution time, on the other hand, can be statistically calculated based on historical execution records after excluding anomalies; it represents a relatively stable execution time under normal resource conditions. The total execution time of all target tasks on each task path is calculated, and after being sorted in reverse order, the task path with the largest total execution time is identified as the target task's critical path based on runtime. Without path optimization or single-job execution time optimization, the total execution time of the critical path is a necessary and uncompressible time.

[0060] Optionally, in step S302, the source table to data tasks are filtered using the historical runtime efficiency of the scheduled tasks and the preset time efficiency baseline to obtain a list of target tasks to be optimized, including: filtering the time efficiency of source table to data tasks using the source table to data time efficiency data in the historical runtime efficiency of the scheduled tasks and the source table to data time efficiency baseline in the preset time efficiency baseline; identifying source table to data tasks whose time efficiency exceeds the source table to data time efficiency data or the source table to data time efficiency baseline as target tasks to be optimized; and arranging the running time of the target tasks to be optimized from longest to shortest to obtain a list of target tasks to be optimized.

[0061] Specifically, the historical data on the arrival time of all source tables in the historical runtime of the scheduling task (i.e., the aforementioned data on the arrival time of the source tables) is obtained. All tasks that exceed the data on the arrival time of the source tables or the baseline requirements for the arrival time of the source tables are filtered out. The tasks are then sorted in reverse order of runtime to obtain the list of source tables that need to be improved and their priorities, i.e., the aforementioned list of target tasks to be optimized.

[0062] Optionally, in step S302, tasks in the critical path of the target task are filtered using the historical runtime of the scheduled task and a preset runtime baseline to obtain a list of target tasks to be optimized, including: comparing the total runtime of the target task's critical path with the runtime baseline of the critical path in the preset runtime baseline; if the total runtime of the target task's critical path is greater than the runtime baseline of the critical path, then all target tasks in the target task's critical path are taken as target tasks to be optimized; and the runtime of the target tasks to be optimized is arranged from longest to shortest to obtain a list of target tasks to be optimized.

[0063] Specifically, after determining the critical path of the target task, the total running time of all tasks on each path is calculated, and the total running time is compared with the critical path running time baseline in the preset time baseline. If the total running time of the critical path exceeds the critical path running time baseline, it indicates that the critical path needs to be optimized. Then, all target tasks on the critical path are taken as target tasks to be optimized and arranged in reverse order of running time to obtain a list of target tasks to be optimized.

[0064] Optionally, in step S302, all target tasks in the target task list are filtered using the historical runtime of the scheduled task and a preset runtime baseline to obtain a list of target tasks to be optimized, including: obtaining the runtime of each target task in the historical runtime of the scheduled task; comparing the runtime of the target task with the runtime baseline of a single task in the preset runtime baseline; if the runtime of the target task exceeds the runtime baseline, the target task is designated as a target task to be optimized; and arranging the runtime of the target tasks to be optimized from longest to shortest to obtain a list of target tasks to be optimized.

[0065] Specifically, the execution time of a single task refers to the duration from the start to the end of a single task. Based on the target task and its upstream task relationships, a list of all upstream tasks for each target task can be obtained. The execution time of each target task in the historical execution time of the scheduled tasks is taken as the execution time of the target task. This time is compared with the baseline execution time of a single task, and all target tasks that exceed the baseline requirement are filtered out. These target tasks are then sorted in descending order of execution time to obtain a list of target tasks to be optimized.

[0066] Based on the above technical solutions, Figure 4 This is a flowchart of another timeliness optimization method provided by an embodiment of the present invention, such as... Figure 4 As shown, after obtaining the list of target tasks to be optimized in S104, this timeliness optimization method also includes:

[0067] S401 visualizes the list of target tasks to be optimized in a table format.

[0068] Specifically, once the list of target tasks to be optimized is obtained, it can be visualized so that users can intuitively understand which target tasks can be optimized in terms of timeliness, thereby improving the overall completion time of the target tasks to be analyzed. Specifically, the target tasks to be optimized can be displayed in the following types of tables: (1) Optimization of source table to data tasks. This table will list all source table to data tasks that the target task depends on and whose runtime efficiency does not meet the baseline requirements, including the basic information of the source table, the timeliness information of the source table, and the optimization priority of the source table; (2) Optimization of tasks with long running time. This table will list all upstream tasks that the target task depends on and whose running time does not meet the baseline requirements, including the basic information of the task, the running time information of the task, and the optimization priority of the task; (3) Optimization of critical paths. This table will list all task paths for which the total running time of the target task does not meet the baseline requirements, including the path relationship, the basic information of the tasks on the path, and the running time information of the tasks on the path; (4) Optimization effect. This table will list the historical trend of the above indicators and the results compared with the user's optimization records to reflect the optimization effect.

[0069] In this embodiment of the invention, big data batch processing technology is used to collect, clean, and integrate the data required for analysis. Finally, based on the target task parameters input by the user, such as the target task to be analyzed and various timeliness baseline requirements, the timeliness optimization results are automatically and flexibly displayed. By providing an end-to-end service, the complex analysis process is shielded from the user, allowing them to focus on the issues to be improved, thereby increasing analysis efficiency and accuracy. This avoids the difficulty of using DAG diagrams to view the upstream and downstream relationships of tasks, which makes it difficult to batch and quickly apply the data directly to timeliness optimization analysis.

[0070] Figure 5 This is a structural diagram of a time-efficiency optimization device provided in an embodiment of the present invention, as shown below. Figure 5 As shown, the timeliness optimization device includes:

[0071] The data acquisition unit 51 is used to acquire scheduling task data in the scheduling platform, wherein the scheduling task data is the configuration data of the scheduling platform when performing task scheduling and the operation log data of the scheduling platform.

[0072] Data processing unit 52 is used to perform dependency processing on scheduling task data to obtain task dependency data for each scheduling task, wherein the task dependency data represents the logical order in which each scheduling task occurs.

[0073] The task acquisition unit 53 is used to sequentially acquire the full-link task relationship of each target task from the task dependency relationship data based on the target task list. The full-link task relationship includes the topic, level, subsequent tasks and preceding tasks of the target task. The level of the last target task to be run in the target task list is recorded as level 0.

[0074] The task determination unit 54 is used to filter target tasks based on the full-link task relationship and the target task list, using the historical runtime efficiency of the scheduled tasks and the preset time baseline, to obtain the target task list to be optimized.

[0075] Optionally, before the data acquisition unit 51 acquires the scheduling task data from the scheduling platform, the timeliness optimization device further includes:

[0076] The parameter receiving unit is used to receive and store the target task parameters input by the user. The target task parameters include: a list of target tasks, the scheduling time to be analyzed, the expected time for all target tasks to be completed, and a preset time baseline.

[0077] Optionally, the task determination unit 54 includes:

[0078] The first filtering subunit is used to filter source table to data tasks and critical paths of target tasks based on the full-link task relationship and the target task list.

[0079] The second filtering subunit is used to filter the source table to data tasks, the tasks in the critical path of the target task, and all target tasks in the target task list by using the historical runtime efficiency of the scheduling task and the preset time efficiency baseline, so as to obtain the target task list to be optimized.

[0080] Optionally, the first filtering sub-unit is specifically used for:

[0081] In the end-to-end task relationship, the most original task in each task path is taken as the source table to data task. The source table to data task refers to the task in the target task that transmits source table data to the database.

[0082] Calculate the total execution time of all target tasks on each task path in the entire task chain, and take the task path with the largest total execution time as the critical path of the target task.

[0083] Optionally, the second filtering subunit is specifically used for:

[0084] The timeliness of source table to data tasks is filtered by using the source table to data timeliness data in the historical runtime timeliness of the scheduled tasks and the source table to data timeliness baseline in the preset timeliness baseline. Source table to data tasks whose timeliness exceeds the source table to data timeliness data or the source table to data timeliness baseline are selected as target tasks to be optimized.

[0085] Arrange the execution times of the target tasks to be optimized from longest to shortest to obtain a list of target tasks to be optimized.

[0086] Optionally, the second filtering subunit is also used for:

[0087] Compare the total execution time of the critical path of the target task with the critical path execution time baseline in the preset time baseline;

[0088] If the total execution time of the critical path of the target task is greater than the critical path execution efficiency baseline, then all target tasks in the critical path of the target task will be regarded as target tasks to be optimized.

[0089] Arrange the execution times of the target tasks to be optimized from longest to shortest to obtain a list of target tasks to be optimized.

[0090] Optionally, the second filtering subunit is also used for:

[0091] Obtain the execution time of each target task in the historical runtime of the scheduled task;

[0092] Compare the execution time of the target task with the execution time baseline of a single task in the preset time baseline;

[0093] If the execution time of the target task exceeds the runtime efficiency baseline, then the target task will be designated as a target task to be optimized.

[0094] Arrange the execution times of the target tasks to be optimized from longest to shortest to obtain a list of target tasks to be optimized.

[0095] Optionally, after the task determination unit 54 obtains the list of target tasks to be optimized, the timeliness optimization device further includes:

[0096] The visualization unit is used to visually display the list of target tasks to be optimized in tabular form.

[0097] The device provided in this embodiment of the invention has the same implementation principle and technical effect as the aforementioned method embodiment. For the sake of brevity, any parts not mentioned in the device embodiment can be referred to the corresponding content in the aforementioned method embodiment.

[0098] The time efficiency optimization device provided in this embodiment of the invention has the same technical features as the time efficiency optimization method provided in the above embodiments, so it can also solve the same technical problems and achieve the same technical effects.

[0099] Figure 6 This is a schematic diagram of the structure of a time-optimization device provided in an embodiment of the present invention, as shown below. Figure 6 As shown, the timeliness optimization device includes a processor 61, a memory 62, an input device 63, and an output device 64; the number of processors 61 in the timeliness optimization device can be one or more. Figure 6 Taking a processor 61 as an example; the processor 61, memory 62, input device 63, and output device 64 in the timeliness optimization device can be connected via a bus or other means. Figure 6 Taking the example of a connection between China and Israel via a bus.

[0100] The memory 62, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the timeliness optimization method in this embodiment of the invention (e.g., the data acquisition unit 51, data processing unit 52, task acquisition unit 53, and task determination unit 54 in the timeliness optimization device). The processor 61 executes various functional applications and data processing of the timeliness optimization device by running the software programs, instructions, and modules stored in the memory 62, thereby implementing the aforementioned timeliness optimization method.

[0101] The memory 62 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a given function; the data storage area may store data created based on terminal usage. Furthermore, the memory 62 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory, or other non-volatile solid-state storage device. In some instances, the memory 62 may further include memory remotely located relative to the processor 61, which can be connected to a time-optimization device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0102] Input device 63 can be used to receive input digital or character information, and to generate key signal inputs related to user settings and function control of the timeliness optimization device. Output device 64 may include display devices such as a display screen.

[0103] This invention also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform a time optimization method.

[0104] Specifically, the timeliness optimization method includes:

[0105] Obtain scheduling task data from the scheduling platform, which includes the configuration data of the scheduling platform when performing task scheduling and the running log data of the scheduling platform;

[0106] Dependency processing is performed on the scheduling task data to obtain the task dependency data of each scheduling task, where the task dependency data represents the logical order in which each scheduling task occurs;

[0107] Based on the target task list, the full-link task relationship of each target task is obtained sequentially from the task dependency data. The full-link task relationship includes the target task's theme, level, subsequent tasks, and preceding tasks. The level of the last target task run in the target task list is recorded as level 0.

[0108] Based on the full-link task relationships and the target task list, the target tasks are filtered using the historical runtime efficiency of the scheduled tasks and the preset time baseline to obtain the target task list to be optimized.

[0109] Of course, the computer-executable instructions provided in the embodiments of the present invention are not limited to the method operations described above, but can also perform related operations in the time optimization method provided in any embodiment of the present invention.

[0110] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0111] It is worth noting that in the embodiments of the above-mentioned timeliness optimization device, the various units and modules included are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0112] This invention also provides a computer program product, which includes computer-executable instructions that, when executed by a computer processor, are used to perform the time optimization method provided in any embodiment of this invention.

[0113] Of course, the computer program product provided in the embodiments of this application has computer-executable instructions that are not limited to the method operations described above, and can also perform related operations in the methods provided in any embodiment of the present invention.

[0114] In the description of the embodiments of the present invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the present invention based on the specific circumstances.

[0115] Finally, it should be noted that the above are merely preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. An aging optimization method, characterized by, The aging optimization method comprises: obtaining scheduling task data in a scheduling platform, wherein the scheduling task data is configuration data of the scheduling platform when performing task scheduling and running log data of the scheduling platform; processing the scheduling task data to obtain task dependency relationship data of each scheduling task, wherein the task dependency relationship data represents a logical order in which each scheduling task occurs; obtaining full-link task relationship of each target task from the task dependency relationship data based on a target task list, wherein the full-link task relationship includes a subject, a level, a post-task and a pre-task of the target task, and the level of a last running target task in the target task list is recorded as the 0th level; filtering the target task based on the full-link task relationship and the target task list, using scheduling task historical running aging and a preset aging baseline, to obtain a to-be-optimized target task list; the filtering of the target task based on the full-link task relationship and the target task list, using the scheduling task historical running aging and the preset aging baseline, to obtain the to-be-optimized target task list comprises: filtering a source table-to-number task and a target task critical path in the target task based on the full-link task relationship and the target task list; filtering the source table-to-number task, the task in the target task critical path and all target tasks in the target task list based on the scheduling task historical running aging and the preset aging baseline, to obtain the to-be-optimized target task list of the target task; wherein the filtering of the source table-to-number task in the target task based on the full-link task relationship and the target task list comprises: regarding a most source task in each task path in the full-link task relationship as the source table-to-number task.

2. The aging optimization method of claim 1, wherein, Before obtaining the scheduling task data in the scheduling platform, the aging optimization method further comprises: receiving and storing target task parameters input by a user, wherein the target task parameters include the target task list, a scheduling time to be analyzed, an expected aging of completion of all the target tasks and the preset aging baseline.

3. The aging optimization method of claim 1, wherein, the filtering of the source table-to-number task and the target task critical path in the target task based on the full-link task relationship and the target task list comprises: the source table-to-number task refers to a task that transmits source table data to a database in the target task; calculating a total running time consumption of all target tasks on each task path in the full-link task relationship, and regarding a task path with the largest total running time consumption as the target task critical path.

4. The aging optimization method of claim 1, wherein, the filtering of the source table-to-number task based on the scheduling task historical running aging and the preset aging baseline, to obtain the to-be-optimized target task list of the target task comprises: Screen the time limit of the source table to number task by using the source table to number time limit data in the scheduling task history running time limit and the source table to number time limit baseline in the preset time limit baseline, and take the source table to number task whose time limit exceeds the source table to number time limit data or the source table to number time limit baseline in the source table to number task as a target task to be optimized; Arrange the running time consumption of the target task to be optimized from long to short to obtain the target task to be optimized list.

5. The aging optimization method of claim 1, wherein, Screen the task in the target task critical path by using the scheduling task history running time limit and the preset time limit baseline to obtain the target task to be optimized list of the target task includes: Compare the running time consumption sum of the target task critical path with the critical path running time limit baseline in the preset time limit baseline; If the running time consumption sum of the target task critical path is greater than the critical path running time limit baseline, take all the target tasks in the target task critical path as the target task to be optimized; Arrange the running time consumption of the target task to be optimized from long to short to obtain the target task to be optimized list.

6. The aging optimization method of claim 1, wherein, Screen all the target tasks in the target task list by using the scheduling task history running time limit and the preset time limit baseline to obtain the target task to be optimized list of the target task includes: Obtain the running time consumption of each target task in the scheduling task history running time limit; Compare the running time consumption of the target task with the single task running time limit baseline in the preset time limit baseline; If the running time consumption of the target task exceeds the running time limit baseline, take the target task as the target task to be optimized; Arrange the running time consumption of the target task to be optimized from long to short to obtain the target task to be optimized list.

7. The aging optimization method of claim 1, wherein, After obtaining the target task to be optimized list, the time limit optimization method further includes: Visually display the target task to be optimized list in the form of a table.

8. The aging optimization method of claim 1, wherein, The calculation method of the scheduling task history running time limit includes: Calculate the statistical index of the time limit of completing each target task within a preset time and calculate the running time consumption of completing all the target tasks within the preset time based on the full link task relationship, wherein the statistical index includes minimum value, mean value, median, standard deviation, standard rate, process capability index and sigma.

9. An aging optimization apparatus characterized by comprising: The time limit optimization device includes: A data acquisition unit is configured to acquire scheduling task data in a scheduling platform, wherein the scheduling task data is configuration data of the scheduling platform when performing task scheduling and running log data of the scheduling platform; A data processing unit is configured to process the scheduling task data in a dependency relationship to obtain task dependency relationship data of each scheduling task, wherein the task dependency relationship data represents a logical order of each scheduling task; The task acquisition unit is configured to acquire full-link task relations of each target task in sequence from the task dependency data based on a target task list, wherein the full-link task relations include a subject, a level, a post-task and a pre-task of the target task, and the level of a last-run target task in the target task list is recorded as level 0; The to-be-optimized task determination unit is configured to filter the target tasks based on the full-link task relations and the target task list, and obtain a to-be-optimized target task list by using a scheduling task historical running time efficiency and a preset time efficiency baseline; The to-be-optimized task determination unit comprises: A first filtering sub-unit configured to filter source table-to-number tasks and target task critical paths in the target tasks based on the full-link task relations and the target task list; A second filtering sub-unit configured to filter the source table-to-number tasks, tasks in the target task critical paths and all target tasks in the target task list by using the scheduling task historical running time efficiency and the preset time efficiency baseline, and obtain the to-be-optimized target task list of the target tasks. The first filtering sub-unit is specifically configured to: regard a most source task in each task path in the full-link task relations as the source table-to-number task.

10. An aging optimization device, characterized by, The time efficiency optimization device comprises: one or more processors; a storage device configured to store one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the time efficiency optimization method according to any one of claims 1-8.

11. A computer readable storage medium having stored thereon a computer program, characterized in that The program is executed by the processor to implement the time efficiency optimization method according to any one of claims 1-8.

12. A computer program product, characterised in that, The program is executed by the processor to implement the time efficiency optimization method according to any one of claims 1-8. The program is executed by the processor to implement the time efficiency optimization method according to any one of claims 1-8.

Images