Processing method and device for blood relationship simulation execution discovery delay risk of scheduling task, equipment and storage medium

Abstract

The invention discloses a processing method for discovering a delay risk through blood relationship simulation execution of a scheduling task. The processing method comprises the following steps: establishing a task blood relationship of a plurality of tasks; generating a task time consumption list, wherein the task time consumption list comprises the average time consumption of each task; when a task completion event notification of a task is obtained, calculating estimated completion time of the remaining task based on the task blood relationship and the task time consumption list; high-priority tasks in the remaining tasks are identified, and required completion time of the high-priority tasks is obtained; and comparing the pre-estimated completion time and the required completion time of the high-priority task, and sending out an alarm notification based on a comparison result. According to the processing method, the task blood relationship is established and the task time consumption list is generated, so that simulation execution and pre-estimation management of the scheduling task in the big data field are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly relates to a processing method, device, equipment and storage medium for discovering latency risks through blood relationship simulation execution of scheduling tasks. Background Art

[0002] In the field of big data, scheduling tasks are a core component of data processing and analysis. Every day, big data teams need to run a large number of scheduling tasks to clean, process, and summarize data, and finally generate the results required for business analysis. Taking the big data team of an online car-hailing platform as an example, nearly 3,000 scheduling tasks need to be processed every day, and there are usually complex dependencies between these tasks. For example:

[0003] Task 1: Calculate the online duration of each driver in a day.

[0004] Task 2: Calculate the number of orders completed by each driver in a day.

[0005] Task 3: Depends on Task 1 and Task 2, and calculates the industry common metric TPH (average number of orders per hour of a driver).

[0006] Each task is marked with three priorities: high, medium, and low according to its business importance and time requirement. For high-priority tasks, the team will set corresponding monitoring alarms to ensure that the tasks are completed on time. For example, high-priority Task 3 is required to be completed before 06:00. If it fails to be completed before this time point, the system will automatically issue an alarm.

[0007] However, the applicant's research found that there is a significant defect in the existing monitoring mechanism: the alarm is usually triggered after the task has timed out. This means that even if the system issues an alarm, the task delay has already occurred and cannot be adjusted or remedied. For example, assume that Task 3 is required to be completed before 06:00, and it usually completes around 05:40. However, if the running time of its upstream Task 1 is extended by 1 hour due to performance fluctuations of some nodes in the big data cluster, then Task 2 and its subsequent Task 3 will also be delayed by 1 hour accordingly. Finally, Task 3 is completed at 06:40, and when the system issues an alarm at 06:00, the task delay has already been irreparable.

[0008] This kind of delay not only affects the timeliness of the business, but also may cause obstacles to subsequent data analysis and decision-making processes. Therefore, the existing monitoring mechanism cannot meet the requirement of early warning of task delay risks. Summary of the Invention

[0009] In order to overcome the above technical defects, the present invention provides a processing method, device, equipment and storage medium for discovering latency risks through blood relationship simulation execution of scheduling tasks.

[0010] To solve the above problems, the present invention is implemented according to the following technical solutions:

[0011] In a first aspect, the present invention provides a processing method for simulating the execution of task lineage to discover latency risks in task scheduling, including the following steps:

[0012] Establish the task lineage relationships of multiple tasks, where the task lineage relationships are the dependencies and data flows between multiple tasks;

[0013] Generate a task duration list, where the task duration list includes the average duration of each task;

[0014] When a task completion event notification of a task is obtained, based on the task lineage relationships and the task duration list, calculate the estimated completion time of the remaining tasks;

[0015] Identify high-priority tasks among the remaining tasks and obtain the required completion time of the high-priority tasks;

[0016] Compare the estimated completion time and the required completion time of the high-priority tasks, and send an alarm notification based on the comparison result.

[0017] Combined with the first aspect, the present invention also provides a first preferred implementation manner of the first aspect. Specifically, the task lineage relationships include pre-tasks and post-tasks, and the post-tasks depend on the pre-tasks.

[0018] Combined with the first aspect, the present invention also provides a second preferred implementation manner of the first aspect. Specifically, the task lineage relationships adopt a splay tree structure, and each node of the splay tree structure represents a task, and the edges represent the dependencies between tasks.

[0019] Combined with the first aspect, the present invention also provides a third preferred implementation manner of the first aspect. Specifically, the average duration of each task is calculated by calculating the average duration of each task to complete the task within a preset time range.

[0020] Combined with the first aspect, the present invention also provides a fourth preferred implementation manner of the first aspect. Specifically, based on the task lineage relationships and the task duration list, calculating the estimated completion time of the remaining tasks specifically includes:

[0021] Based on the task lineage relationships, identify all post-tasks that depend on the tasks that generate the task completion event notification;

[0022] Based on the dependencies in the task lineage relationships, the task duration list, and the current time, calculate the estimated completion time of the post-tasks.

[0023] In a second aspect, the present invention further provides a processing device for scheduling tasks to discover latency risks through lineage simulation execution, including:

[0024] A building module for building the task lineage relationships of multiple tasks, where the task lineage relationships are the dependency relationships and data flows between multiple tasks;

[0025] A generating module for generating a task duration list, where the task duration list includes the average duration of each task;

[0026] A calculating module for calculating the estimated completion time of the remaining tasks based on the task lineage relationships and the task duration list when a task completion event notification of a task is obtained;

[0027] An identifying module for identifying high-priority tasks among the remaining tasks and obtaining the required completion time of the high-priority tasks;

[0028] A comparing module for comparing the estimated completion time and the required completion time of the high-priority tasks and sending an alarm notification based on the comparison result.

[0029] Combined with the second aspect, the present invention further provides a first preferred implementation manner of the second aspect. Specifically, the task lineage relationships include pre-tasks and post-tasks, and the post-tasks depend on the pre-tasks;

[0030] The task lineage relationships adopt a splay tree structure, and each node of the splay tree structure represents a task, and the edges represent the dependency relationships between tasks.

[0031] Combined with the second aspect, the present invention further provides a second preferred implementation manner of the second aspect. Specifically, the calculating module calculates the estimated completion time of the remaining tasks based on the task lineage relationships and the task duration list, which specifically includes:

[0032] Based on the task lineage relationships, identifying all post-tasks that depend on the tasks generating the task completion event notification;

[0033] Based on the dependency relationships in the task lineage relationships, the task duration list, and the current time, calculating the estimated completion time of the post-tasks.

[0034] In a third aspect, the present invention further provides an electronic device, where the electronic device includes:

[0035] At least one processor; and a memory communicatively connected to the at least one processor;

[0036] Wherein, the memory stores a computer program executable by the at least one processor, and the computer program is

[0037] The at least one processor executes to enable the at least one processor to execute a processing method for simulating the execution of a task pedigree to discover latency risks as described in the first aspect.

[0038] Fourthly, the present invention also provides a computer-readable storage medium storing a computer program,

[0039] The computer program is used to enable a processor to implement a processing method for simulating the execution of a task pedigree to discover latency risks as described in the first aspect when executed.

[0040] Compared with the prior art, the beneficial effects of the present invention are:

[0041] The present invention provides a processing method for simulating the execution of a task pedigree to discover latency risks in task scheduling, including the following steps: establishing a task pedigree relationship for multiple tasks, where the task pedigree relationship is the dependency relationship and data flow between multiple tasks; generating a task time-consuming list, where the task time-consuming list includes the average time-consuming of each task; when a task completion event notification of a task is obtained, based on the task pedigree relationship and the task time-consuming list, calculating the estimated completion time of the remaining tasks; identifying high-priority tasks among the remaining tasks, and obtaining the required completion time of the high-priority tasks; comparing the estimated completion time and the required completion time of the high-priority tasks, and sending an alarm notification based on the comparison result.

[0042] The processing method of the present invention realizes the simulated execution and estimated management of task scheduling in the big data field by establishing a task pedigree relationship and generating a task time-consuming list. This method can update the estimated completion time of the remaining tasks in real time during the actual execution of the tasks and perform special monitoring on high-priority tasks. By comparing the estimated completion time and the required completion time, this method can identify potential latency risks in advance and send an alarm, thereby allowing the big data team to take measures in a timely manner, optimize resource allocation and task scheduling, and improve the reliability of task execution and the timeliness of business decisions. This method not only enhances the management ability of task latency risks, but also supports the business continuity of the enterprise and the accuracy of decision-making, significantly improving the efficiency and effectiveness of the big data team in task scheduling and monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The following further details the specific embodiments of the present invention with reference to the accompanying drawings, where:

[0044] Figure 1 is a technical flowchart of a processing method for simulating the execution of a task pedigree to discover latency risks in task scheduling according to an embodiment of the present invention;

[0045] Figure 2It is the algorithm flowchart of a processing method for discovering delay risks in blood relationship simulation execution of scheduling tasks in an embodiment of the present invention;

[0046] Figure 3 It is the extended tree structure diagram of the task blood relationship in an embodiment of the present invention;

[0047] Figure 4 It is the module diagram of a processing device for discovering delay risks in blood relationship simulation execution of scheduling tasks in an embodiment of the present invention;

[0048] Figure 5 It is the structural schematic diagram of an electronic device in an embodiment of the present invention. Detailed implementation manners

[0049] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0050] In the field of big data, scheduling tasks are a core component of data processing and analysis. Every day, big data teams need to run a large number of scheduling tasks to clean, process, and summarize data, and finally generate the results required for business analysis. Each task is marked with three priorities: high, medium, and low according to its business importance and time limit requirements. For high-priority tasks, the team will set corresponding monitoring alarms to ensure that the tasks are completed on time. For example, high-priority task 3 is required to be completed before 06:00. If it fails to be completed before this time point, the system will automatically issue an alarm.

[0051] However, the applicant's research found that there is a significant defect in the existing monitoring mechanism: the alarm is usually triggered after the task has timed out. This means that even if the system issues an alarm, the delay of the task has already occurred and cannot be adjusted or remedied. For example, assume that task 3 is required to be completed before 06:00, and it usually completes around 05:40. However, if the running time of its upstream task 1 is extended by 1 hour due to performance fluctuations of some nodes in the big data cluster, then task 2 and its subsequent task 3 will also be delayed by 1 hour accordingly. Finally, task 3 is completed at 06:40, and when the system issues an alarm at 06:00, the delay of the task has already been irreparable. This kind of delay not only affects the timeliness of the business, but also may cause obstacles to the subsequent data analysis and decision-making processes. Therefore, the existing monitoring mechanism cannot meet the demand for early warning of task delay risks.

[0052] The present invention provides a processing method for simulating the execution of task lineage to discover latency risks in scheduling tasks, including the following steps: establishing the task lineage relationships of multiple tasks, where the task lineage relationships are the dependency relationships and data flows between multiple tasks; generating a task time-consuming list, where the task time-consuming list includes the average time consumption of each task; when a task completion event notification of a task is obtained, based on the task lineage relationships and the task time-consuming list, calculating the estimated completion time of the remaining tasks; identifying high-priority tasks among the remaining tasks, and obtaining the required completion time of the high-priority tasks; comparing the estimated completion time and the required completion time of the high-priority tasks, and sending an alarm notification based on the comparison result.

[0053] The processing method of the present invention realizes the simulated execution and estimated management of scheduling tasks in the big data field by establishing task lineage relationships and generating a task time-consuming list. This method can update the estimated completion time of the remaining tasks in real time during the actual execution of tasks, and conduct special monitoring for high-priority tasks. By comparing the estimated completion time and the required completion time, this method can identify potential latency risks in advance and send an alarm, thereby allowing the big data team to take timely measures to optimize resource allocation and task scheduling, improving the reliability of task execution and the timeliness of business decisions. This method not only enhances the management ability of task latency risks, but also supports the business continuity of enterprises and the accuracy of decision-making, significantly improving the efficiency and effectiveness of the big data team in task scheduling and monitoring.

[0054] Therefore, referring to Figure 1 , the embodiment of the present invention provides a flowchart of a processing method for simulating the execution of task lineage to discover latency risks in scheduling tasks. This method can be executed by a processing device for simulating the execution of task lineage to discover latency risks in scheduling tasks, and this device can be implemented in the form of hardware and / or software, and this device can be configured in a computer. As Figure 1 shown, this method includes:

[0055] S100: Establish the task lineage relationships of multiple tasks, where the task lineage relationships are the dependency relationships and data flows between multiple tasks.

[0056] In the present invention, the task lineage relationship refers to the dependency relationships and data flows between multiple tasks. By establishing the task lineage relationships, the sequence and data transfer path between tasks can be clearly understood. This step is the basis of the entire method, because subsequent latency risk analysis and alarm notifications rely on the dependency relationships between tasks.

[0057] In the present invention, the task lineage relationships include pre-tasks and post-tasks, and the post-tasks depend on the pre-tasks. As Figure 3As shown, the task lineage relationship adopts an splay tree structure. Each node in the splay tree structure represents a task, and the edges represent the dependency relationships between tasks.

[0058] In the splay tree structure:

[0059] Node: Each node represents a task.

[0060] Edge: The edge represents the dependency relationship between tasks, that is, the relationship between the pre-task and the post-task. The post-task depends on the completion of the pre-task to start execution.

[0061] Splay operation: When accessing a certain task, the task node is moved to the root of the tree through the splay operation, thereby improving the subsequent access efficiency.

[0062] The advantage of the splay tree structure lies in dynamic adjustment and efficient query, which is especially suitable for frequent task status updates and dependency relationship queries in the task scheduling scenario.

[0063] S200: Generate a task time-consuming list, and the task time-consuming list includes the average time-consuming of each task.

[0064] In a specific implementation, the task time-consuming list records the average time-consuming of each task. This can be obtained through historical data statistics or updated by real-time monitoring of the task execution time. These time-consuming data can be stored in a table or database for subsequent estimation calculations.

[0065] In a specific implementation, the average time-consuming of each task is calculated by calculating the average time-consuming of each task to complete the task within a preset time range. Exemplarily, the preset time range can be 5 days, 10 days, 15 days, etc.

[0066] S300: When receiving the task completion event notification of a task, based on the task lineage relationship and the task time-consuming list, calculate the estimated completion time of the remaining tasks.

[0067] In a specific implementation, based on the task lineage relationship and the task time-consuming list, calculating the estimated completion time of the remaining tasks specifically includes:

[0068] S310: Based on the task lineage relationship, identify all the post-tasks that depend on the completion of the generated task event notification.

[0069] In a specific implementation, the task lineage relationship is a visual representation of the dependency relationship between tasks, used to clarify the sequence and data flow of tasks. In task scheduling, when a task is completed, it is necessary to identify all the post-tasks that depend on the completion of this task to start. This step is achieved by analyzing the task lineage relationship graph, where the downstream nodes of each task node are its post-tasks.

[0070] Implementation method

[0071] Identify subsequent tasks: When receiving a task completion event notification, starting from this task node, traverse all its downstream nodes. For example, when task B is completed, identify task C as a subsequent task; if task C has a downstream task D, then task D is also a subsequent task.

[0072] Support multi-level dependencies: Support the identification of multi-level subsequent tasks, that is, not only identify directly dependent tasks. For example, when task B is completed, identify task C and task D (both task D and task C are respectively dependent on task B).

[0073] S320: Calculate the estimated completion time of subsequent tasks based on the dependency relationships in the task lineage, the task duration list, and the current time.

[0074] In this step, this step recursively calculates the estimated completion time of subsequent tasks based on the task lineage, the task duration list, and the current time. The task duration list records the average duration of each task and is used to estimate the execution time of the task.

[0075] Implementation method

[0076] Obtain the current time: Record the time when the task completion event notification arrives as the starting point for calculation.

[0077] Calculate the estimated completion time of direct subsequent tasks: For each direct subsequent task, its estimated completion time = current time + the average duration of this task. For example, task B is completed at 10:00, and the average duration of task C is 15 minutes, then the estimated completion time of task C is 10:15.

[0078] Recursively calculate the estimated completion time of indirect subsequent tasks: For each indirect subsequent task, its estimated completion time = the estimated completion time of its directly dependent task + the average duration of this task.

[0079] For example, the estimated completion time of task C is 10:15, and the average duration of task D is 5 minutes, then the estimated completion time of task D is 10:20.

[0080] Consider task concurrency: If multiple tasks can be executed in parallel, the estimated completion time should consider the concurrent execution of tasks.

[0081] For example, after task B is completed, task C and task D can be executed in parallel, then the estimated completion times of task C and task D are 10:15 and 10:05 respectively.

[0082] S400: Identify high-priority tasks among the remaining tasks and obtain the required completion times of high-priority tasks.

[0083] In the present invention, a high-priority task refers to a task that has a greater impact on the business, and its delay may lead to serious consequences. This step requires identifying these tasks and obtaining their required completion times.

[0084] Specifically, when defining tasks, assign priorities to each task (such as high priority, medium priority, low priority). Set clear required completion times for high-priority tasks. Store the task priorities and required completion times in a database.

[0085] S500: Compare the estimated completion time and the required completion time of high-priority tasks, and send an alarm notification based on the comparison result.

[0086] It can be understood that if the estimated completion time of a high-priority task is later than the required completion time, an alarm notification is sent. The alarm notification can be sent to relevant personnel by means such as email, text message, instant messaging tool, etc.

[0087] By comparing the estimated completion time and the required completion time of high-priority tasks. If the estimated completion time is later than the required completion time, an alarm notification is triggered. Options such as email, text message, instant messaging tool, etc. can be selected.

[0088] In summary, the processing method of the present invention can dynamically calculate the estimated completion time of tasks and timely discover the delay risk of high-priority tasks by establishing task lineage relationships and task duration lists. Through the alarm notification mechanism, relevant personnel can take measures in advance, optimize task scheduling, and reduce business losses.

[0089] Such as Figure 4 shown, a processing device for discovering delay risks through blood relationship simulation execution of scheduling tasks according to the present invention includes:

[0090] A building module, which is used to establish task lineage relationships of multiple tasks, and the task lineage relationships are the dependency relationships and data flows between multiple tasks;

[0091] A generating module, which is used to generate a task duration list, and the task duration list includes the average duration of each task;

[0092] A calculating module, which is used to calculate the estimated completion time of the remaining tasks based on the task lineage relationships and the task duration list when obtaining a task completion event notification of a task;

[0093] An identifying module, which is used to identify high-priority tasks among the remaining tasks and obtain the required completion times of the high-priority tasks;

[0094] A comparison module for comparing the estimated completion time and the required completion time of high-priority tasks and issuing an alarm notification based on the comparison result.

[0095] Preferably, the task lineage includes a pre-task and a post-task, and the post-task depends on the pre-task; the task lineage adopts a splay tree structure, and each node of the splay tree structure represents a task, and the edge represents the dependency relationship between tasks.

[0096] Preferably, the calculation module calculates the estimated completion time of the remaining tasks based on the task lineage and the task duration list, specifically including:

[0097] Based on the task lineage, identify all post-tasks that depend on the task that generates the task completion event notification;

[0098] Based on the dependency relationship in the task lineage, the task duration list and the current time, calculate the estimated completion time of the post-task.

[0099] Figure 5 FIG. shows a schematic structural diagram of an electronic device 10 that can be used to implement an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device (such as a helmet, glasses, a watch, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0100] As Figure 5 shown, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. Among them, the memory stores a computer program executable by at least one processor, and the processor 11 can execute various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0101] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disc, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0102] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a processing method for solving the discovery of latency risks in pedigree simulation execution for task scheduling.

[0103] In some embodiments, a processing method for solving the discovery of latency risks in pedigree simulation execution for task scheduling can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the processing method for solving the discovery of latency risks in pedigree simulation execution for task scheduling described above can be executed. Alternatively, in other embodiments, the processor 11 can be configured to execute a processing method for solving the discovery of latency risks in pedigree simulation execution for task scheduling in any other suitable manner (e.g., by means of firmware).

[0104] The various embodiments of the systems and technologies described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs, which can be executed and / or interpreted on a programmable system including at least one programmable processor, the programmable processor can be a special or general programmable processor, and can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0105] A computer program for implementing the method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer programs are executed by the processor, the functions / operations specified in the flowchart and / or block diagram are implemented. The computer programs can be executed entirely on the machine, partially on the machine, executed partially on the machine as an independent software package and partially on a remote machine, or executed entirely on a remote machine or server.

[0106] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include electrical connections based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0107] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the electronic device. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0108] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), blockchain network, and the Internet.

[0109] A computing system can include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The client - server relationship is created by computer programs running on respective computers and having a client - server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0110] An embodiment of the present invention also provides a computer program product, which includes a computer program that, when executed by a processor, implements a processing method for scheduling tasks to discover latency risks in lineage simulation execution as provided in the embodiment of the present invention.

[0111] In the process of implementing the computer program product, computer program code for performing the operations of the present invention can be written in one or more programming languages or combinations thereof. The programming languages include object - oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user computer, partially on the user computer, executed as a stand - alone software package, partially on the user computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user computer through any type of network - including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., by connecting through an Internet service provider via the Internet).

[0112] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and no limitation is made herein.

[0113] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub - combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A method for processing delay risks discovered by lineage simulation execution of scheduling tasks, characterized in that: The steps include: Establishing a task lineage relationship of multiple tasks, wherein the task lineage relationship is a dependency relationship and data flow between the multiple tasks; Generate a task time consumption list, wherein the task time consumption list includes an average time consumption of each task; When a task completion event notification is obtained, the estimated completion time of the remaining tasks is calculated based on the task lineage relationship and the task time-consuming list; Identify high-priority tasks among the remaining tasks and obtain the required completion time of the high-priority tasks; Compare the estimated completion time and required completion time of high-priority tasks, and issue an alarm notification based on the comparison results.

2. A method for processing delay risks discovered by lineage simulation execution for scheduling tasks according to claim 1, characterized in that: The task blood relationship includes a predecessor task and a successor task, and the successor task depends on the predecessor task.

3. A method for processing delay risk discovered by lineage simulation execution for scheduling tasks according to claim 2, characterized in that: The task kinship relationship adopts a spreading tree structure, each node of the spreading tree structure represents a task, and the edges represent the dependency relationship between tasks.

4. A method for processing delay risk discovered by lineage simulation execution for scheduling tasks according to claim 2, characterized in that: The average time taken for each task is calculated by calculating the average time taken for each task to be completed within the preset time range.

5. A method for processing delay risk discovered by lineage simulation execution for scheduling tasks according to claim 2, characterized in that: Based on the task relationship and the task time list, the estimated completion time of the remaining tasks is calculated, including: Based on the task lineage relationship, identifying all subsequent tasks that depend on the task that generates the task completion event notification; Based on the dependency relationship in the task lineage relationship, the task time-consuming list and the current time, the estimated completion time of the subsequent task is calculated.

6. A processing device for discovering delay risks by lineage simulation execution of scheduling tasks, characterized in that: include: An establishment module, which is used to establish a task lineage relationship of multiple tasks, wherein the task lineage relationship is a dependency relationship and data flow between the multiple tasks; A generation module, which is used to generate a task time consumption list, wherein the task time consumption list includes an average time consumption of each task; A calculation module, which is used to calculate the estimated completion time of the remaining tasks based on the task lineage relationship and the task time-consuming list when obtaining the task completion event notification of the task; An identification module, which is used to identify high-priority tasks among the remaining tasks and obtain the required completion time of the high-priority tasks; The comparison module is used to compare the estimated completion time and the required completion time of high-priority tasks, and issue an alarm notification based on the comparison result.

7. A processing device for discovering delay risk by lineage simulation execution of scheduling tasks according to claim 6, characterized in that: The task blood relationship includes a predecessor task and a successor task, and the successor task depends on the predecessor task; The task kinship relationship adopts a spreading tree structure, each node of the spreading tree structure represents a task, and the edges represent the dependency relationship between tasks.

8. The processing device for discovering delay risk by lineage simulation execution for scheduling tasks according to claim 6, characterized in that: The calculation module calculates the estimated completion time of the remaining tasks based on the task blood relationship and the task time-consuming list, specifically including: Based on the task lineage relationship, identifying all subsequent tasks that depend on the task that generates the task completion event notification; Based on the dependency relationship in the task lineage relationship, the task time-consuming list and the current time, the estimated completion time of the subsequent task is calculated.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively coupled to the at least one processor; In which, the memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute a processing method for discovering delay risks by performing lineage simulation for scheduling tasks as described in any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that: The computer readable storage medium stores a computer program, The computer program is used to enable a processor to implement a method for processing lineage simulation execution for scheduling tasks to discover delay risks as described in any one of claims 1 to 5 when executing the program.

Images