Batch job duration analysis method and device, equipment, medium and program product

Abstract

The invention provides a batch job duration analysis method which can be applied to the technical field of big data, and relates to the field of distribution and artificial intelligence. The analysis method comprises the following steps: in response to an analysis request of batch job duration, obtaining a job list of a target application, execution duration of each job in the job list and a calling relationship among the jobs from a distributed batch processing database; determining the weight of each job according to the execution duration; determining an execution path of each job according to the calling relationship; determining a shortest weighted value path of each job from the execution paths based on the weight; and determining the batch job duration of each job according to the shortest weighted value path. The invention further provides a batch operation duration analysis device and equipment, a medium and a program product.

Description

Technical Field

[0001] This disclosure relates to the fields of big data, distributed systems, and artificial intelligence, and more specifically to a method, apparatus, device, medium, and program product for analyzing batch operation duration. Background Technology

[0002] In the field of distributed technology, as mainframe services are shifted to off-frame environments, Distributed Batch Framework (DBF) is increasingly hosting core applications. Based on DBF runtime statistics, it can be inferred that all applications consume significant resources. By using batch tasks, batch jobs can be started on demand, improving the resource utilization of distributed batch programs.

[0003] During batch task execution, the batch execution window is divided by estimating the waiting time and invocation path of individual jobs. Currently, the assessment of the waiting time of individual job invocation mainly relies on human experience, resulting in low accuracy. Summary of the Invention

[0004] In view of the above problems, this disclosure provides analysis methods, apparatus, devices, media and program products for improving the accuracy of batch execution window division and batch job duration.

[0005] According to a first aspect of this disclosure, a method for analyzing batch job duration is provided, comprising: in response to a batch job duration analysis request, obtaining a job list of a target application, the execution duration of each job in the job list, and the calling relationship between the jobs from a distributed batch processing database; determining the weight of each job based on the execution duration; determining the execution path of each job based on the calling relationship; determining the shortest weighted path for each job from the execution paths based on the weight; and determining the batch job duration of each job based on the shortest weighted path.

[0006] According to embodiments of this disclosure, the above-mentioned method of obtaining a job list of a target application, the execution duration of each job in the job list, and the calling relationship between the jobs from a distributed batch processing database includes: obtaining a job information data table of batch jobs from the distributed batch processing database; the job information data table includes the job identifier of each individual job, the application identifier to which it belongs, the job identifier of the preceding job, the application identifier to which the preceding job belongs, and the execution duration; determining the application identifier to be the job identifier of the individual job of the target application based on the job identifier of each individual job and the application identifier to which it belongs; obtaining a job list of the target application based on the job identifier of the individual job of the target application; determining the preceding job of each job in the job list based on the job identifier of the preceding job of each individual job; determining the application identifier to which the preceding job of each job in the job list belongs based on the application identifier to which the preceding job of each individual job belongs; and determining the calling relationship between the jobs in the job list based on the preceding job of each job in the job list and the application identifier to which the preceding job of each job in the job list belongs.

[0007] According to embodiments of this disclosure, determining the calling relationship between jobs in the job list based on the preceding jobs of each job in the job list and the application identifiers of the preceding jobs of each job in the job list includes: when the application identifier of the preceding job of a job in the job list is not the target application, setting the preceding job of the job in the job list as the root node; when the application identifier of the preceding job of a job in the job list is the target application, obtaining the calling relationship between the jobs in the job list and the preceding jobs; and determining the calling relationship between each job in the job list based on the calling relationship between the jobs in the job list and the preceding jobs.

[0008] According to embodiments of this disclosure, each job has at least one execution path. The process of determining an execution path for a job in the job list based on the aforementioned calling relationship includes: taking a job in the job list as the current job, determining the preceding job of the current job based on the aforementioned calling relationship; if the preceding job of the current job is not the aforementioned root node, taking the preceding job of the current job as the current job, updating the current job, and continuing to determine the preceding job of the current job based on the aforementioned calling relationship until the preceding job of the current job is the aforementioned root node; if the preceding job of the current job is the aforementioned root node, the forward jobs included between the jobs in the job list and the preceding job of the current job constitute an execution path for a job in the job list.

[0009] According to embodiments of this disclosure, determining the shortest weighted path for each job from the execution paths based on the aforementioned weights includes: for a job in the job list, calculating the weighted value of each execution path of all execution paths of the job in the job list based on the aforementioned weights; selecting the execution path with the smallest weighted value from all execution paths of the job in the job list, and taking the execution path with the smallest weighted value as the shortest weighted path of the job in the job list; and determining the shortest weighted path for each job based on the shortest weighted path of the job in the job list.

[0010] According to embodiments of this disclosure, calculating the weighted value of all execution paths of a job in a job list based on the aforementioned weights includes: determining the weight of the preceding job included in each execution path of a job in a job list based on the aforementioned weights; and determining the weighted value of each execution path of a job in a job list based on the weight of the preceding job included in each execution path of a job in a job list.

[0011] According to embodiments of this disclosure, the batch processing time of each job is determined based on the execution time of the preceding job included in the shortest weighted path.

[0012] A second aspect of this disclosure provides a batch job duration analysis apparatus, comprising: a data acquisition module, configured to, in response to a batch job duration analysis request, acquire from a distributed batch processing database a job list of a target application, the execution duration of each job in the job list, and the calling relationship between the jobs; a weight determination module, configured to determine the weight of each job based on the execution duration, wherein the weight of each job is inversely proportional to the execution duration; a path determination module, configured to determine the execution path of each job based on the calling relationship; a target path determination module, configured to determine the shortest weighted path for each job from the execution paths based on the weight; and a duration determination module, configured to determine the batch job duration of each job based on the shortest weighted path.

[0013] A third aspect of this disclosure provides an electronic device comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0014] A fourth aspect of this disclosure also provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0015] The fifth aspect of this disclosure also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method.

[0016] According to embodiments of this disclosure, the execution path with the smallest weighted value is selected from multiple execution paths of each job as the shortest weighted path of the job. The smaller the weighted value of the execution path, the longer the execution time of the job. Therefore, the shortest weighted path of the job is actually the longest waiting time required for the job to be executed during batch job execution. By using the longest waiting time required for the job to be executed as the batch job duration of each job, each job can be reliably executed during batch job execution. This makes it easier for operation and maintenance personnel to estimate the waiting time of individual jobs in batch jobs. Furthermore, the shortest weighted path of the job can facilitate operation and maintenance personnel to trace the time distribution of batch job execution paths, thereby improving the accuracy of batch execution window division. Attached Figure Description

[0017] The foregoing contents, as well as other objects, features, and advantages of this disclosure, will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0018] Figure 1 The illustration schematically depicts an application scenario of a method, apparatus, device, medium, and program product for analyzing batch job duration according to embodiments of the present disclosure;

[0019] Figure 2 A flowchart illustrating a method for analyzing batch job duration according to an embodiment of the present disclosure is shown schematically.

[0020] Figure 3 This illustration schematically shows a diagram illustrating multiple job invocation relationships according to embodiments of the present disclosure;

[0021] Figure 4 A directed weight graph according to an embodiment of the present disclosure is illustrated schematically;

[0022] Figure 5 The illustration schematically depicts a scenario diagram of a method for analyzing batch job duration according to another embodiment of the present disclosure;

[0023] Figure 6 A directed graph data structure diagram according to an embodiment of the present disclosure is illustrated schematically;

[0024] Figure 7 A flowchart illustrating a method for analyzing batch job duration according to another embodiment of this disclosure is shown schematically;

[0025] Figure 8 A schematic block diagram illustrating a device for analyzing batch job duration according to an embodiment of the present disclosure is shown; and

[0026] Figure 9A block diagram of an electronic device suitable for implementing a batch job duration analysis method according to an embodiment of the present disclosure is shown schematically. Detailed Implementation

[0027] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0028] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0029] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0030] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0031] In the technical solution disclosed herein, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with relevant laws, regulations, and standards, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding operation entry points are provided for users to choose to authorize or refuse.

[0032] In scenarios involving automated decision-making using personal information, the methods, devices, and systems provided in this disclosure all offer users corresponding entry points for choosing to agree to or reject the automated decision-making results. If the user chooses to reject, the process proceeds to the expert decision-making stage. Here, "automated decision-making" refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests, or economic, health, and credit status through computer programs, and then making a decision. Here, "expert decision-making" refers to the activity of making decisions by personnel who specialize in a particular field, possess specialized experience, knowledge, and skills, and have reached a certain level of professional expertise.

[0033] Distributed batch frameworks are batch execution tools for open platforms. On this platform, distributed applications are deployed independently; however, during batch execution, there is significant interaction between batch jobs across applications. In production, accurately defining batch execution windows requires estimating the waiting time and call path for each individual job after batch execution begins. Currently, assessing the waiting time for a single job to be called primarily relies on manual experience.

[0034] The embodiments of this disclosure provide a method for analyzing batch job duration, including: in response to a batch job duration analysis request, obtaining a job list of a target application, the execution duration of each job in the job list, and the calling relationship between each job from a distributed batch processing database; determining the weight of each job based on the execution duration; determining the execution path of each job based on the calling relationship; determining the shortest weighted path for each job from the execution paths based on the weight; and determining the batch job duration of each job based on the shortest weighted path.

[0035] According to embodiments of this disclosure, the execution path with the smallest weighted value is selected from multiple execution paths of each job as the shortest weighted path of the job. Since the smaller the weight, the longer the execution time of the job, the shortest weighted path of the job is actually the longest waiting time required for the job to be executed during batch job execution. By using the longest waiting time required for the job to be executed as the batch job duration of each job, each job can be reliably executed during batch job execution. This makes it easier for operation and maintenance personnel to estimate the waiting time of individual jobs in batch jobs. Furthermore, the shortest weighted path of the job can facilitate operation and maintenance personnel to trace the time distribution of batch job execution paths, thereby improving the accuracy of batch execution window division.

[0036] Figure 1 The illustration schematically depicts an application scenario of a method, apparatus, device, medium, and program product for analyzing batch job duration according to embodiments of the present disclosure.

[0037] like Figure 1As shown, application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0038] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0039] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0040] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0041] It should be noted that the batch job duration analysis method provided in this embodiment can generally be executed by server 105. Correspondingly, the batch job duration analysis device provided in this embodiment can generally be located in server 105. The batch job duration analysis method provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the batch job duration analysis device provided in this embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0042] It should be understood that Figure 1The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0043] The following will be based on Figure 1 The described scene, through Figures 2-7 The method for analyzing batch job duration according to the disclosed embodiments is described in detail.

[0044] Figure 2 A flowchart illustrating a method for analyzing batch job duration according to an embodiment of the present disclosure is shown schematically.

[0045] like Figure 2 As shown, the batch job duration analysis method 200 of this embodiment includes operations S210 to S250.

[0046] In operation S210, in response to the batch job duration analysis request, the job list of the target application, the execution duration of each job in the job list, and the calling relationship between each job are obtained from the distributed batch processing database.

[0047] According to embodiments of this disclosure, when a request to analyze the duration of batch jobs is received, historical monitoring data of DBF batch jobs is used to mine job lists for each application, the execution duration of each job included in the job list, and the calling relationships between jobs in the job list of the same application from the distributed batch processing database. Based on the mined content, the duration of batch jobs for each application is analyzed. This disclosure uses the target application as an example for illustration. The analysis process for the duration of batch jobs in other applications is consistent with that of the target application and will not be repeated here.

[0048] According to embodiments of this disclosure, by utilizing historical monitoring data of DBF batch jobs to analyze the batch job duration of the target application, the reliability and authenticity of the data source are ensured, thereby making the final batch job duration more accurate.

[0049] In this embodiment of the disclosure, obtaining the job list of the target application, the execution duration of each job in the job list, and the calling relationship between the jobs from the distributed batch processing database includes: obtaining a job information data table of batch jobs from the distributed batch processing database; the job information data table includes the job identifier of each individual job, the application identifier to which it belongs, the job identifier of the preceding job, the application identifier to which the preceding job belongs, and the execution duration; determining the application identifier to be the job identifier of the individual job of the target application based on the job identifier of each individual job and the application identifier to which it belongs; obtaining the job list of the target application based on the job identifier of the individual job of the target application; determining the preceding job of each job in the job list based on the job identifier of the preceding job of each individual job; determining the application identifier to which the preceding job of each job in the job list belongs based on the application identifier to which the preceding job of each individual job belongs; and determining the calling relationship between the jobs in the job list based on the preceding job of each job in the job list and the application identifier to which the preceding job of each job in the job list belongs.

[0050] According to embodiments of this disclosure, the job information data table is traversed to find the job identifier of a single job belonging to the target application, thereby obtaining a job list for the target application. Based on the job identifier of the preceding job of each job in the job list, the preceding job of each job in the job list is found, and it is determined whether the preceding job of each job in the job list belongs to the target application. Based on the determination result, the calling relationship between the jobs in the job list is determined. The determination result can be categorized as either the preceding job of each job in the job list containing a preceding job that does not belong to the target application, or all the preceding jobs of each job in the job list belong to the target application.

[0051] Table 1. Job Information Data Table

[0052]

[0053] Taking the job information data table shown in Table 1 as an example, the following explanation will illustrate the job information data table. The job information data table is stored in the DBF database using the data structure of the JOB_INSTANCE table. In the job information data table, JOB_ID represents the job identifier of a single job; APP_ID represents the application identifier to which a single job belongs; DEPEND_JOB_ID represents the job identifier of the preceding job of a single job, and DEPEND_APP_ID represents the application identifier to which the preceding job of a single job belongs; START_TIMESTAMP represents the trigger time of a single job; and END_TIMESTAMP represents the end time of a single job. Taking a single job with the job identifier JOB0 as an example, the application to which the single job with the job identifier JOB0 belongs is the application represented by the application identifier APP0. The preceding job to the single job with the job identifier JOB0 is the single job represented by the job with the job identifier JOB1. The application to which the preceding job to the single job with the job identifier JOB0 belongs is the application represented by the application identifier APP0. The trigger time of the single job with the job identifier JOB0 is T_JOB0_JOB1_S, and the end time of the single job with the job identifier JOB0 is T_JOB0_JOB1_E. The execution time of the single job with the job identifier JOB0 is END_TIMESTAMP - START_TIMESTAMP of the current node.

[0054] According to embodiments of this disclosure, the calling relationship between each job in the job list and whether there is a preceding job that does not belong to the target application can be determined by using a job information data table, thereby enabling a more accurate calling relationship between each job in the job list.

[0055] In this embodiment of the disclosure, the calling relationship between each job in the job list is determined based on the preceding job of each job in the job list and the application identifier of the preceding job of each job in the job list. This includes: when the application identifier of the preceding job of a job in the job list is not the target application, setting the preceding job of the job in the job list as the root node; when the application identifier of the preceding job of a job in the job list is the target application, obtaining the calling relationship between the jobs in the job list and the preceding job; and determining the calling relationship between each job in the job list based on the calling relationship between the jobs in the job list and the preceding job.

[0056] According to embodiments of this disclosure, if the application identifier of the preceding job in the job list is not the target application, it indicates that the target application is making a cross-application call, which falls under the scenario of an extended application call. When the target application makes a cross-application call, the preceding job in the job list is set as the root node.

[0057] Figure 3 A schematic diagram illustrating multiple job call relationships according to embodiments of the present disclosure is shown.

[0058] like Figure 3 As shown, jobs A, B, C, D, E, F, G, H, J, X, and Y are multiple jobs included in the job list of the target application. Among them, job X is a cross-application job, and the previous job of job X is another application. For jobs whose previous job is another application, they are classified as cross-application jobs. Therefore, the previous job of job X whose previous job is another application is set as the root node of the current application.

[0059] According to embodiments of this disclosure, by determining the root node, the calling relationship of the target application is obtained, such that the calling relationship of the target application all starts from the root node, so as to calculate the batch job duration of a single job based on the calling relationship of the job list of the target application.

[0060] In operation S220, the weight of each job is determined based on its execution duration. The execution duration is inversely proportional to the weight.

[0061] In this embodiment of the disclosure, the inverse relationship between execution time and weight means that the longer the execution time, the smaller the weight. Therefore, the functional relationship between execution time and weight can be an inverse proportional function, a linear function with a negative slope and execution time as the independent variable, an exponential function with a base between 0 and 1, or a composite function with a decreasing relationship. For example, the reciprocal of the job's execution time can be used as the job's weight.

[0062] In operation S230, the execution path of each job is determined based on the calling relationship.

[0063] In this embodiment of the disclosure, each job has at least one execution path. The process of determining an execution path of a job in the job list based on the calling relationship includes: taking a job in the job list as the current job, determining the preceding job of the current job based on the calling relationship; if the preceding job of the current job is not the root node, taking the preceding job of the current job as the current job, updating the current job, and continuing to determine the preceding job of the current job based on the calling relationship until the preceding job of the current job is the root node; if the preceding job of the current job is the root node, the forward jobs included between the jobs in the job list and the preceding job of the current job constitute an execution path of a job in the job list.

[0064] like Figure 3As shown, taking task J as an example, the preceding task of task J is task E, the preceding tasks of task E are tasks G and C, the preceding task of task C is task A, the preceding task of task A is the root node, the preceding task of task G is task B, and the preceding task of task B is task A. Therefore, the execution path of task J includes the root node, task A, task C, and task E. After task E is executed, task J begins execution. The execution path of task J also includes the root node, task A, task B, task G, and task E. After task E is executed, task J begins execution. Therefore, task J has two execution paths.

[0065] According to embodiments of this disclosure, by determining the execution path of each job, the selected execution path can include the execution paths of all jobs in the job list when selecting the execution path, avoiding omissions of execution paths and making the processing results of processing the execution path more accurate.

[0066] In operation S240, based on the weights, the shortest weighted path for each job is determined from the execution path.

[0067] In this embodiment of the disclosure, determining the shortest weighted path for each job from the execution paths based on weights includes: for a job in the job list, calculating the weighted value of each execution path of all execution paths of the job in the job list based on weights; selecting the execution path with the smallest weighted value from all execution paths of the job in the job list, and taking the execution path with the smallest weighted value as the shortest weighted path of the job in the job list; and determining the shortest weighted path for each job based on the shortest weighted path of the job in the job list.

[0068] According to embodiments of this disclosure, each job in the job list of the target application has at least one execution path. Taking one execution path of each job as an example, the weighted value of the execution path is calculated based on the weight obtained in operation S220, thereby obtaining the weighted value of each execution path included in each job. The weighted values ​​of each execution path included in each job are compared, and the execution path with the smaller weighted value is selected as the shortest weighted path for that job. This yields the shortest weighted path for each job in the job list.

[0069] According to embodiments of this disclosure, by determining the shortest weighted path for each job, it is possible to quickly locate the problem when batch job execution issues occur, making it easier for maintenance personnel to trace the execution status of batch jobs.

[0070] In this embodiment of the disclosure, calculating the weighted value of all execution paths of a job in the job list based on weights includes: determining the weight of the preceding job included in each execution path of a job in the job list based on weights; and determining the weighted value of each execution path of a job in the job list based on the weight of the preceding job included in each execution path of a job in the job list.

[0071] According to embodiments of this disclosure, the execution path of each job in the job list is composed of the preceding jobs along that execution path, such as... Figure 3 As shown, this is the execution path of job J. Therefore, when calculating the weighted value of the execution path, the weights of the preceding jobs on the execution path are added together to obtain the weighted value of each execution path for a job in the job list. For example... Figure 3 As shown, taking the execution path of job J consisting of the root node, job A, job C, and job E as an example, the weighted value of the execution path of job J is the sum of the weights of job A, job C, and job E.

[0072] Figure 4 A directed weighted graph according to an embodiment of the present disclosure is illustrated schematically.

[0073] like Figure 4 As shown, a directed weighted graph can be generated by assigning weights to each job and defining their calling relationships. The directed weighted graph includes nodes a, b, c, and d. Taking an example where each node corresponds to a job, the execution time of node b is obtained using graph theory algorithms. This time can be viewed as the weight Kab of the edge from node a to node b in the directed weighted graph. The weights of other edges in the directed weighted graph are determined in the same way as the weight of the edge from node a to node b, and will not be elaborated here. Then, graph theory algorithms are applied to calculate the shortest paths from the starting point to each node in the directed weighted graph.

[0074] In the embodiments of this disclosure, since the analysis result of batch job duration determines the longest path from the starting point to each node in the directed weighted graph, and this path is taken as the critical path, and this critical path is used as the estimated batch waiting time for node jobs, the reciprocal of the execution duration is used as the weight. The shortest path result output by the graph theory algorithm is actually the execution path with the smallest weighted value. Since the execution duration is inversely proportional to the weight, the execution duration is also inversely proportional to the weighted value. The shortest path weighted value obtained for each job is actually the longest waiting time required after each job is suspended. Therefore, the shortest path result output by the graph theory algorithm is the longest path from the starting point of the directed weighted graph to each node. The graph theory algorithm selected is a shortest path algorithm, such as Dijkstra's algorithm.

[0075] According to embodiments of this disclosure, by using the calling relationships between jobs as paths in a directed weighted graph, the shortest weighted path is calculated using the principles of graph theory algorithms. Compared to existing technologies that rely on human experience, this method can obtain a more accurate shortest weighted path through scientific calculation, laying the foundation for obtaining the batch processing time of each job.

[0076] When operating S250, the batch processing time of each job is determined based on the shortest weighted path.

[0077] In this embodiment of the disclosure, the batch processing time of each job is determined based on the execution time of the preceding job included in the shortest weighted path.

[0078] According to the embodiments of this disclosure, the weight of each preceding job on the shortest weighted path is obtained. When the weight is set to the reciprocal of the execution time, the execution time of each preceding job can be obtained based on the weight of each preceding job. The execution times of all preceding jobs on the execution path are added together to obtain the batch job duration of the job. Since the smaller the weight, the longer the execution time, the shortest weighted path of the job corresponds to the longest batch job duration of the job, which is the longest waiting time for the job to be suspended.

[0079] According to embodiments of this disclosure, by using the reciprocal of the job's execution time as the job's weight, the execution path with the smallest weight value is selected from multiple execution paths of each job as the job's shortest weighted path. Since the smaller the weight, the longer the job's execution time, the job's shortest weighted path is actually the longest waiting time required for the job to be executed during batch job execution. By using the longest waiting time required for a job to be executed as the batch job duration for each job, each job can be reliably executed during batch job execution. This makes it easier for operations and maintenance personnel to estimate the waiting time of individual jobs in a batch job. Furthermore, the shortest weighted path of the job allows operations and maintenance personnel to easily trace the time distribution of batch job execution paths, thereby improving the accuracy of batch execution window division.

[0080] Figure 5 The illustration depicts a scenario diagram of a method for analyzing batch job duration according to another embodiment of the present disclosure.

[0081] like Figure 5As shown, the interface for analyzing batch job duration using graph theory algorithms includes a time inverter 501, a job information storage unit 502, a cross-application job processor 503, a directed graph generator 504, a directed graph memory 505, an analysis unit 506, a path persistence unit 507, and a time restorer 508. The analysis unit 506 includes a shortest path memory 5061, a judge 5062, and a predecessor memory 5063. The job information storage unit 502 refers to the job information data table shown in Table 1. The time inverter 501 is used to calculate the weights between jobs; the formula for the weights can be the reciprocal of the execution time. The cross-application job processor 503 is used in DBF batch processing when jobs appear... Figure 3 In the cross-application call scenario shown, the cross-application job is set as the root node. The directed graph generator 504 refers to traversing the job information data table shown in Table 1 to obtain the predecessor and neighbor jobs of the target application's optimized job. The directed graph memory 505 refers to a directed graph composed of arrays and linked lists. All jobs are stored in the array, and each job's neighbor jobs are stored in the linked list, facilitating traversal of the node. The analysis unit 506 is used to analyze the directed weighted graph. Specifically, the shortest path memory 5061 has an array data structure, with one job corresponding to one array element. The element value represents the sum of the weights of all nodes on the shortest path from the root node to that node. The root node is initialized to 0, and other job nodes are initialized to infinity. The judge 5062 also has an array data structure, with one job corresponding to one array element. The initial value of each element is N. Once the shortest path to that node is determined, the element's value is set to Y. That is, when the element's value is Y, it indicates that the shortest path to that node has been determined using graph theory algorithms; when the element's value is N, it indicates that the shortest path to that node has not yet been determined. The data structure in the predecessor memory 5063 is an array, with one array element corresponding to one job. The element's value is the parent node of that job node, i.e., the previous node. The initial value is -1.

[0082] The data structure in the path persistence unit 507 is a database table. Each job node has one record, representing the longest execution path from the root node to that node. The execution path results are retrieved from the predecessor memory 5063 according to the predecessor parent node. The execution time of each node is obtained from the time inverter 501, and the execution time of a node's execution path is the sum of the times of all nodes on that path. The time restorer 508 is the inverse operation of the time inverter 501.

[0083] Figure 6 A directed graph data structure diagram according to an embodiment of the present disclosure is illustrated schematically.

[0084] like Figure 6As shown, the directed graph consists of an array and linked lists. All jobs are stored in the array, and the neighboring nodes of each job node are stored in the linked lists for easy traversal. Figure 3 As shown, the root node's neighbors are jobs A and X. Job A's neighbors are jobs B, C, and D. Job B's neighbor is job G. Job C's neighbor is job E. Job D's neighbor is job F. Job E's neighbor is job J. Job F's neighbor is job Y. Job X's neighbor is job Y. Job G's neighbors are jobs H and E. The neighbors of job H, job J, and job Y are the end nodes. The end node indicates that the job processing in the target application has been completed.

[0085] Figure 7 A flowchart illustrating a method for analyzing batch job duration according to another embodiment of this disclosure is shown schematically.

[0086] like Figure 7 As shown, the batch job duration analysis method of this embodiment includes operations S701 to S711.

[0087] Use S701 to obtain the job list.

[0088] On the date the batch job is to be executed, the job list for the target application is retrieved based on the target application's identifier. For example, during the testing phase, if a batch job for the target application is expected to be executed on a certain date in the following month from the current time, to ensure the smooth execution of the batch job, the suspension duration of individual jobs within the batch job is tested at the current time. A specific date in the following month can be selected from the current time. Target applications that execute on different dates may contain different job lists. For example, for a payroll application, the job list for the 5th of each month and the job list for the 6th of each month may be different.

[0089] Store the task list in a format such as Figure 5 In the array of the directed graph memory 505 shown.

[0090] In operation S702, traverse the job information data table.

[0091] In operation S703, it is determined whether the previous job of the current job is another application. If the previous job of the current job is another application, operation S704 is executed; if the previous job of the current job is not another application, operation S705 is executed.

[0092] When operating S704, the previous job is set as the root node job.

[0093] If none of the applications to which a task belongs belong to the target application to which the task belongs, then the tasks to which the task belongs belong to the cross-application. The task is stored in the linked list corresponding to the root node of the array element in the directed graph memory 505.

[0094] When operating S705, the preceding job is set as a dependent job. That is, the preceding job is only the preceding job of this job, and this job has a dependency relationship with the preceding job.

[0095] When the current job is set as a dependent job, the job is stored in the linked list corresponding to the previous job in the array element of the directed graph memory 505.

[0096] In operation S706, the reciprocal of the job execution time is calculated and used as the weight of the job in the job list.

[0097] In operation S707, it is determined whether the job information data table has been completely traversed. If the job information data table has been completely traversed, operation S708 is executed; if the job information data table has not been completely traversed, operation S702 is executed.

[0098] When operating S708, a directed weighted graph is generated.

[0099] In operation S709, determine the shortest weighted path for each job in the directed weighted graph.

[0100] The shortest path in the directed weighted graph of operation S708 is determined by applying graph theory algorithms, thereby obtaining the shortest weighted path.

[0101] When operating the S710, determine the job on the shortest path.

[0102] When operating S711, the longest duration of batch jobs is obtained.

[0103] Based on the above-described method for analyzing batch job duration, this disclosure also provides a device for analyzing batch job duration. The following will be combined with... Figure 8 The device is described in detail.

[0104] Figure 8 A schematic block diagram of a device for analyzing batch job duration according to an embodiment of the present disclosure is shown.

[0105] like Figure 8 As shown, the batch job duration analysis device 800 of this embodiment includes a data acquisition module 810, a weight determination module 820, a path determination module 830, a target path determination module 840, and a duration determination module 850.

[0106] The data acquisition module 810 is used to respond to a batch job duration analysis request by obtaining the job list of the target application, the execution duration of each job in the job list, and the calling relationships between the jobs from the distributed batch processing database. In one embodiment, the data acquisition module 810 can be used to perform the operation S210 described above, which will not be repeated here.

[0107] The weight determination module 820 is used to determine the weight of each job based on the execution duration. In one embodiment, the weight determination module 820 can be used to perform the operation S220 described above, which will not be repeated here.

[0108] The path determination module 830 is used to determine the execution path of each job based on the calling relationship. In one embodiment, the path determination module 830 can be used to perform the operation S230 described above, which will not be repeated here.

[0109] The target path determination module 840 is used to determine the shortest weighted path for each job from the execution path based on the weights. In one embodiment, the target path determination module 840 can be used to perform the operation S240 described above, which will not be repeated here.

[0110] The duration determination module 850 is used to determine the batch processing duration of each job based on the shortest weighted path. In one embodiment, the duration determination module 850 can be used to perform the operation S230 described above, which will not be repeated here.

[0111] According to embodiments of this disclosure, the data acquisition module 810 includes a data table acquisition submodule, a job identifier determination submodule, a list determination submodule, a job determination submodule, an application identifier determination submodule, and a relationship determination submodule. The data table acquisition submodule is used to acquire a job information data table of batch jobs from a distributed batch processing database; the job information data table includes the job identifier of each individual job, its corresponding application identifier, the job identifier of the preceding job, the corresponding application identifier of the preceding job, and the execution duration. The job identifier determination submodule is used to determine the job identifier of the individual job whose corresponding application identifier is the target application based on the job identifier and the corresponding application identifier of each individual job. The list determination submodule is used to obtain the job list of the target application based on the job identifier of the individual job. The job determination submodule is used to determine the preceding job of each job in the job list based on the job identifier of the preceding job of each individual job. The application identifier determination submodule is used to determine the corresponding application identifier of the preceding job of each job in the job list based on the corresponding application identifier of the preceding job of each individual job. The relationship determination submodule is used to determine the calling relationship between jobs in the job list based on the preceding jobs of each job in the job list and the application identifier to which the preceding jobs of each job in the job list belong.

[0112] According to embodiments of this disclosure, the relationship determination submodule includes a root node determination unit, a first relationship determination unit, and a second relationship determination unit. The root node determination unit is configured to set the preceding job in the job list as the root node when the application identifier of the preceding job in the job list is not the target application. The first relationship determination unit is configured to obtain the calling relationship between the jobs in the job list and the preceding job when the application identifier of the preceding job in the job list is the target application. The second relationship determination unit is configured to determine the calling relationship between each job in the job list based on the calling relationship between the jobs in the job list and the preceding job.

[0113] According to embodiments of this disclosure, the target path determination module 840 includes a calculation submodule, a selection submodule, and a shortest path determination submodule. The calculation submodule is used to calculate, based on weights, the weighted value of each execution path of a job in the job list for that job. The selection submodule is used to select the execution path with the smallest weighted value from all execution paths of a job in the job list and use the execution path with the smallest weighted value as the shortest weighted path of the job in the job list. The shortest path determination submodule is used to determine the shortest weighted path for each job based on the shortest weighted path of the job in the job list.

[0114] According to embodiments of this disclosure, the calculation submodule includes a weight determination unit and a weighted value determination unit. The weight determination unit is used to determine, based on the weights, the weights of the preceding jobs included in each execution path of a job in the job list; the weighted value determination unit is used to determine the weighted value of each execution path of a job in the job list according to the weights of the preceding jobs included in each execution path of a job in the job list.

[0115] According to embodiments of this disclosure, any multiple modules among the data acquisition module 810, weight determination module 820, path determination module 830, target path determination module 840, and duration determination module 850 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the data acquisition module 810, weight determination module 820, path determination module 830, target path determination module 840, and duration determination module 850 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuitry, or implemented in any one of the three implementation methods of software, hardware, and firmware, or in a suitable combination of any of these. Alternatively, at least one of the data acquisition module 810, weight determination module 820, path determination module 830, target path determination module 840, and duration determination module 850 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0116] Figure 9 A block diagram of an electronic device suitable for implementing a batch job duration analysis method according to an embodiment of the present disclosure is shown schematically.

[0117] like Figure 9 As shown, an electronic device 900 according to an embodiment of the present disclosure includes a processor 901, which can perform various appropriate actions and processes according to a program stored in ROM (Read-Only Memory) 902 or a program loaded from storage portion 908 into RAM (Random Access Memory) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0118] RAM 903 stores various programs and data required for the operation of electronic device 900. Processor 901, ROM 902, and RAM 903 are interconnected via bus 904. Processor 901 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 902 and / or RAM 903. It should be noted that the programs may also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0119] According to embodiments of this disclosure, the electronic device 900 may further include an I / O (Input / Output) interface 905, which is also connected to a bus 904. The electronic device 900 may also include one or more of the following components connected to the I / O interface 905: an input section 906 including a keyboard, mouse, etc.; an output section 907 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 910 as needed so that computer programs read from it can be installed into the storage section 908 as needed.

[0120] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0121] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 902 and / or RAM 903 and / or one or more memories other than ROM 902 and RAM 903 described above.

[0122] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code enables the computer system to implement the batch job duration analysis method provided in embodiments of this disclosure.

[0123] When the computer program is executed by the processor 901, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0124] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 909, and / or installed from a removable medium 911. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0125] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0126] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on a user's computing device, partially on a user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0127] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0128] Those skilled in the art will understand that the features described in the various embodiments of this disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments of this disclosure can be combined and / or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0129] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A method for analyzing the duration of batch operations, characterized in that, The analytical method includes: In response to a batch job duration analysis request, the system retrieves the target application's job list, the execution duration of each job in the job list, and the calling relationships between the jobs from the distributed batch processing database. The weight of each task is determined based on the execution duration. Based on the aforementioned call relationships, determine the execution path for each job; Based on the weights, determine the shortest weighted path for each job from the execution paths; and The batch processing time for each job is determined based on the shortest weighted path.

2. The analytical method according to claim 1, characterized in that, The step of obtaining the job list of the target application, the execution time of each job in the job list, and the calling relationship between the jobs from the distributed batch processing database includes: Obtain the job information data table of batch jobs from the distributed batch processing database; the job information data table includes the job identifier of each individual job, the application identifier to which it belongs, the job identifier of the preceding job, the application identifier to which the preceding job belongs, and the execution time; Based on the job identifier and the application identifier of each individual job, the application identifier is determined as the job identifier of the individual job of the target application; Based on the job identifier of a single job in the target application, a job list for the target application is obtained; Based on the job identifier of the preceding job of each individual job, determine the preceding job of each job in the job list; Based on the application identifier of the preceding task of each individual task, determine the application identifier of the preceding task of each task in the task list; The calling relationship between the jobs in the job list is determined based on the preceding jobs of each job in the job list and the application identifier to which the preceding jobs of each job in the job list belong.

3. The analytical method according to claim 2, characterized in that, Based on the preceding jobs in the job list and the application identifiers of the preceding jobs in the job list, the calling relationships between the jobs in the job list are determined, including: When the application identifier of the preceding job in the job list is not the target application, the preceding job in the job list is set as the root node; When the application identifier of the preceding job in the job list is the target application, the calling relationship between the job in the job list and the preceding job is obtained; Based on the calling relationships between the jobs in the job list and the preceding jobs, determine the calling relationships between the jobs in the job list.

4. The analytical method according to claim 3, characterized in that, Each job must have at least one execution path; The process of determining an execution path for a job in the job list based on the aforementioned calling relationship includes: Taking one of the jobs in the job list as the current job, determine the previous job of the current job according to the calling relationship; If the previous job of the current job is not the root node, the previous job of the current job is taken as the current job, the current job is updated, and the previous job of the current job is determined according to the calling relationship until the previous job of the current job is the root node. When the preceding job of the current job is the root node, the preceding jobs between the jobs in the job list and the preceding job of the current job constitute an execution path of a job in the job list.

5. The analytical method according to claim 4, characterized in that, The step of determining the shortest weighted path for each job from the execution path based on the weights includes: For a job in the job list, based on the weight, calculate the weighted value of each execution path of all execution paths of the job in the job list; Select the execution path with the smallest weighted value from all execution paths of a job in the job list, and use the execution path with the smallest weighted value as the shortest weighted value path of a job in the job list; Determine the shortest weighted path for each task based on the shortest weighted path of a task in the task list.

6. The analytical method according to claim 5, characterized in that, Based on the aforementioned weights, calculate the weighted value of all execution paths for a job in the job list, including: Based on the weights, determine the weights of the preceding tasks included in each execution path of a task in the task list; The weighted value of each execution path of a task in the task list is determined based on the weight of the preceding task included in each execution path of the task in the task list.

7. The analytical method according to claim 4, characterized in that, The batch processing time of each job is determined based on the execution time of the preceding job included in the shortest weighted path.

8. A device for analyzing the duration of batch operations, characterized in that, The analytical device includes: The data acquisition module is used to respond to the analysis request for batch job duration by obtaining the job list of the target application, the execution duration of each job in the job list, and the calling relationship between each job from the distributed batch processing database. The weight determination module is used to determine the weight of each job based on the execution time; wherein the weight of each job is inversely proportional to the execution time. The path determination module is used to determine the execution path of each job based on the calling relationship; A target path determination module is used to determine the shortest weighted path for each job from the execution paths based on the weights; and The duration determination module is used to determine the batch processing duration of each job based on the shortest weighted path.

9. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 7.

Images