Batch job delay estimation method, device, equipment, medium and program product

Abstract

The embodiment of the invention provides a batch job delay estimation method and device, equipment, a medium and a program product, and relates to the field of financial science and technology or other related fields. In response to a detected preset change event, obtaining a link model of a distributed system for executing batch jobs, the link model comprising a plurality of nodes for executing the batch jobs in topological connection, the topological connection between the nodes is used for representing file dependence, job dependence and online service calling dependence when the nodes process batch jobs; determining a target node and a target online service corresponding to the preset change event, and respective service delay time of the target node and the target online service; generating a full-link response delay estimation result of the batch job subjected to the preset change event based on the link model and respective service delay time of the target node and the target online service; the method improves the evaluation efficiency and accuracy.

Description

Technical Field

[0001] This application relates to the field of financial technology or other related fields, and in particular to a batch operation delay estimation method, apparatus, equipment, medium and program product. Background Technology

[0002] Batch job processing is typically managed and executed centrally through a batch job scheduling platform. Based on a preset time plan or dependencies between jobs, the platform distributes batch job tasks defined by each application to the corresponding batch executors. The batch executors are responsible for running the specific job logic, processing batch files or database data generated by upstream jobs, and generating new files for downstream applications to continue processing, thereby achieving cross-application collaborative batch processing.

[0003] In existing technologies, methods for estimating batch job delays primarily rely on the subjective experience and judgment of development and operations personnel. When certain batch jobs are suspended or delayed, the responsible parties need to manually analyze the dependencies between jobs and assess the potential business impact of the suspension and its duration.

[0004] However, as systems grow in scale and inter-application dependencies become increasingly complex, manual analysis struggles to comprehensively and accurately identify all affected paths and business processes. Secondly, the evaluation results are highly dependent on individual experience, lacking standardization and repeatability, making it difficult to guarantee their reliability. Therefore, experience-based evaluation methods are inefficient and prone to bias. Summary of the Invention

[0005] This application provides a batch job delay estimation method, apparatus, equipment, medium, and program product to solve the problems of low efficiency and easy deviation of experience-based evaluation methods.

[0006] In a first aspect, this application provides a method for estimating batch job delays, including:

[0007] In response to a detected preset change event, a link model of the distributed system for executing batch jobs is obtained. The link model includes multiple nodes connected in a topology for executing the batch jobs. The topology connections between nodes are used to characterize file dependencies, job dependencies, and online service call dependencies when nodes process batch jobs.

[0008] Determine the target node and target online service corresponding to the preset change event, as well as the service delay time of each of the target node and the target online service;

[0009] Based on the link model and the service latency of the target node and the target online service, the full-link response latency estimate of the batch job affected by the preset change event is generated.

[0010] Secondly, this application provides a batch job delay estimation device, comprising:

[0011] The link model acquisition module is used to acquire the link model of the distributed system for executing batch jobs in response to a detected preset change event. The link model includes multiple nodes connected in the topology for executing the batch jobs. The topology connection between the nodes is used to characterize the file dependencies, job dependencies and online service call dependencies when the nodes process the batch jobs.

[0012] The target determination module is used to determine the target node and target online service corresponding to the preset change event, as well as the service delay time of the target node and the target online service respectively;

[0013] The impact assessment module is used to generate an estimate of the end-to-end response delay of the batch job affected by the preset change event, based on the link model and the service delay times of the target node and the target online service.

[0014] Thirdly, this application provides an electronic device, including: a processor and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method provided in the first aspect above.

[0015] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method provided in the first aspect above.

[0016] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method provided in the first aspect above.

[0017] The batch job latency estimation method provided in this application accurately determines the full-link topology of batch job execution in a distributed system by constructing a link model that includes file dependencies and online service call dependencies. When a preset change event is detected, the method can quickly locate the affected target nodes and online services, quantify their service latency, and then automatically generate response latency estimation results covering the entire path based on the link model. This evaluation method, which deeply integrates the impact of change events with system dependencies, not only automates and dynamizes latency prediction, but also improves the efficiency and accuracy of batch job latency estimation by eliminating the evaluation bias caused by the fragmentation of dependencies in traditional methods. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] Figure 1 A flowchart illustrating a batch job delay estimation method provided in an embodiment of this application;

[0020] Figure 2 This is a diagram showing the element relationships in the node model provided in the embodiments of this application;

[0021] Figure 3 A schematic diagram of a batch job delay estimation device provided in an embodiment of this application;

[0022] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0023] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0024] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0025] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, have taken necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation access points for users to choose to authorize or refuse.

[0026] Furthermore, the technical solution involved in this application, which involves big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.) and the use of artificial intelligence technology for automated decision-making, and makes decisions that have a significant impact on personal rights based on the results of automated decision-making, provides users with corresponding operation entry points for users to choose to agree to or reject the results of automated decision-making; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0027] It should be noted that the batch job delay estimation system, batch job delay estimation method and request initiation provided in this application can be used in the fintech field, or in any field other than fintech. The application field of the batch job delay estimation system, batch job delay estimation method and request initiation in this application is not limited.

[0028] Batch job processing typically relies on a batch job scheduling platform for unified management and scheduling. This platform distributes batch job tasks defined by various applications to corresponding batch executors for execution based on preset time plans or dependencies between jobs. The batch executors are responsible for running the specific job logic, including processing and transforming batch files or database data processed by upstream jobs, and generating new data files for downstream applications to continue processing, thereby realizing a batch collaborative processing flow between applications.

[0029] In existing technologies, the assessment methods for the impact of batch job delays mainly rely on the subjective judgment of development and operations personnel. When some jobs are suspended or delayed, relevant personnel need to manually identify the dependencies between jobs, assess the potential business scope affected by the delay, and the duration of the impact.

[0030] However, as the system scales up and the dependencies between applications become increasingly complex, manual analysis is no longer sufficient to comprehensively and accurately identify all affected operational paths and business processes. Furthermore, the evaluation results are highly dependent on individual experience, lacking unified standards and repeatable procedures, leading to low reliability of the evaluation results.

[0031] The batch job latency estimation method, apparatus, equipment, media, and program products provided in this application aim to solve the aforementioned technical problems of the prior art. After detecting a pre-defined change event—the deployment of a business system from a single city to a multi-location, multi-center deployment—the first step is to obtain the link model of batch job execution in the distributed system. This model, using nodes and their topological connections, accurately presents file dependencies, job dependencies, and online service call dependencies during job processing, thereby constructing a complete job execution dependency path. Next, the target nodes affected by this change and their associated online services are identified, and their respective service latency times are determined. Since changes in the link layout may cause batch online service calls to change from local calls to cross-city calls, increasing online service response time, and increasing transmission time due to increased distance for upstream and downstream batch file dependencies, the overall link timeliness is affected. Furthermore, the adjustment may cause the various links in the batch link to be dispersed across different cities, creating cross-city dependency scenarios. Based on this, by combining the link model and service latency information, the overall link response latency caused by the change event is analyzed and estimated, ultimately achieving automated and accurate assessment of the impact of batch job latency, effectively improving assessment efficiency and accuracy.

[0032] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0033] Figure 1 A flowchart illustrating a batch job delay estimation method provided in this application embodiment. Figure 1 The method provided in this application can be executed by a batch job scheduling platform, or by a device that is communicatively connected to the batch job scheduling platform, such as a computer or server. Figure 1 As shown, the batch job delay estimation method provided in this embodiment includes the following steps:

[0034] S101: In response to a detected preset change event, obtain the link model of the distributed system used to execute batch jobs. The link model includes multiple nodes connected in the topology for executing batch jobs. The topology connections between nodes are used to characterize the file dependencies, job dependencies, and online service call dependencies when nodes process batch jobs.

[0035] Among them, preset change events are specific events that are pre-set in the business system and may cause changes in the system architecture or link layout (such as multi-site and multi-center deployment), thereby affecting the timeliness of batch job processing and online service response. Their business impact and feasibility need to be assessed in advance.

[0036] The link model is a model in a distributed system consisting of multiple nodes that execute batch jobs connected in a specific topology. It is used to clearly show the file dependencies and online service call dependencies when nodes process batch jobs.

[0037] Specifically, adjustments to the batch job scheduling platform's configuration parameters, such as modifying the CPU utilization threshold for a critical node processing batch jobs, or changing the job scheduling time interval; changes in hardware resources, such as adding a server node for executing batch jobs, or removing a node due to failure; or changes in the attributes of the batch jobs themselves, such as a sudden and significant increase in the job's data volume, or a reset of the job's priority. Once the batch job scheduling platform detects these pre-defined change events, it will immediately trigger the subsequent operation of obtaining the link model.

[0038] By acquiring the link model, the scope and extent of the impact of change events on the batch job scheduling platform can be identified in a timely manner, allowing for proactive countermeasures. For example, if a new node is added, the link model can quickly determine the connection relationships and dependencies between this node and other nodes, thereby enabling the rational allocation of job tasks and ensuring the efficient operation of the system.

[0039] The link model primarily consists of multiple nodes connected in the topology to execute batch jobs. These nodes are the basic processing units in a distributed system, each undertaking a specific batch job processing task. The topological connections between nodes contain rich information, representing file dependencies, job dependencies, and online service call dependencies when nodes process batch jobs. File dependencies indicate that some nodes need to rely on specific files generated or provided by other nodes when processing jobs; only by obtaining these files can they continue their processing flow. Online service call dependencies indicate that some nodes need to call online services provided by other nodes during job execution, completing complex job processing through this service interaction. For example, in a batch data processing job, one node might be responsible for data collection and preliminary cleaning, generating intermediate files, while another node relies on these intermediate files for further data analysis and mining. Simultaneously, during the analysis process, this node might also need to call statistical computing online services provided by other nodes to obtain more accurate analysis results. Through the topological connections in the link model, these complex dependencies are clearly presented, providing a strong guarantee for the efficient and stable execution of batch jobs in a distributed system.

[0040] S102: Determine the target node and target online service corresponding to the preset change event, as well as the service delay time of the target node and target online service respectively.

[0041] The service latency time can be the unavailability time of the target node or the additional processing time of the target online service.

[0042] When the preset change event is a link layout adjustment, determine the corresponding target nodes, target online services, and their respective service latency, and first analyze the specific content of the change event. If the link layout adjustment involves system software updates, identify which nodes will install the new software; these nodes are potential target nodes. Simultaneously, identify external online services that the software update depends on, such as database queries and authentication, as target online services. If the adjustment is hardware-related, such as adding storage devices or changing the deployment location of existing devices, then the nodes that directly interact with the device and are responsible for data read and write operations are the target nodes, and the online services providing storage management interfaces are the target online services. After determining the target nodes and online services, analyze and determine their respective service latency during the link layout adjustment process, considering the change operation procedures and technical requirements.

[0043] For node unavailability caused by software upgrades, the download, installation, configuration, and restart times required for the upgrade should be estimated. Simultaneously, during the actual upgrade process, the node status should be monitored, and if unexpected situations such as software compatibility issues or installation errors occur, the estimated service latency should be adjusted promptly. For node unavailability caused by hardware changes, such as hardware replacement or repair, the hardware manufacturer's repair time commitment should be referenced, combined with the actual on-site repair situation, such as the skill level of the repair personnel and the timeliness of spare parts supply, to determine the node's service latency. When the preset change event is a layout change, since layout changes mainly manifest as stable changes in upstream and downstream distances, the target nodes on the links affected by the layout adjustment, involving service call dependencies and file transfer dependencies, and the target online services that these target nodes depend on, can be identified first. For target nodes and target online services, changes in service latency can be automatically assessed using other online models based on factors such as message transmission time and processing performance efficiency to determine their respective service latency. Changes in file transfer latency are automatically calculated by establishing a model based on factors such as file size, transmission distance, bandwidth, and network line conditions.

[0044] Determining the service latency of the target online service requires considering both service dependencies and the scope of the change's impact. Firstly, analyze other services and resources the target online service depends on, such as databases and network connections. If these dependencies are also affected by the change event, the service latency of the target online service will depend on the recovery time of these dependencies. For example, if the database the online service depends on is undergoing maintenance and upgrades, the service latency must at least cover the time of the database maintenance and upgrade, as well as the time for the online service to re-establish its connection with the database and synchronize data. Secondly, consider the direct impact of the change event on the online service itself, such as modifications to service configurations and code updates. By simulating the change process in a small-scale test environment, recording the times the online service stops, changes, and restarts, and then making appropriate adjustments based on the actual situation in the production environment, the service latency of the target online service can be determined.

[0045] S103: Based on the link model and the service latency of the target node and the target online service, generate the full-link response latency estimation results of batch jobs affected by preset change events.

[0046] Based on the link model, the flow path of batch jobs throughout the system can be clearly understood. For example, in a data processing batch job, starting from the data acquisition node, the data will sequentially pass through the cleaning node, the analysis node, and finally reach the storage node. Each node is connected through specific file transfers or online service calls. The link model clarifies the input and output of the job at each stage and the interaction methods between nodes.

[0047] Based on the constructed link model, the service latency of target nodes and target online services is analyzed to accurately identify the nodes and services that have a critical impact on batch job response latency. For example, if an online service changes from a local call to a cross-city call, increasing the online service response time, and a target node in the link that performs data cleaning tasks has a 2-hour service latency, and this node is a critical hub in the data flow of the link model, subsequent analysis nodes must wait for it to complete its data cleaning work and obtain the processed data before proceeding with subsequent processing. Simultaneously, if a target online service (such as an online service used for data verification) has a 1-hour service latency, and analysis nodes need to frequently call this service during data processing, this will also affect the batch job response time. By analyzing the position and role of the service latency of each target node and target online service in the link model, the key links causing batch job stagnation or delays can be identified.

[0048] After identifying key impacting nodes and services, their impact on the end-to-end response latency of batch jobs can be comprehensively assessed to generate an estimate. For the service latency of target nodes, the data backlog and subsequent processing wait time caused by node unavailability are calculated based on their upstream and downstream relationships in the link model. For example, if a data acquisition node is unavailable for one hour with a data acquisition rate of 10GB per hour, while the cleaning node has a processing capacity of 8GB per hour, then after the acquisition node recovers, the cleaning node will need to spend additional time processing the backlogged data; this time will be included in the end-to-end response latency. The service latency of target online services is also considered, impacting node processing efficiency. For instance, if the data verification online service is unavailable, the analysis node may need to use a backup but less efficient verification method, leading to increased processing time. By summing all the latency caused by the service latency of target nodes and target online services and combining it with the normal processing time of batch jobs, an estimate of the end-to-end response latency affected by the preset change event can be generated.

[0049] The batch job latency estimation method provided in this application obtains a link model of the distributed system used to execute batch jobs in response to a detected preset change event. The link model includes multiple nodes connected in the topology for executing batch jobs. The topology connections between nodes represent file dependencies, job dependencies, and online service call dependencies when nodes process batch jobs. The method determines the target node and target online service corresponding to the preset change event, as well as the service latency of each target node and target online service. Based on the link model and the service latency of each target node and target online service, the method generates an end-to-end response latency estimation result for batch jobs affected by the preset change event. By constructing a link model that includes file dependencies and call dependencies, and combining the service latency of the target node and online service, this method achieves automated and accurate evaluation of the end-to-end response latency of batch jobs in the distributed system affected by change events, thus improving evaluation efficiency and accuracy.

[0050] In some embodiments, the node information includes the application to which the node belongs and the application to which the online call service belongs; determining the target node and target online service corresponding to the preset change event, as well as the service delay time of the target node and target online service respectively, specifically implemented as follows:

[0051] Based on the change order for the preset change event, determine the target application involved in the preset change event;

[0052] Based on the node information of each node in the link model, the nodes and online calls belonging to the target application are determined respectively, which are the target nodes and target online calls;

[0053] Extract the change time from the change order of the preset event;

[0054] Based on the change time, determine the service delay time for the target node and the target online service.

[0055] Among them, the change order is a formal document that records in detail key information of the preset change event (such as the target application involved, the change time, etc.) to guide the implementation and tracking of change operations.

[0056] Specifically, when a pre-defined change event occurs, a corresponding change order is generated, which clearly indicates the target application involved in the event. The target application is the core object of the entire change operation. It can be a specific batch job scheduling platform, such as the order processing application in an online shopping system; or it can be a basic service application, such as a database management application. By analyzing the change order, the target application of the pre-defined change event can be accurately determined.

[0057] After identifying the target application, filtering can be performed based on the node information of each node in the link model. Nodes whose application belongs to the target application are directly associated with the target application and are identified as target nodes. Online calls whose service belongs to the target application indicate that the services these online calls depend on are provided by the target application. These online calls will be affected when a change event occurs, so they are identified as target online calls.

[0058] In addition to the target application information, the change order also records the change time. The change time is a key factor in determining the service latency of the target node and the target online service. The change time typically includes the start time and the estimated end time. Based on this change time, combined with the characteristics of the target node and the target online service and their roles in the system, their respective service latency is determined. For example, if the target node needs a software upgrade, and the change order shows an upgrade time of 2 hours, then during these 2 hours, the target node will be unable to process batch jobs normally, and its service latency is 2 hours. For the target online service, if the change involves a service restart or configuration adjustment, the service interruption time can be estimated based on the change time, thereby determining its service latency.

[0059] In some embodiments, based on the link model and the service latency times of the target node and the target online service, an end-to-end response latency estimate for batch jobs affected by preset change events is generated, including:

[0060] Based on the service latency of the target node and the target online service, determine the response latency increment of the target online service;

[0061] Based on the file dependencies and call dependencies of nodes in the link model, as well as the response latency increment of the target online service, determine the response latency increment of downstream nodes affected by the target node and the target online service in the link model.

[0062] Based on the calculation of the response latency increment of each node, the end-to-end response latency estimation results of batch jobs affected by preset change events are generated.

[0063] The service latency of the target node and the target online service is a key indicator for evaluating the impact of pre-defined change events on the system. When the target online service becomes unavailable, its response latency increases significantly. The response latency increment is determined by comparing the average response time of the target online service under normal availability with the estimated recovery time and related processing time during the unavailability period. For example, if a target online service normally takes an average of 100 milliseconds to process a request, and if it becomes unavailable for 30 minutes due to a pre-defined change event, during this period, related requests will back up. After recovery, processing the backlogged requests and new requests may take up to 500 milliseconds due to factors such as system resource contention. In this case, the response latency increment of the target online service would be 400 milliseconds.

[0064] In the link model, file dependencies and call dependencies exist between nodes. When downstream nodes process batch jobs, they typically rely on file data provided by upstream nodes (including target nodes) or call upstream target online services. If the target online service experiences an increase in response latency, it will directly impact the processing flow of downstream nodes that depend on it. Downstream nodes will be blocked while waiting for the target online service's response, thus extending their own response time. Based on the dependencies in the link model, the degree of dependence of each downstream node on the target online service is determined. If 80% of a downstream node's processing tasks require calling the target online service, then the increase in response latency of the target online service will be largely propagated to that downstream node. Furthermore, considering file dependencies, if the target node is unavailable, causing downstream nodes to be unable to obtain the required files in a timely manner, it will also halt downstream node processing, further exacerbating response latency. It may even lead to situations where increased distance between upstream and downstream batch file dependencies results in longer transmission times, affecting the overall link's timeliness and preventing downstream nodes from completing batch jobs on time.

[0065] After determining the response latency increment of each affected node, all nodes in the entire link model are analyzed and calculated. Starting from the initial node of the batch job, the response latency increment of each node is accumulated sequentially along the direction of data flow and processing. For links with parallel processing branches, the latency increment of each branch is calculated separately, and then combined according to the processing rules of the merging nodes. For example, in some systems, the maximum latency of the parallel branches is taken as the merged latency. Through this node-by-node, branch-by-branch calculation method, the response latency increments of all nodes are summarized, and finally, the full-link response latency estimate of the batch job affected by the preset change event is generated.

[0066] Optionally, based on the service latency of the target node and the target online service, the response latency increment of the target online service is determined. Specific implementation methods include:

[0067] Input the service latency of the target node and the target service into the effectiveness analysis model to obtain the response latency increment of the target online service.

[0068] Among them, the effectiveness analysis model is a tool for determining the incremental response latency of the target online service.

[0069] Specifically, before inputting the service latency of the target node and the target online service into the model, a comprehensive initialization and parameter setting of the model is required. This includes collecting historical data of the system under normal operating conditions, such as the average response time and response time fluctuation range of the target online service under different loads and time periods; at the same time, the interaction logic between the target node and the target online service must be clearly defined, such as the frequency of the target node calling the target online service and the amount of data called each time.

[0070] Once the service latency times of the target node and the target online service are obtained, they are organized and input according to the format required by the model. The performance analysis model will perform multi-dimensional analysis of these service latency times. On the one hand, longer service latency times often mean more request backlogs. After service recovery, processing backlogged requests will lead to a significant increase in response time. For example, if the target online service is unavailable for one hour, there may be thousands of requests backlogged during this period. After recovery, the system needs to process these backlogged requests and newly arrived requests simultaneously, which will inevitably cause a significant increase in response latency. On the other hand, if unavailability occurs during peak system business periods, the backlog effect caused by unavailability will be more pronounced due to the already large request volume, and the increase in response latency will be even greater. The model will also consider the impact of target node unavailability on the target online service. If the target node is an important data source for the target online service, its unavailability will prevent the target online service from obtaining complete data for a period of time, thereby affecting its processing efficiency and response time.

[0071] Optionally, the response latency increment includes file transfer latency and job processing latency; based on the file dependencies and call dependencies of nodes in the link model, and the response latency increment of the target online service, the response latency increment of downstream nodes affected by the target node and the target online service in the link model is determined, including:

[0072] Based on the link model, target nodes, and target online services, determine the links affected by preset change events;

[0073] The nodes in the affected link are traversed according to the breadth-first strategy. For each traversed node, the time change of batch files reaching the node is calculated based on the size of the upstream files indicated by the file dependency of the node and the deployment area of ​​the upstream application, so as to obtain the file transmission delay of the node. The upstream application is the application to which the upstream node of the node belongs.

[0074] If the node has online service call dependencies, the time change of the node's online service is calculated based on the service, application, and cluster to which the call dependency belongs, so as to obtain the job processing delay of the node.

[0075] Among them, the impact link is a complete path based on the link model, which starts from the target node and target online service involved in the preset change event and connects all relevant nodes and online services directly or indirectly affected by the event.

[0076] Breadth-first search is a graph traversal algorithm that starts from a starting point, visits all the nearest nodes, and then visits the nodes of subsequent levels outwards.

[0077] Specifically, starting from the target node and the target online service, the impact chain traces back and forward along the connections between nodes and between nodes and online services in the chain model. For example, if the target node is a data computation node, the impact chain will trace upwards to the data acquisition node that provides it with data, and downwards to the data storage node that receives its computation results; for the target online service, it will identify the node that calls the service and the nodes where other services that the service depends reside. All nodes and online services directly or indirectly affected by the preset change event are linked together to form a complete impact chain.

[0078] After identifying the affected links, nodes in the affected links are traversed using a breadth-first search strategy. The breadth-first strategy first visits the node closest to the starting point and then expands outwards. For each traversed node, based on its file dependency information, the size of the upstream file and the deployment zone of the upstream application are obtained. The size of the upstream file is one of the key factors affecting transmission time; the larger the file, the longer the transmission time. The deployment zone of the upstream application determines the network distance and bandwidth for file transmission. For example, if the upstream application is deployed in a zone far from the current node and has limited network bandwidth, the file transmission time will increase. Using a pre-defined file transmission time calculation formula, such as transmission time = file size / network bandwidth + network latency (network latency can be estimated based on the network topology between zones and historical data), the time variation for batch files to reach the node is calculated, thus obtaining the file transmission latency of that node.

[0079] The system explicitly identifies the service, application, and cluster to which the service depends. The application and cluster reflect the service's runtime environment and resource allocation. Different applications and clusters have varying performance characteristics, significantly impacting service processing time. For example, an online service running on a high-load cluster will take longer to process requests than a similar service running on a low-load cluster. Then, by considering the online service's historical processing time data, current system load, and the potential impact of pre-defined changes (such as request backlog due to the unavailability of the target online service, causing it to spend extra time processing backlogged requests on the current node), and using appropriate algorithms—such as machine learning-based predictive models or models calculated using empirical formulas—the system calculates the time variation for the node to call the online service, thus determining the node's job processing latency.

[0080] In some embodiments, the processing time for each node to generate the corresponding file is determined based on the calculated response latency increment of each node, the processing time for each node to generate the file, and the time for the upstream node to transmit the file to each node.

[0081] If a node's processing time for generating the corresponding file exceeds the corresponding threshold, then backtracking is performed along the reverse path of the link model, starting from that node. Based on the node information of the backtracked node, the key node or link that caused the file generation timeout is identified and output.

[0082] Determine the processing time for each node to generate the corresponding file. For example, if a node originally processed tasks quickly, but its response latency increases by 30 milliseconds due to a change event, it will consume an additional 30 milliseconds when processing subsequent tasks. Simultaneously, each node's file generation itself has a base processing time, which depends on the node's hardware performance, such as CPU processing power and memory size, as well as the complexity of the software algorithm. For instance, a node with lower specifications processing a complex algorithm to generate files may have a longer base processing time than a high-performance node. Furthermore, the time it takes for files to be transferred from upstream nodes to each node is also crucial. File transfer is affected by file size; large file transfers will inevitably take longer. When transferring across regions, factors such as network bandwidth and network latency will also increase the transfer time. By adding the response latency increment of each node, its own base processing time for generating files, and the time it takes for files to be transferred from upstream nodes to that node, the actual processing time for each node to generate the corresponding file can be accurately determined. For example, if the basic file generation time of a certain node is 80 milliseconds, the response latency increment is 20 milliseconds, and the upstream file transfer time is 40 milliseconds, then the processing time for that node to generate the corresponding file is 140 milliseconds.

[0083] After obtaining the processing time for each node to generate the corresponding file, it needs to be compared with a pre-set threshold. This threshold is determined based on a combination of factors, including system performance requirements, business needs, and historical data. If the processing time for a node to generate a file exceeds the corresponding threshold, it indicates that the node has timed out during file generation. For example, if the system stipulates that the processing time for a critical node to generate a file should not exceed 150 milliseconds, but the actual calculated processing time for that node is 180 milliseconds, this indicates that the node has a file generation timeout issue.

[0084] When a node is found to have timed out during file generation, the process begins by tracing back along the topology recorded in the link model. The topology in the link model encompasses key information such as file dependencies and online call dependencies between nodes, providing a clear path for backtracking and troubleshooting. During the backtracking process, each node involved in the file generation is examined layer by layer.

[0085] For upstream job nodes that provide input files, check if there are any delays that cause the input files to arrive late. For example, if an upstream job node fails to generate and transmit the input file on time due to excessive load or a malfunction, the current node will spend extra time waiting for the input file, resulting in a file generation timeout.

[0086] The file transfer process is checked, and the file transfer latency formula is used to analyze whether factors such as file size or cross-regional transmission are causing timeouts. If the file is too large, or if the transmission crosses multiple regions, or if the network conditions are unstable, the file transfer time may exceed the normal range, affecting the file generation at the current node.

[0087] Analyze the current node's job processing flow and calculate job processing latency to see if increased processing time is due to factors such as the cumulative latency of online calls or the node's own response delay. For example, if the current node needs to call multiple online services when processing files, delays in these online services or a slowdown in the node's own response will increase the file generation processing time.

[0088] At the same time, it is also important to pay attention to basic information such as the park where the node is deployed and its resource configuration. The network environment and power supply conditions may vary in different parks, and the adequacy of node resource configurations such as CPU and memory will also affect its processing capacity.

[0089] By conducting a comprehensive analysis of the backtracking nodes, we can identify the key nodes or processes that cause file generation timeouts and output the results in a clear and accurate manner.

[0090] In some embodiments, such as Figure 2 As shown, the nodes in the link model correspond to node models. The node model is used to describe the node information, including the element information of the three entity elements: job processing, online call user, and file generation, as well as the relationships between the elements.

[0091] The relationships between elements include file dependencies describing upstream files that job processing depends on, upstream jobs that job processing depends on, invocation dependencies describing online calls, and relationships between files generated by job processing.

[0092] The node model, with each independent processing unit as its object, is comprised of three entity elements and four relationships: In terms of entity elements, job processing is the core, while also encompassing online calls and file generation as related elements; in terms of relationships, it includes three types of dependencies (upstream job dependency, upstream file dependency, and online call dependency, all of which may exist partially or entirely, and each forming a one-to-many relationship with job processing) and one type of generation relationship (job processing may generate one or more new batch files, but not all job processing will trigger file generation). This model, by structurally expressing the internal logic of nodes, provides a basic framework for global analysis and performance optimization of batch processing chains.

[0093] Job processing is the core manifestation of the node function in the link model. It is associated with a specific batch executor node name to clarify the execution subject, the application to which it belongs defines its business scope, and the deployment park and city reflect its physical location and resource environment. It has three key dependencies that affect the execution order and data acquisition of the job. The execution plan determines the triggering method of the job, which may be executed after the upstream job is completed or according to the timed rules. The average execution time directly reflects the processing efficiency of the job, and the serviceable latency time clarifies the time range in which the job can provide services normally. These elements together constitute a description of job processing.

[0094] Online calls are a crucial step in the interaction between nodes and external services. They identify the specific service being called through a clear service name, while the application to which the call belongs indicates its place within the overall batch job scheduling platform. Group information helps in the classification and management of related services. Deployment campus and city indicate the physical location of the service, which is related to network latency and service availability. Average response time reflects the efficiency of the service in processing requests and is an important indicator of service performance. Service latency defines the time period during which the service can respond to calls normally. Average call count reflects the frequency of service usage in business operations. These factors collectively manage online calls.

[0095] File generation specifies the exact filename of the generated file for identification and location; the average file size reflects the capacity characteristics of the generated file; the storage path indicates the location of the file in the system, while the storage park and city describe the physical environment of the file storage; the average file processing time reflects the efficiency of the entire process from the start of generation to the completion of storage; and the average data volume (number of executions) shows the scale of information contained in the file from a data dimension. These elements together constitute file generation.

[0096] The relationships between elements construct the logic of collaborative operation among the various parts within a node. Among them, file dependencies clarify the job processing's dependence on upstream files, defining the source of the data files required by the job processing; the dependencies on upstream jobs streamline the execution order of job processing, ensuring the continuity of the business process; call dependencies detail the association between online calls, ensuring that online services interact according to the correct logic; and the file relationships generated by job processing clearly define the correspondence between job processing and output files, facilitating the tracking and management of the generated results.

[0097] In some embodiments, the preset change events include at least one of production change, off-site disaster recovery drill, and multi-site multi-center migration deployment.

[0098] Production change refers to the critical operation process in the development of information systems or batch job scheduling platforms, in which new system versions, functional modules, optimization strategies, or newly added hardware equipment and adjusted configuration parameters that have undergone comprehensive testing, rigorous verification and meet the online standards are migrated from the development and testing environment to the actual production environment, so that they can be officially put into operation to support business operations, improve system performance or meet new business needs.

[0099] Off-site disaster recovery drills simulate disaster scenarios in geographically separated backup data centers. By performing operations such as data recovery, system switching, and business takeover, the reliability, integrity, and takeover capabilities of the off-site disaster recovery system are verified, ensuring that business operations can be quickly restored and losses reduced when a real disaster occurs.

[0100] Multi-site, multi-center migration deployment is a distributed system architecture strategy that deploys applications or services in multiple geographically dispersed data centers (such as a three-site, five-center architecture) to achieve cross-regional load balancing, fault isolation, and data synchronization. It ensures that traffic can be automatically switched to other normal nodes when any node fails. At the same time, it uses a unitized architecture (such as binding data centers to user groups) to resolve data consistency conflicts, ultimately achieving the goal of high system availability, fault tolerance, and low-latency access.

[0101] Figure 3 This is a schematic diagram of a batch job delay estimation device provided in an embodiment of this application. Figure 3 As shown, the batch job delay estimation device provided in this embodiment includes: a link model acquisition module 201, a target determination module 202, and an impact assessment module 203.

[0102] Link model acquisition module 201 is used to acquire the link model of the distributed system for executing batch jobs in response to a detected preset change event. The link model includes multiple nodes for executing batch jobs with topological connections. The topological connections between nodes are used to characterize the file dependencies, job dependencies, and online service call dependencies when nodes process batch jobs.

[0103] The target determination module 202 is used to determine the target node and target online service corresponding to the preset change event, as well as the service delay time of the target node and target online service respectively;

[0104] The impact assessment module 203 is used to generate an end-to-end response delay estimate for batch jobs affected by preset change events, based on the link model and the service delay times of the target node and the target online service.

[0105] Optionally, the target determination module 202 is specifically used for:

[0106] Based on the change order for the preset change event, determine the target application involved in the preset change event;

[0107] Based on the node information of each node in the link model, the nodes and online calls belonging to the target application are determined respectively, which are the target nodes and target online calls;

[0108] Extract the change time from the change order of the preset event;

[0109] Based on the change time, determine the service delay time for the target node and the target online service.

[0110] Optionally, the impact assessment module 203 is also used for:

[0111] Based on the service latency of the target node and the target online service, determine the response latency increment of the target online service;

[0112] Based on the file dependencies and call dependencies of nodes in the link model, as well as the response latency increment of the target online service, determine the response latency increment of downstream nodes affected by the target node and the target online service in the link model.

[0113] Based on the calculation of the response latency increment of each node, the end-to-end response latency estimation results of batch jobs affected by preset change events are generated.

[0114] Optionally, the batch job delay estimation device includes: a delay increment determination module 204, used for:

[0115] Input the service latency of the target node and the target service into the effectiveness analysis model to obtain the response latency increment of the target online service.

[0116] Optionally, the batch job delay estimation device includes: a node delay module 205, used for:

[0117] Based on the link model, target nodes, and target online services, determine the links affected by preset change events;

[0118] The nodes in the affected link are traversed according to the breadth-first strategy. For each traversed node, the time change of batch files reaching the node is calculated based on the size of the upstream files indicated by the file dependency of the node and the deployment area of ​​the upstream application, so as to obtain the file transmission delay of the node. The upstream application is the application to which the upstream node of the node belongs.

[0119] If the node has online service call dependencies, the time change of the node's online service is calculated based on the service, application, and cluster to which the call dependency belongs, so as to obtain the job processing delay of the node.

[0120] Optionally, the batch job delay estimation device includes: a node determination module 206, used for:

[0121] Based on the calculation of the response latency increment of each node, the processing time of each node generating files, and the time of upstream node files being transmitted to each node, the processing time of each node generating the corresponding file is determined.

[0122] If a node is found where the processing time for generating the corresponding file exceeds the corresponding threshold, then backtracking is performed along the reverse path of the link model, starting from that node. Based on the node information of the backtracked node, the key node or link that caused the file generation timeout is identified and output.

[0123] Optionally, the batch job delay estimation device includes: an element relationship module 207, used for:

[0124] The nodes in the link model correspond to the node model. The node model is used to describe the node information, including the element information of the three entity elements: job processing, online call user, and file generation, as well as the relationships between the elements.

[0125] The relationships between elements include file dependencies describing upstream files that job processing depends on, upstream jobs that job processing depends on, invocation dependencies describing online calls, and relationships between files generated by job processing.

[0126] Optionally, the batch job delay estimation device includes: an event type determination module 208, used for:

[0127] Preset change events include at least one of the following: production change, off-site disaster recovery drill, and multi-site, multi-center migration deployment.

[0128] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 4 As shown, the electronic device of this embodiment may include: at least one processor 301; and a memory 302 communicatively connected to at least one processor; wherein the memory 302 stores instructions that can be executed by at least one processor 301, and the instructions are executed by at least one processor 301 to cause the electronic device to perform the method as described in any of the above embodiments.

[0129] Optionally, the memory 302 can be either standalone or integrated with the processor 301. When the memory 302 is set up independently, the device also includes a bus for connecting the memory 302 and the processor 301.

[0130] The implementation principle and technical effects of the electronic device provided in this embodiment can be found in the foregoing embodiments, and will not be repeated here.

[0131] This application also provides a computer-readable storage medium storing computer-executable instructions. When the computer-executable instructions are executed by a processor, the methods provided in any of the foregoing embodiments can be implemented.

[0132] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the method provided in any of the foregoing embodiments.

[0133] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0134] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0135] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0136] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0137] Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as resistive random access memory (RRAM), dynamic random access memory (DRAM), static random access memory (SRAM), enhanced dynamic random access memory (EDRAM), high-bandwidth memory (HBM), hybrid memory cube (HMC), etc.

[0138] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0139] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0140] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0141] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A batch job delay estimation method, characterized in that, include: In response to a detected preset change event, a link model of the distributed system for executing batch jobs is obtained. The link model includes multiple nodes connected in a topology for executing the batch jobs. The topology connections between nodes are used to characterize file dependencies, job dependencies, and online service call dependencies when nodes process batch jobs. Determine the target node and target online service corresponding to the preset change event, as well as the service delay time of each of the target node and the target online service; Based on the link model and the service latency of the target node and the target online service, the full-link response latency estimate of the batch job affected by the preset change event is generated.

2. The method according to claim 1, characterized in that, The node information includes the application to which it belongs and the application to which the online call service belongs; determining the target node and target online service corresponding to the preset change event, as well as the service delay time of the target node and the target online service respectively, includes: Based on the change order of the preset change event, determine the target application involved in the preset change event; Based on the node information of each node in the link model, the nodes and online calls belonging to the target application are determined respectively, which are the target nodes and target online calls; Extract the change time from the change order of the preset event; Based on the change time, the service delay time of the target node and the target online service is determined respectively.

3. The method according to claim 1, characterized in that, The step of generating an end-to-end response latency estimate for the batch job affected by the preset change event, based on the link model and the respective service latency times of the target node and the target online service, includes: Based on the service latency of the target node and the target online service, determine the response latency increment of the target online service; Based on the file dependencies and call dependencies of the nodes in the link model, and the response latency increment of the target online service, determine the response latency increment of the downstream nodes in the link model that are affected by the target node and the target online service; Based on the calculated response latency increment of each node, the estimated end-to-end response latency of the batch job affected by the preset change event is generated.

4. The method according to claim 3, characterized in that, The step of determining the response latency increment of the target online service based on the service latency time of the target node and the target online service includes: Input the service delay times of the target node and the target service into the effectiveness analysis model to obtain the response delay increment of the target online service.

5. The method according to claim 3, characterized in that, The response latency increment includes file transfer latency and job processing latency; determining the response latency increment of downstream nodes in the link model affected by the target node and the target online service, based on the file dependencies and call dependencies of nodes in the link model and the response latency increment of the target online service, includes: Based on the link model, the target node, and the target online service, determine the link affected by the preset change event; The nodes in the affected link are traversed according to a breadth-first search strategy. For each traversed node, the time change of the batch files reaching the node is calculated based on the size of the upstream files indicated by the file dependency of the node and the deployment zone of the upstream application, so as to obtain the file transmission delay of the node. The upstream application is the application to which the upstream node of the node belongs. If the node has online service call dependencies, then based on the service, application, and cluster corresponding to the call dependency, the time change of the node's online service is calculated to obtain the job processing delay of the node.

6. The method according to claim 3, characterized in that, The method further includes: Based on the calculation of the response latency increment of each node, the processing time of each node generating files, and the time of upstream node file transmission to each node, the processing time of each node generating the corresponding file is determined. If there is a node whose processing time for generating the corresponding file exceeds the corresponding threshold, then backtracking is performed along the reverse of the link model starting from that node. Based on the node information of the backtracked node, the key node or link that caused the file generation timeout is identified and output.

7. The method according to any one of claims 1-6, characterized in that, The nodes in the link model correspond to node models. The node models are used to describe the node information of the nodes, including the element information of the three entity elements of job processing, online user calling, and file generation, as well as the relationship between the elements. The relationships between elements include file dependencies describing upstream files that the job processing depends on, upstream jobs that the job processing depends on, call dependencies describing the online calls, and relationships between files generated by the job processing.

8. The method according to any one of claims 1-6, characterized in that, The preset change events include at least one of the following: production change, off-site disaster recovery drill, and multi-site, multi-center migration deployment.

9. A batch operation delay estimation device, characterized in that, include: The link model acquisition module is used to acquire the link model of the distributed system for executing batch jobs in response to a detected preset change event. The link model includes multiple nodes connected in the topology for executing the batch jobs. The topology connection between the nodes is used to characterize the file dependencies, job dependencies and online service call dependencies when the nodes process the batch jobs. The target determination module is used to determine the target node and target online service corresponding to the preset change event, as well as the service delay time of the target node and the target online service respectively; The impact assessment module is used to generate an estimate of the end-to-end response delay of the batch job affected by the preset change event, based on the link model and the service delay times of the target node and the target online service.

10. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-8.

12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1-8.

Images