Risk mining method and device in batch program running

Abstract

The invention discloses a risk mining method and device in batch program running, batch benchmark nodes are selected in the batch program running process, the benchmark nodes are reference standard nodes with representativeness and stability, for the batch benchmark nodes, difference calculation is carried out on subsequent nodes and the benchmark nodes to obtain a time difference, and the time difference is calculated according to the difference. And performing time prediction algorithm modeling through the time difference to dynamically predict the operation end time of the subsequent node of the preset future date, and if the operation end time of the subsequent node of the preset future date meets a risk condition, generating an operation risk report.

Description

Technical Field

[0001] The present application relates to the technical field of risk management, and more specifically, to a method and device for risk mining in batch program operation. Background Art

[0002] Batch programs in the financial field refer to programs that process, analyze, exchange or convert large amounts of data inside or outside a financial institution. They are usually executed according to predetermined times or event triggers, such as daily reconciliation, settlement, liquidation, report generation, etc.

[0003] The execution results of batch programs directly affect the business operations, risk management and compliance supervision of financial institutions. Therefore, the execution quality and efficiency of batch programs are crucial. However, due to the complexity, diversity and dependencies of batch programs, various risks may arise during their execution, such as data errors, program failures, timeout delays, dependency failures, etc. These risks may cause the execution results of batch programs to be inaccurate, incomplete or untimely, thus causing significant losses or risks to financial institutions.

[0004] Therefore, how to effectively monitor, identify and warn of risks during batch program execution is an urgent problem to be solved in this application. Summary of the invention

[0005] In view of this, the present application discloses a method and device for risk mining in batch program operation, aiming to effectively monitor, identify and warn of risks in the batch program execution process.

[0006] In order to achieve the above purpose, the disclosed technical solution is as follows:

[0007] In a first aspect, the present application discloses a risk mining method in batch program operation, the method comprising:

[0008] Selecting batches of benchmark nodes during batch program operation; wherein the benchmark nodes are representative and stable reference standard nodes;

[0009] For a batch of benchmark nodes, the difference between the subsequent nodes and the benchmark nodes is calculated to obtain the time difference;

[0010] A time prediction algorithm is modeled using the time difference to dynamically predict the operation end time of subsequent nodes on a preset future date;

[0011] If the operation end time of the subsequent node of the preset future date meets the risk condition, an operation risk report is generated.

[0012] Preferably, the step of selecting batches of benchmark nodes during batch program execution includes:

[0013] During the batch program running process, select a fixed node to start execution within a preset time period;

[0014] The nodes that start execution regularly within the preset time period are determined as the benchmark nodes of the batch.

[0015] Preferably, for the batch of benchmark nodes, performing difference calculation between the subsequent nodes and the benchmark node to obtain the time difference includes:

[0016] For batches of benchmark nodes, obtain the actual start time or actual end time of subsequent nodes, and obtain the actual start time or actual end time of the benchmark node;

[0017] Calculate the difference between the actual start time of the subsequent node and the actual start time of the benchmark node to obtain a time difference;

[0018] or,

[0019] The actual end time of the subsequent node is calculated to differ from the actual end time of the benchmark node to obtain a time difference.

[0020] Preferably, the time prediction algorithm modeling is performed using the time difference to dynamically predict the operation end time of the subsequent node of the preset future date, including:

[0021] Establishing a linear regression model according to the time difference;

[0022] Obtain the actual start time and end time of the benchmark node of the preset future date, and obtain the statistical characteristics of the historical data;

[0023] According to the linear regression model, the time prediction algorithm, the actual start time of the benchmark node of the preset future date, the actual end time of the benchmark node of the preset future date and the statistical characteristics of the historical data, the operation end time of the subsequent node of the preset future date is dynamically predicted; wherein, the time prediction algorithm is a time prediction algorithm for distributed representation of entities and relationships.

[0024] Preferably, if the operation end time of the subsequent node of the preset future date meets the risk condition, generating an operation risk report includes:

[0025] Comparing the operation end time of the subsequent node of the preset future date with the historical average upper limit; wherein the historical average upper limit is determined by the average value of the operation end time of the subsequent node calculated according to the historical data and a preset threshold value;

[0026] If the operation end time of the subsequent node of the preset future date is greater than the historical average upper limit, it is determined that the operation end time of the subsequent node of the preset future date meets the risk condition;

[0027] When the operation end time of the subsequent node of the preset future date meets the risk condition, an operation risk report is generated.

[0028] Preferably, it also includes:

[0029] If the operation end time of the subsequent node of the preset future date is less than or equal to the historical average upper limit, it is determined that the operation end time of the subsequent node of the preset future date does not meet the risk condition, and it is determined that there is no operation risk for the subsequent node.

[0030] The second aspect of the present application discloses a risk mining device in batch program operation, the device comprising:

[0031] A selection unit, used for selecting batches of benchmark nodes during the batch program running process; wherein the benchmark nodes are reference standard nodes with representativeness and stability;

[0032] A calculation unit, for calculating the difference between the subsequent nodes and the benchmark nodes for a batch of benchmark nodes, to obtain a time difference;

[0033] A dynamic prediction unit, used to perform time prediction algorithm modeling through the time difference, so as to dynamically predict the operation end time of the subsequent node of the preset future date;

[0034] A generating unit is used to generate an operation risk report if the operation end time of the subsequent node of the preset future date meets the risk condition.

[0035] Preferably, the selection unit includes:

[0036] A selection module is used to select a fixed node to start execution within a preset time period during the batch program running process;

[0037] The first determination module is used to determine the nodes that start execution fixedly within the preset time period as the benchmark nodes of the batch.

[0038] Preferably, the computing unit comprises:

[0039] A first acquisition module is used to acquire the actual start time or actual end time of subsequent nodes and the actual start time or actual end time of the benchmark node for a batch of benchmark nodes;

[0040] A first calculation module, used for calculating the difference between the actual start time of the subsequent node and the actual start time of the benchmark node to obtain a time difference;

[0041] or,

[0042] The second calculation module is used to calculate the difference between the actual end time of the subsequent node and the actual end time of the benchmark node to obtain a time difference.

[0043] Preferably, the dynamic prediction unit comprises:

[0044] An establishment module is used to establish a linear regression model according to the time difference;

[0045] A second acquisition module is used to acquire the actual start time of the benchmark node of the preset future date and the actual end time of the benchmark node of the preset future date, and acquire the statistical characteristics of the historical data;

[0046] A dynamic prediction module is used to dynamically predict the operation end time of subsequent nodes of a preset future date based on the linear regression model, the time prediction algorithm, the actual start time of the benchmark node of a preset future date, the actual end time of the benchmark node of a preset future date and the statistical characteristics of the historical data; wherein the time prediction algorithm is a time prediction algorithm for distributed representation of entities and relationships.

[0047] It can be seen from the above technical solution that the present application discloses a method and device for risk mining in batch program operation, in which a batch of benchmark nodes are selected during the batch program operation, wherein the benchmark node is a reference standard node with representativeness and stability, and for the batch of benchmark nodes, the difference between the subsequent nodes and the benchmark node is calculated to obtain the time difference, and the time prediction algorithm is modeled based on the time difference to dynamically predict the operation end time of the subsequent node of the preset future date, and if the operation end time of the subsequent node of the preset future date meets the risk condition, an operation risk report is generated.

[0048] Through the above scheme, there is no need to manually configure the running line mining algorithm of the batch alarm baseline. It is only necessary to select the benchmark nodes of the batch, calculate the time difference between the subsequent nodes and the benchmark nodes, and model the difference data, so as to dynamically predict the running end time of the subsequent nodes of the preset future date through the established linear regression model. If the running end time of the subsequent nodes of the preset future date meets the risk conditions, various risks that may occur during the batch program execution, such as data errors, program failures, timeout delays, dependency failures, etc., can be automatically mined, realizing effective monitoring, identification and early warning of risks in the batch program execution process. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0050] Figure 1 A flow chart of a risk mining method in batch program operation disclosed in an embodiment of the present application;

[0051] Figure 2 A schematic diagram of a batch program execution process disclosed in an embodiment of the present application;

[0052] Figure 3 A schematic diagram of the structure of a risk mining device in batch program operation disclosed in an embodiment of the present application;

[0053] Figure 4 This is a schematic diagram of the structure of an electronic device disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0055] In this application, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0056] As can be seen from the background technology, the execution results of batch programs directly affect the business operations, risk management and compliance supervision of financial institutions. Therefore, the execution quality and efficiency of batch programs are crucial. However, due to the complexity, diversity and dependence of batch programs, various risks may occur during their execution, such as data errors, program failures, timeout delays, dependency failures, etc. These risks may cause the execution results of batch programs to be inaccurate, incomplete or untimely, thereby causing significant losses or risks to financial institutions. Therefore, how to effectively monitor, identify and warn of risks in the execution process of batch programs is a problem that this application urgently needs to solve.

[0057] In order to solve the above problems, the present application discloses a method and device for risk mining in batch program operation, which does not require manual configuration of the operation line mining algorithm of the batch alarm baseline. It is only necessary to select the benchmark nodes of the batch, calculate the time difference between the subsequent nodes and the benchmark nodes, and model the difference data, so as to dynamically predict the operation end time of the subsequent nodes of the preset future date through the established linear regression model. If the operation end time of the subsequent nodes of the preset future date meets the risk conditions, various risks that may occur during the execution of the batch program, such as data errors, program failures, timeout delays, dependency failures, etc., are automatically mined for risks, thereby effectively monitoring, identifying and warning the risks in the execution of the batch program. The specific implementation method is specifically described through the following embodiments.

[0058] refer to Figure 1 As shown, a risk mining method in batch program operation disclosed in an embodiment of the present application is provided. The risk mining method in batch program operation mainly includes the following steps:

[0059] S101: Selecting a batch of benchmark nodes during batch program execution; wherein the benchmark nodes are reference standard nodes that are representative and stable.

[0060] In S101 , during the batch program running process, nodes that start execution regularly within a preset time period are selected, and the nodes that start execution regularly within the preset time period are determined as benchmark nodes.

[0061] Selecting a batch benchmark node means selecting one or more representative and stable nodes as reference standards in batch operation. Generally, a scheduled running node is selected as a benchmark node, that is, a node with a fixed start and end in each time period. For example, in daily batch operation, the node that starts at 0:00 am every day can be selected as a benchmark node.

[0062] Batch programs include but are not limited to batch programs in the financial field.

[0063] Batch programs in the financial field refer to programs that process, analyze, exchange or convert large amounts of data inside or outside a financial institution. They are usually executed according to a predetermined time or event trigger, such as daily reconciliation, settlement, liquidation, report generation, etc. The execution results of batch programs directly affect the business operations, risk management and compliance supervision of financial institutions. Therefore, the execution quality and efficiency of batch programs are crucial.

[0064] The preset time period can be from 0:00 to 2:00 in the morning every day, or from 3:00 to 4:00, etc. The preset time period is set according to actual conditions and this application does not make any specific limitations.

[0065] In order to facilitate the understanding of the process of selecting batch benchmark nodes during batch program operation, combined with Figure 2 Let's take an example:

[0066] For example, suppose a financial institution needs to execute a batch program that obtains data from multiple data sources, aggregates, verifies, converts, analyzes, and generates a report file, which is then sent to relevant departments and institutions.

[0067] The batch program needs to be executed at 8:00 a.m. every working day and completed before 9:00 a.m. The batch program involves multiple subprograms, tasks, events, etc. In the daily batch operation, the node that starts execution at 8:00 a.m. every working day is selected as the benchmark node.

[0068] S102: For the batch of benchmark nodes, calculate the difference between the subsequent nodes and the benchmark node to obtain the time difference.

[0069] For batches of benchmark nodes, calculating the time difference between subsequent nodes and the benchmark node refers to calculating the time difference between each subsequent node and the benchmark node based on historical data.

[0070] Among them, benchmark nodes refer to one or more representative and stable nodes selected in batch operation and used as reference standards.

[0071] Subsequent nodes are nodes that start executing after the benchmark node.

[0072] A node is a basic unit in batch operation, which can be a program, a command, a script, or a data file.

[0073] The time difference is the difference between the actual start time of the subsequent node and the actual start time of the benchmark node, or the difference between the actual end time of the subsequent node and the actual end time of the benchmark node. For example, in a daily batch run, the difference between the actual start time of each subsequent node and the actual start time of the benchmark node can be calculated.

[0074] Specifically for the batch of benchmark nodes, the difference between the subsequent nodes and the benchmark nodes is calculated to obtain the time difference process as shown in A1-A3.

[0075] A1: For batches of benchmark nodes, obtain the actual start time or actual end time of subsequent nodes, and obtain the actual start time or actual end time of the benchmark node;

[0076] A2: Calculate the difference between the actual start time of the subsequent node and the actual start time of the benchmark node to obtain the time difference.

[0077] or,

[0078] A3: Calculate the difference between the actual end time of the subsequent node and the actual end time of the benchmark node to obtain the time difference.

[0079] S103: Modeling a time prediction algorithm based on the time difference to dynamically predict the operation end time of the subsequent nodes of a preset future date (a new day).

[0080] In S103, a mathematical model is established based on the above difference data, and a neural symbolic reasoning engine (NSRE) time prediction algorithm is performed based on the mathematical model to dynamically predict the operation end time of the subsequent node of the preset future date.

[0081] Modeling refers to establishing a mathematical model based on difference data, such as a linear regression model, a neural network model, etc., which is used to describe the relationship between the difference data and the end time of the node operation.

[0082] The preset future date is a new day. For example, if the current date is November 10, the NSRE time prediction algorithm is performed on the mathematical model on November 10 to dynamically predict the operation end time of the subsequent nodes on November 11, or dynamically predict the operation end time of the subsequent nodes on November 12.

[0083] Specifically, the process of modeling the time prediction algorithm through the time difference to dynamically predict the operation end time of the subsequent node of the preset future date is shown in B1-B3.

[0084] B1: Establish a linear regression model based on the time difference.

[0085] B2: Obtain the actual start time and the actual end time of the benchmark node of the preset future date, and obtain the statistical characteristics of the historical data.

[0086] For example, in daily batch operation, a linear regression model can be established based on the difference data between each subsequent node and the benchmark node. The actual start time and actual end time of the benchmark node of the new day, as well as the mean and variance of the difference data, can be used to dynamically predict the operation end time of the subsequent nodes of the new day.

[0087] B3: Based on the linear regression model, the NSRE time prediction algorithm, the actual start time of the benchmark node of the preset future date, the actual end time of the benchmark node of the preset future date and the statistical characteristics of historical data, the operation end time of the subsequent node of the preset future date is dynamically predicted; wherein the time prediction algorithm is a time prediction algorithm for the distributed representation of entities and relationships.

[0088] Entities include but are not limited to programs, data, tasks, events, etc.

[0089] Relationships include but are not limited to dependence, influence, transmission, etc.

[0090] The detailed implementation principle of the NSRE time prediction algorithm, including entity and relationship representation, entity and relationship reasoning, and entity and relationship query.

[0091] 1) Entity and relationship representation:

[0092] This solution uses a neural network model, such as a graph embedding model or a knowledge graph embedding model, to map various entities (such as programs, data, tasks, events, etc.) and relationships (such as dependencies, influences, transfers, etc.) involved in the batch program execution process into a low-dimensional vector space, thereby achieving a distributed representation of entities and relationships. In this way, the powerful learning ability of neural networks can be used to capture the semantic information and potential rules between entities and relationships. Specifically, formula (1) can be used to calculate the vector representation of entities and relationships.

[0093]

[0094] Among them, e i is the vector representation of the i-th entity; E i is the original feature of the i-th entity; f is a neural network, such as a multilayer perceptron or a convolutional neural network; r j is the vector representation of the jth relationship; R j is the original feature of the jth relationship; g is a neural network model, such as a multilayer perceptron or a convolutional neural network.

[0095] 2) Entity and relationship reasoning: Using symbolic logic models, such as first-order logic or description logic, various entities and relationships involved in the batch program execution process are abstracted into a set of logical formulas or ontology descriptions, thereby realizing symbolic reasoning of entities and relationships. In this way, the strict normativeness of symbolic logic can be used to express the clear meaning and constraints between entities and relationships. Specifically, the following formula (2) can be used to express the logical relationship between entities and relationships:

[0096]

[0097] Among them, r1, r2, r3, r4, and r5 are all relationships, such as dependency, influence, and transfer; e1 and e2 are different entity types, such as program, data, task, and event; ∧ is a logical connective, indicating and; It is a logical connective, indicating implication; It is a logical conjunction, indicating negation; x, y, and z are all entities.

[0098] 3) Entity and relationship queries:

[0099] The neural network model and the symbolic logic model can be connected by using the neural symbolic interface module to realize the neural symbolic query of entities and relationships. In this way, the vector representation provided by the neural network model can be used as the input or output of the symbolic logic model, thereby enhancing the expression and generalization ability of the symbolic logic model. Specifically, the neural symbolic query can be realized by using formula (3):

[0100]

[0101] Among them, q is the vector representation of the query; Q is the original feature of the query; h is a neural network model, such as a recurrent neural network or a self-attention network, which is used to convert the query into a vector; a is the logical representation of the answer; s is a symbolic logic model, such as first-order logic or a description logic inference engine, which is used to reason based on the query and entity relationship; e is the vector representation of the entity; r is the vector representation of the relationship; A is the original feature of the answer; i is a neural network model, such as a multilayer perceptron or a convolutional neural network, which is used to convert the answer into the original feature.

[0102] S104: If the operation end time of the subsequent node of the preset future date meets the risk condition, an operation risk report is generated.

[0103] In S104, if it is found that the operation end time of the subsequent node of the preset future date exceeds (ie is greater than) the historical average upper limit, it is determined that the operation end time of the subsequent node of the preset future date meets the risk condition, and an operation risk report is generated.

[0104] It should be noted that the historical average upper limit refers to the average value of the node operation end time calculated based on historical data plus a preset threshold.

[0105] The operation risk report is used to report operation risks. Operation risk refers to judging whether there is operation risk based on the comparison between the predicted node operation end time and historical data.

[0106] Specifically, if the operation end time of the subsequent node of the preset future date meets the risk condition, the process of generating the operation risk report is shown in C1-C3.

[0107] C1: Compare the running end time of the subsequent nodes of the preset future date with the historical average upper limit; wherein the historical average upper limit is determined by the average value of the running end time of the subsequent nodes calculated based on the historical data and the preset threshold.

[0108] The preset threshold can be set according to the actual situation, such as standard deviation, percentage, etc. If the predicted node operation end time exceeds the historical average upper limit, it means that the node may be abnormal or delayed, and there is an operation risk. At this time, it is necessary to report the operation risk and give corresponding prompts or suggestions, such as reminding relevant personnel to check the node status, adjust the node priority, adjust the batch operation plan, etc.

[0109] C2: If the running end time of the subsequent node of the preset future date is greater than the historical average upper limit, it is determined that the running end time of the subsequent node of the preset future date meets the risk condition.

[0110] C3: Generate an operation risk report when the operation end time of the subsequent node of the preset future date meets the risk condition.

[0111] If the operation end time of the subsequent node of the preset future date is less than or equal to the historical average upper limit, it is determined that the operation end time of the subsequent node of the preset future date does not meet the risk condition, and it is determined that there is no operation risk for the subsequent node.

[0112] The core idea of ​​this application is to comprehensively utilize rules, machine learning and risk identification technology based on neural symbolic reasoning to construct a multi-level, multi-dimensional, multi-modal risk mining framework, so as to achieve comprehensive, accurate, timely and intelligent identification and early warning of risks in the execution process of batch programs through a multi-level, multi-dimensional, multi-modal risk mining framework.

[0113] The present application also provides a risk mining system in batch program operation, and the risk mining system in batch program operation includes the following components:

[0114] Data collection component, used to collect historical data and real-time data of batch operations, including the start time, end time, dependencies, etc. of each node;

[0115] Data processing components, used to clean, convert, and store the collected data;

[0116] Data analysis component, used to analyze the data after cleaning, conversion, storage, etc., including selecting batch benchmark nodes, calculating the time difference between subsequent nodes and benchmark nodes, modeling the difference data, and dynamically predicting the end time of the new day's node operation;

[0117] The risk judgment component is used to compare the predicted node operation end time with historical data to determine whether there is an operation risk;

[0118] The risk reporting component is used to report operational risks when they exist and give corresponding prompts or suggestions.

[0119] This application can not only effectively monitor, identify and warn of risks in the execution of batch programs, but also provide corresponding processing suggestions or measures, thereby improving the execution quality and efficiency of batch programs and reducing the operating costs and risk losses of financial institutions.

[0120] The advantages of this application are as follows:

[0121] First, this application realizes risk mining of batch operation by selecting batch benchmark nodes, calculating the time difference between subsequent nodes and benchmark nodes, modeling the difference data, and dynamically predicting the end time of the node operation of the new day;

[0122] Second, it has high versatility and scalability, and can be applied to the risk identification of financial batch program operations of different types, scales, and scenarios;

[0123] Third, it has high interpretability and reliability, and can provide clear, reasonable and verifiable risk identification results and recommended solutions.

[0124] This application does not require manual configuration of the operation line mining algorithm of the batch alarm baseline. It can adaptively mine daily batch risks based on historical operation conditions, and through dynamic evaluation and intelligent recommendation of risk events that occur during the operation of financial batch programs, it realizes effective management and control of the operation risks of financial batch programs.

[0125] The beneficial effect of the embodiment of the present application: there is no need to manually configure the running line mining algorithm of the batch alarm baseline. It is only necessary to select the benchmark nodes of the batch, calculate the time difference between the subsequent nodes and the benchmark nodes, and model the difference data, so as to dynamically predict the running end time of the subsequent nodes of the preset future date through the established linear regression model. If the running end time of the subsequent nodes of the preset future date meets the risk conditions, various risks that may occur during the execution of the batch program, such as data errors, program failures, timeout delays, dependency failures, etc., are automatically mined, and the risks in the execution of the batch program are effectively monitored, identified and warned of.

[0126] Based on the above embodiments Figure 1 A risk mining method in batch program operation is disclosed, and an embodiment of the present application also discloses a risk mining device in batch program operation, such as Figure 3 As shown, the risk mining device in the batch program operation includes:

[0127] The selection unit 301 is used to select a batch of benchmark nodes during the batch program running process; wherein the benchmark node is a reference standard node with representativeness and stability;

[0128] The calculation unit 302 is used to calculate the difference between the subsequent nodes and the benchmark node for the batch of benchmark nodes to obtain the time difference;

[0129] The dynamic prediction unit 303 is used to perform time prediction algorithm modeling through the time difference to dynamically predict the operation end time of the subsequent node of the preset future date;

[0130] The generating unit 304 is used to generate an operation risk report if the operation end time of the subsequent node of the preset future date meets the risk condition.

[0131] Furthermore, the selection unit 301 includes:

[0132] A selection module is used to select a fixed node to start execution within a preset time period during the batch program running process;

[0133] The first determination module is used to determine nodes that start execution within a preset time period as benchmark nodes of the batch.

[0134] Furthermore, the calculation unit 302 includes:

[0135] A first acquisition module is used to acquire the actual start time or actual end time of subsequent nodes and the actual start time or actual end time of the benchmark node for a batch of benchmark nodes;

[0136] The first calculation module is used to calculate the difference between the actual start time of the subsequent node and the actual start time of the benchmark node to obtain a time difference;

[0137] or,

[0138] The second calculation module is used to calculate the difference between the actual end time of the subsequent node and the actual end time of the benchmark node to obtain a time difference.

[0139] Furthermore, the dynamic prediction unit 303 includes:

[0140] Establish a module for establishing a linear regression model based on time difference;

[0141] A second acquisition module is used to acquire the actual start time of the benchmark node of the preset future date and the actual end time of the benchmark node of the preset future date, and acquire the statistical characteristics of the historical data;

[0142] The dynamic prediction module is used to dynamically predict the operation end time of the subsequent nodes of the preset future date based on the linear regression model, the time prediction algorithm, the actual start time of the benchmark node of the preset future date, the actual end time of the benchmark node of the preset future date and the statistical characteristics of historical data; wherein the time prediction algorithm is a time prediction algorithm for the distributed representation of entities and relationships.

[0143] Furthermore, the generating unit 304 includes:

[0144] A comparison module, used to compare the operation end time of the subsequent node of the preset future date with the historical average upper limit; wherein the historical average upper limit is determined by the average value of the operation end time of the subsequent node calculated based on the historical data and a preset threshold value;

[0145] A determination module, for determining that the operation end time of the subsequent node of the preset future date meets the risk condition if the operation end time of the subsequent node of the preset future date is greater than the historical average upper limit;

[0146] The generation module is used to generate an operation risk report when the operation end time of the subsequent node of the preset future date meets the risk condition.

[0147] Furthermore, the risk mining device in batch program operation also includes:

[0148] A determination unit is used to determine that the operation end time of the subsequent node of the preset future date does not meet the risk condition if the operation end time of the subsequent node of the preset future date is less than or equal to the historical average upper limit, and determine that there is no operation risk for the subsequent node.

[0149] The beneficial effects of the implementation of this application: There is no need to manually configure the running line mining algorithm of the batch alarm baseline. It is only necessary to select the benchmark nodes of the batch, calculate the time difference between the subsequent nodes and the benchmark nodes, and model the difference data, so as to dynamically predict the running end time of the subsequent nodes of the preset future date through the established linear regression model. If the running end time of the subsequent nodes of the preset future date meets the risk conditions, various risks that may occur during the execution of the batch program, such as data errors, program failures, timeout delays, dependency failures, etc., are automatically mined, and the risks in the execution of the batch program are effectively monitored, identified and warned of.

[0150] An embodiment of the present application further provides a storage medium, which includes stored instructions, wherein when the instructions are executed, the device where the storage medium is located is controlled to execute the risk mining method in the batch program operation as described above.

[0151] The present application also provides an electronic device, whose structural diagram is as follows: Figure 4 As shown, it specifically includes a memory 401 and one or more instructions 402, wherein the one or more instructions 402 are stored in the memory 401 and are configured to be executed by one or more processors 403 to execute the one or more instructions 402 to execute the risk mining method in the above-mentioned batch program operation.

[0152] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0153] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0154] The steps in the methods of the embodiments of the present application can be adjusted in order, combined, and deleted according to actual needs.

[0155] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

[0156] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

[0157] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A risk mining method in batch program operation, characterized in that: The method comprises: Selecting batches of benchmark nodes during batch program operation; wherein the benchmark nodes are representative and stable reference standard nodes; For a batch of benchmark nodes, the difference between the subsequent nodes and the benchmark nodes is calculated to obtain the time difference; A time prediction algorithm is modeled using the time difference to dynamically predict the operation end time of subsequent nodes on a preset future date; If the operation end time of the subsequent node of the preset future date meets the risk condition, an operation risk report is generated.

2. The method according to claim 1, characterized in that The step of selecting batches of benchmark nodes during batch program operation includes: During the batch program running process, select a fixed node to start execution within a preset time period; The nodes that start execution fixedly within the preset time period are determined as the benchmark nodes of the batch.

3. The method according to claim 1, characterized in that For the batch of benchmark nodes, performing difference calculation between the subsequent nodes and the benchmark node to obtain the time difference includes: For batches of benchmark nodes, obtain the actual start time or actual end time of subsequent nodes, and obtain the actual start time or actual end time of the benchmark node; Calculate the difference between the actual start time of the subsequent node and the actual start time of the benchmark node to obtain a time difference; or, The actual end time of the subsequent node is calculated to differ from the actual end time of the benchmark node to obtain a time difference.

4. The method according to claim 1, characterized in that: The time prediction algorithm modeling is performed by using the time difference to dynamically predict the operation end time of the subsequent node of the preset future date, including: Establishing a linear regression model according to the time difference; Obtain the actual start time and end time of the benchmark node of the preset future date, and obtain the statistical characteristics of the historical data; According to the linear regression model, the time prediction algorithm, the actual start time of the benchmark node of the preset future date, the actual end time of the benchmark node of the preset future date and the statistical characteristics of the historical data, the operation end time of the subsequent node of the preset future date is dynamically predicted; wherein, the time prediction algorithm is a time prediction algorithm for distributed representation of entities and relationships.

5. The method according to claim 1, characterized in that If the operation end time of the subsequent node of the preset future date meets the risk condition, an operation risk report is generated, including: Comparing the operation end time of the subsequent node of the preset future date with the historical average upper limit; wherein the historical average upper limit is determined by the average value of the operation end time of the subsequent node calculated according to the historical data and a preset threshold value; If the operation end time of the subsequent node of the preset future date is greater than the historical average upper limit, it is determined that the operation end time of the subsequent node of the preset future date meets the risk condition; When the operation end time of the subsequent node of the preset future date meets the risk condition, an operation risk report is generated.

6. The method according to claim 5, characterized in that Also includes: If the operation end time of the subsequent node of the preset future date is less than or equal to the historical average upper limit, it is determined that the operation end time of the subsequent node of the preset future date does not meet the risk condition, and it is determined that there is no operation risk for the subsequent node.

7. A risk mining device in batch program operation, characterized in that: The device comprises: A selection unit, used for selecting batches of benchmark nodes during the batch program running process; wherein the benchmark nodes are reference standard nodes with representativeness and stability; A calculation unit, for calculating the difference between the subsequent nodes and the benchmark nodes for a batch of benchmark nodes, to obtain a time difference; A dynamic prediction unit, used to perform time prediction algorithm modeling through the time difference, so as to dynamically predict the operation end time of the subsequent node of the preset future date; A generating unit is used to generate an operation risk report if the operation end time of the subsequent node of the preset future date meets the risk condition.

8. The device according to claim 7, characterized in that The selection unit comprises: A selection module is used to select a fixed node to start execution within a preset time period during the batch program running process; The first determination module is used to determine the nodes that start execution fixedly within the preset time period as the benchmark nodes of the batch.

9. The device according to claim 7, characterized in that The computing unit comprises: A first acquisition module is used to acquire the actual start time or actual end time of subsequent nodes and the actual start time or actual end time of the benchmark node for a batch of benchmark nodes; A first calculation module, used for calculating the difference between the actual start time of the subsequent node and the actual start time of the benchmark node to obtain a time difference; or, The second calculation module is used to calculate the difference between the actual end time of the subsequent node and the actual end time of the benchmark node to obtain a time difference.

10. The device according to claim 7, characterized in that The dynamic prediction unit comprises: An establishment module is used to establish a linear regression model according to the time difference; A second acquisition module is used to acquire the actual start time of the benchmark node of the preset future date and the actual end time of the benchmark node of the preset future date, and acquire the statistical characteristics of the historical data; A dynamic prediction module is used to dynamically predict the operation end time of subsequent nodes of a preset future date based on the linear regression model, the time prediction algorithm, the actual start time of the benchmark node of a preset future date, the actual end time of the benchmark node of a preset future date and the statistical characteristics of the historical data; wherein the time prediction algorithm is a time prediction algorithm for distributed representation of entities and relationships.

Images