Fault prediction method and device for data transmission system, equipment and medium

Abstract

The invention provides a fault prediction method and device for a data transmission system, equipment and a medium, and relates to the field of data analysis, in particular to the field of big data analysis and artificial intelligence. The method comprises the following steps: acquiring a first data transmission quantity of any target module for outputting data to other modules and a second data transmission quantity of other modules for inputting data to the target module at any target moment; respectively comparing the first data transmission quantity with a corresponding first excitation threshold parameter and comparing the second data transmission quantity with a corresponding second excitation threshold parameter by utilizing a preset excitation function, and respectively generating a first output numerical value and a second output numerical value according to comparison results; and according to the first output numerical value and the second output numerical value, calculating an estimated total transmission amount of the target module at the target moment, comparing the estimated total transmission amount with a risk threshold parameter, and according to a comparison result, predicting whether the target module has a fault or not. The method can effectively predict whether the system will have a fault in the future.

Description

Technical Field

[0001] The present application relates to the field of data analysis technology, in particular to the field of big data analysis and artificial intelligence, and specifically to a fault prediction method, device, equipment and medium for a data transmission system. Background Art

[0002] For any system, it can only operate normally if it is composed of various modules and coordinated with each other. The network formed by these modules is the basic layout of the system. Each module in the network has a certain risk of failure, which in turn brings security risks to the normal operation of the entire network system.

[0003] In the financial field, a data transmission system such as a financial data storage and scheduling system is one such system. It consists of multiple components or modules, which are interconnected and have frequent data transmission exchanges to form a data transmission network. How to comprehensively predict and evaluate the system risks in the data transmission system is very important in the financial field. Summary of the invention

[0004] The present application provides a method, device, equipment and medium for predicting a fault of a data transmission system, so as to solve the problem in the prior art that it is impossible to evaluate and predict the faults of the data transmission system as a whole.

[0005] In a first aspect, the present application provides a fault prediction method for a data transmission system, the data transmission system comprising multiple modules, and the method comprising:

[0006] Acquire a first data transmission volume of data output from any target module to other modules, and a second data transmission volume of data input from other modules to the target module at any target time;

[0007] Using a preset excitation function, respectively comparing the first data transmission amount with a corresponding first excitation threshold parameter, and comparing the second data transmission amount with a corresponding second excitation threshold parameter, and respectively generating a first output value and a second output value according to the comparison results, wherein the excitation function is used to measure the load pressure of the target module according to the amount by which the data transmission amount exceeds the corresponding excitation threshold parameter;

[0008] Calculating the estimated total transmission amount of the target module at the target time according to the first output value and the second output value, and comparing the estimated total transmission amount with a risk threshold parameter, and predicting whether the target module fails according to the comparison result;

[0009] The first excitation threshold parameter, the second excitation threshold parameter and the risk threshold parameter are determined by parameter fitting using sample data of data transmission volume and fault information of each module of the data transmission system in different time periods.

[0010] In a second aspect, the present application provides a fault prediction device for a data transmission system, wherein the data transmission system includes multiple modules, and the device includes:

[0011] A data transmission volume acquisition module, used to acquire, at any target moment, a first data transmission volume of data output from any target module to other modules, and a second data transmission volume of data input from other modules to the target module;

[0012] an excitation function processing module, used to compare the first data transmission amount with the corresponding first excitation threshold parameter, and compare the second data transmission amount with the corresponding second excitation threshold parameter using a preset excitation function, and to generate a first output value and a second output value according to the comparison results, respectively, wherein the excitation function is used to measure the load pressure of the target module according to the amount by which the data transmission amount exceeds the corresponding excitation threshold parameter;

[0013] a prediction module, configured to calculate an estimated total transmission amount of the target module at the target time according to the first output value and the second output value, compare the estimated total transmission amount with a risk threshold parameter, and predict whether the target module fails according to the comparison result;

[0014] The first excitation threshold parameter, the second excitation threshold parameter and the risk threshold parameter are determined by parameter fitting using sample data of data transmission volume and fault information of each module of the data transmission system in different time periods.

[0015] In a third aspect, the present application also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a fault prediction method for a data transmission system as described in any one of the embodiments of the present application is implemented.

[0016] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a fault prediction method for a data transmission system as described in any one of the embodiments of the present application.

[0017] In a fifth aspect, the present application further provides a computer program product, including a computer program, which, when executed by a processor, implements a fault prediction method for a data transmission system as described in any one of the embodiments of the present application.

[0018] The fault prediction method, device, equipment and medium of the data transmission system provided by the present application are based on the first data transmission volume output by any target module in the data transmission system to other modules, and the second data transmission volume input by other modules to the target module, and the preset excitation function is used to compare the two with the first excitation threshold parameter and the second excitation threshold parameter respectively, and the first output value and the second output value are generated respectively according to the comparison result. Then, the estimated total transmission volume of the target module is calculated according to the first output value and the second output value, and finally it is compared with the risk threshold parameter, and whether the target module fails is predicted according to the comparison result. Moreover, the first excitation threshold parameter, the second excitation threshold parameter and the risk threshold parameter are determined by parameter fitting using the sample data of the data transmission volume and fault information of each module in the data transmission system in different time periods. It can be seen that the present application measures the load pressure of the target module according to the amount by which the data transmission volume exceeds the corresponding excitation threshold parameter by designing the excitation function and fitting the parameters involved, obtains the estimated total transmission volume by calculation, and realizes fault prediction according to the comparison result of the estimated total transmission volume and the risk threshold parameter. Therefore, the present application can predict whether each module in the data transmission system will fail in the future from the overall perspective, and has good adaptability and effectiveness. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 A flowchart of a method for predicting a fault in a data transmission system provided in an embodiment of the present application;

[0021] Figure 2 A schematic diagram of inter-module data transmission provided in an embodiment of the present application;

[0022] Figure 3 A flowchart for predicting whether a target module fails in a fault prediction method provided in an embodiment of the present application;

[0023] Figure 4 A flow chart of parameter fitting in the fault prediction method provided in an embodiment of the present application;

[0024] Figure 5 A flowchart for determining whether a risk conduction model fails in a fault prediction method provided in an embodiment of the present application;

[0025] Figure 6A schematic diagram of the structure of a fault prediction device for a data transmission system provided in an embodiment of the present application;

[0026] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0027] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0028] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only the parts related to the present application, rather than all structures, are shown in the accompanying drawings.

[0029] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance. The acquisition, storage, use, processing, etc. of data in the technical solution of this application comply with the relevant provisions of national laws and regulations. It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. The purpose is only to illustrate the feasibility of the implementation of the technical solution of this application, but it does not mean that the applicant has or must use the relevant content of the solution.

[0030] Figure 1 A flowchart of a method for predicting a fault in a data transmission system provided in an embodiment of the present application. This embodiment is applicable to predicting whether each module in a data transmission system and the entire system will fail in the future. The method can be performed by a fault prediction device of the data transmission system, which can be implemented in software and / or hardware, preferably configured in an electronic device, such as a computer device or a server. The data transmission system includes multiple modules. Figure 1 As shown, the method specifically includes:

[0031] S101. Obtain, at any target time, a first data transmission rate of data output from any target module to other modules, and a second data transmission rate of data input from other modules to the target module.

[0032] S102, using a preset excitation function, respectively comparing the first data transmission amount with the corresponding first excitation threshold parameter, and comparing the second data transmission amount with the corresponding second excitation threshold parameter, and respectively generating a first output value and a second output value according to the comparison results.

[0033] The excitation function is used to measure the load pressure of the target module according to the amount by which the data transmission volume exceeds the corresponding excitation threshold parameter.

[0034] S103, calculating the estimated total transmission amount of the target module at the target time according to the first output value and the second output value, and comparing the estimated total transmission amount with the risk threshold parameter, and predicting whether the target module fails according to the comparison result.

[0035] The first excitation threshold parameter, the second excitation threshold parameter and the risk threshold parameter are determined by parameter fitting using the sample data of the data transmission volume and fault information of each module of the data transmission system in different time periods.

[0036] Specifically, the data transmission system is composed of multiple modules, each module is responsible for a part of the function, and the combination of multiple modules makes the system as a whole a tool with complete functions and capable of achieving specific goals. Taking the customer data storage and scheduling system commonly used by financial enterprises as an example, it generally includes multiple modules such as data storage area, data cache area, calculation and analysis module, data transmission module, loading and unloading module, front-end display and editing module, and security verification module. For example, these modules are marked as M1, M2, M3, M4, ..., M5, M6, M7, M8, M9, M10, M11, M12, M13, M14, M15, M16, M17, M18, M19, M20, M21, M22, M33, M44, M55, M66, M77, M8 n , n is the total number of modules. Figure 2 A schematic diagram of data transmission between modules provided in an embodiment of the present application. As shown in the figure, these modules are connected to each other through connecting lines, and the connecting lines indicate that there is data transmission between the modules, and a data transmission network is formed as a whole to maintain the normal operation of the enterprise.

[0037] This application is based on the first data transmission rate of each module outputting data to other modules and the second data transmission rate of other modules inputting data to the module, and through the constructed model, it can predict whether the target module will fail in the future, and then realize the failure prediction and risk management of the entire system.

[0038] In the constructed model, the load pressure of the target module is measured by the excitation function according to the amount by which the data transmission volume exceeds the corresponding excitation threshold parameter. Specifically, the first data transmission volume is compared with the corresponding first excitation threshold parameter, and the second data transmission volume is compared with the corresponding second excitation threshold parameter, and the first output value and the second output value are generated according to the comparison results. The first output value and the second output value can be used to further calculate the estimated total transmission volume of the target module at the target time. If the estimated total transmission volume is greater than the risk threshold parameter, it can be predicted that the target module may fail in the future.

[0039] Among them, the first output value and the second output value are generated respectively according to the comparison result, which may specifically include: when the first data transmission volume is greater than the first excitation threshold parameter, according to the amount by which the first data transmission volume exceeds the first excitation threshold parameter, the calculation method defined by the excitation function is performed to generate the first output value; when the second data transmission volume is greater than the second excitation threshold parameter, according to the amount by which the second data transmission volume exceeds the second excitation threshold parameter, the calculation method defined by the excitation function is performed to generate the second output value.

[0040] That is to say, the excitation function can define a specific calculation method, so that when the first data transmission amount is greater than the first excitation threshold parameter or the second data transmission amount is greater than the second excitation threshold parameter, the first output value or the second output value is calculated according to the defined calculation method. The embodiment of the present application does not limit the calculation method defined by the excitation function, and only needs to meet the above-mentioned excitation structure characteristics.

[0041] At the same time, the first excitation threshold parameter, the second excitation threshold parameter and the risk threshold parameter in the embodiment of the present application are determined by parameter fitting using the sample data of the data transmission volume and fault information of each module of the data transmission system in different time periods. For example, the data transmission volume of each module of the system in multiple different time periods and the corresponding information on whether the module will fail in the future can be obtained, and sample data can be constructed based on this, and the above parameters can be fitted using the sample data. In this way, the above parameters determined by fitting are used in the model to achieve the purpose of measuring the load pressure of the target module in the actual scenario according to the amount by which the data transmission volume exceeds the corresponding excitation threshold parameter, and based on the calculated comparison result of the estimated total transmission volume and the risk threshold parameter, predicting whether the target module may fail in the future is achieved.

[0042] The technical solution of the embodiment of the present application designs an excitation function and fits the parameters involved in the model to measure the load pressure of the target module according to the amount by which the data transmission volume exceeds the corresponding excitation threshold parameter, obtains the estimated total transmission volume by calculation, and implements fault prediction based on the comparison result between the estimated total transmission volume and the risk threshold parameter. Therefore, the present application can predict whether each module in the data transmission system will fail in the future as a whole, and has good adaptability and effectiveness.

[0043] Figure 3 This is a flowchart for predicting whether a target module has a fault in the fault prediction method provided in the embodiment of the present application. Figure 3 As shown, the process includes:

[0044] S301. Utilize a pre-built risk transmission model to calculate the estimated total transmission volume of the target module at the target time according to the first output value and the second output value.

[0045] S302: If the estimated total transmission amount is greater than the risk threshold parameter, the risk estimation value is determined to be 1, and a target module failure is predicted.

[0046] Specifically, in order to improve the sensitivity and versatility of the model, this application combines the statistical network model and the deep learning neural network algorithm mechanism to construct a bionic risk conduction model similar to the neuron triggering mechanism.

[0047] First, for the n modules in the data transmission system, the data transmission between the modules can be displayed in the form of a matrix, which is specifically expressed as follows:

[0048]

[0049] Among them, x ij,t represents the amount of data transmitted from module i to module j at time t. ii,t ,i=1,2,…,n}, it means that module i transmits data to itself, which can be uniformly set to x ii,t =0, i=1,2,…,n. In addition, any two modules can have unidirectional data transmission or bidirectional data transmission. If there is no data transmission between two modules, then x is also 0.

[0050] The constructed risk transmission model is expressed as:

[0051]

[0052] Among them, i represents any target module, n represents the total number of modules, t represents any target time, and x ik,t represents the first data transmission volume of any target module i outputting data to other modules k, x ji,trepresents the second data transmission volume of other module j inputting data to target module i, k and j both represent other modules, γ i , τ k and θ i They represent the risk threshold parameter, the first excitation threshold parameter and the second excitation threshold parameter respectively.

[0053] y it represents the risk estimate, that is, whether module i fails. it is 0, indicating no fault occurs. it is 1, indicating a fault has occurred. sgn(x,γ) represents the sign function. When x>γ, sgn(x,γ)=1, otherwise it is 0. f(x ji,t ,θ i ) and f(x ik,t ,τ k ) represents the activation function, when x ji,t Greater than θ i , x ik,t Greater than τ k , the excitation function is excited and generates a first output value and a second output value. α ij , β ik , i1 , i2 It is determined by parameter fitting using the sample data of data transmission volume and fault information of each module of the data transmission system in different time periods.

[0054] The structure of the excitation function is designed based on the neuron structure of the neural network model, and this application does not make fixed requirements for the excitation function. ik,t ,τ k ) is also true. k and θ i The threshold parameters are determined by model fitting. The principle of excitation function design is that each module i has a load limit for data transmission. Whether it is data input from the outside or data output to the outside, if it exceeds a certain amount, it will bring load pressure to the module. The more it exceeds, the greater the load pressure, and the more likely it is to cause the module i to fail.

[0055] In one implementation, it can be expressed as:

[0056]

[0057] In another embodiment, it can also be expressed as:

[0058]

[0059] in, is an indicative function, when (x ji,t-θ i )>0, the indicator function is 1, otherwise it is 0.

[0060] In addition, for the risk transmission model of this application, it is necessary to further explain that ρ i1 and ρ i2 is the weight parameter, which is also determined by model fitting. The only restriction is: i1 +ρ i2 =1, its existence significance is: and These two logistic transformation modules are weighted combined to improve the adaptability of the model.

[0061] Figure 4 This is a flow chart of parameter fitting in the fault prediction method provided in the embodiment of the present application. Figure 4 As shown, the process includes:

[0062] S401. Input sample data of data transmission volume of each module of the data transmission system in different time periods into the risk transmission model for fitting calculation to obtain a risk fitting estimate.

[0063] S402: Calculate the objective function according to the risk fitting estimate and the fault information of each module in the sample data.

[0064] S403. In the risk transmission model, each parameter α ij , β ik , i1 , i2 , τ k ,θ i , γ i The respective search spaces are searched according to the gradient descent method until the objective function meets the convergence conditions, and α is determined. ij , β ik , i1 , i2 , τ k ,θ i , γ i respective values.

[0065] The sample data can be generally labeled as G = {(x ij,t ,y it )|i=1,…,n;j=1,…,n} t∈T , T represents the time span collection of sample data collection, and each time t corresponds to {(x ij,t ,y it )|i=1,…,n;j=1,…,n}, is a sample data.

[0066] The sample data is input into the risk conduction model for fitting calculation and the risk fitting estimate is obtained. Then, the objective function is calculated based on the risk fitting estimate and the fault information of each module in the sample data.

[0067] Among them, the objective function is set to mean square error and is expressed as:

[0068]

[0069] The parameter when MSE is minimized is taken as the final parameter estimate, that is, the parameter α in the risk transmission model ij , β ik , i1 , i2 , τ k ,θ i , γ i In order to achieve the result of minimizing MSE, the present embodiment of the application sets the parameters α of the risk conduction model. ij , β ik , i1 , i2 , τ k ,θ i , γ i The search space is searched according to the gradient descent method. Due to the large number of parameters in the model, it reaches 4n 2 +2n+1, as long as n is greater than 5, the parameter dimension will exceed 100. Therefore, in the embodiment of the present application, the gradient descent method is modified to a certain extent, and replaced with a local optimal solution that is closer to the global optimal solution to improve the iteration efficiency. That is: in the process of searching according to the gradient descent method, for each parameter of the risk conduction model, the initial parameter value of the last failed fitting is used as the center of the circle, and the distance between the parameter value of the last iteration in the last failed fitting process and the center of the circle is the radius r. According to the direction of the largest gradient descent in the last failed fitting process, a new initial parameter value is selected at a distance of 2r from the center of the circle for the next fitting.

[0070] Specifically, for The space formed by all parameters {α 11 ,…,α nn ,β 11 ,…,β nn ,ρ 11 ,…,ρ n1 ,ρ 12 ,…,ρ n2 ,θ1,…,θ n ,τ1,…,τ n ,γ1,…,γ n}Assign initial values, use the conventional gradient descent rule for calculation, start from the initial parameter values, each time according to the objective function MSE value, the parameters of the parameter group are gradient changed, and the current parameter value is displaced by a fixed step length in the direction of gradient descent. The fixed step length can be preset, for example, set to 0.1. Then, in the direction of gradient descent, each unit displacement in this direction completes a parameter iteration until the local optimal point of MSE is found, which is the point where the gradient is 0, that is, the parameter combination point where MSE reaches the regional minimum. If many parameter iterations (for example, 3000 or 5000 times) are performed and the local optimal point is still not found, then it is considered that the parameter group of the model cannot converge, that is, no parameter combination that makes MSE reach the regional minimum value can be found. At this time, it can be determined that the selection of the initial parameter value is a failure, and the initial parameter value needs to be re-selected and the parameter iteration is performed again. If the initial parameter value selected last time did not converge, it proves that the selected initial value is far from the optimal point, and the points within a certain regional range are not suitable for parameter fitting as initial values. Therefore, in the embodiment of the present application, the initial parameter value of the last failure is the center P of the circle, and the distance between the parameter value when the parameter fitting fails (that is, the last iteration) and the center of the circle is the radius r. Then, when the initial value of the parameter group is reselected this time, the new initial parameter value is selected at a distance of 2r from the center of the circle to start the fitting iteration in the direction of the largest gradient descent in the parameter fitting process of the last failure.

[0071] For example, suppose the model now has only two parameters {α, β} that need to be fitted, the last selected parameter initial value combination is {α=1, β=1}, the maximum number of iterations is set to 5000, and after 5000 gradient descent iterations, the last iteration result, that is, the 5000th iteration, is {α=4, β=5}. At this time, the model still has not converged, indicating that the parameter initial value selection {α=1, β=1} is a failure, and it is necessary to reselect the initial value of {α, β} to start a new iteration. The distance between {α=1, β=1} and {α=4, β=5} is At this time, it means that at least in the area with {α=1, β=1} as the center P and radius r=5, there will be no optimal solution for the model parameters, otherwise the model will converge, and the area can be marked as X. At this time, in order to prevent the model parameters from falling into area X again when iterating again next time, you can choose to reselect new initial values ​​of the model parameters at a position 2r away from the center P, but the points that meet this condition constitute a circle with a distance of 2r from the center P, so how to select the initial value points on this circle becomes the key. In this application, the direction in which the gradient descent is the largest in the last failed parameter iteration is used as the direction in which the optimal solution is most likely to appear. Assuming that the vector direction with the largest gradient descent in the last failed iteration is d={0.6,0.8}, then the selection point of the new initial value of the parameter should be expressed as:

[0072] P+d*2r={1,1}+{0.6*2r,0.8*2r}={1+0.6*10,1+0.8*10}={7,9}

[0073] Therefore, {α=7, β=9} becomes the initial value of the next iteration, which is much more efficient than randomly selecting initial values ​​for a new iteration.

[0074] Figure 5 This is a flow chart for determining whether the risk transmission model fails in the fault prediction method provided in the embodiment of the present application. Figure 5 As shown, the process includes:

[0075] S501. Calculate the prediction accuracy of the risk transmission model based on the proportion of the data transmission volume of each module in the data transmission system in the real scenario and the difference between the risk estimate value determined by the risk transmission model based on the data in the real scenario and the actual risk value in the real scenario.

[0076] S502. Determine whether the risk transmission model is invalid based on the prediction accuracy.

[0077] On the basis of the above embodiment, after the model is fitted, in order to truly achieve long-term optimization of the model usage effect, it is necessary to dynamically adjust the model in real time based on the feedback of actual usage results.

[0078] It is the result of model fitting parameters. In this embodiment, the fitted parameters are used for risk analysis of new data in real scenarios, and are corrected through trial feedback of real scenario data. The difference between the risk estimate value determined by the risk transmission model based on the data in the real scenario and the real risk value in the real scenario is compared, and the prediction accuracy of the risk transmission model is calculated in combination with the proportion of data transmission volume of each module in the data transmission system in the real scenario. The prediction accuracy is used to determine whether the risk transmission model is invalid.

[0079] Specifically, assume that the data of the real scene is marked as T * G is the time span set of all real scene data. * Input the risk transmission model for correction and determine whether the model has failed. Before determining failure, a quantitative system for measuring the degree of failure needs to be built. It can be calculated That is because Estimated Next, compare the true value With estimated value The difference is used to judge the accuracy of the prediction effect. The lower the accuracy, the higher the failure rate of the model. However, since the importance of each module is inconsistent, some modules transmit a large amount of data, while others transmit a small amount of data. Therefore, it is necessary to assign different weights to each module and use the weights as a basis for measuring the accuracy of the model. Therefore, the following accuracy calculation formula is constructed:

[0080]

[0081] in is an indicative function, when hour, Otherwise it is 0. is the total amount of data transmitted to and from module i, It indicates the proportion of data transmission volume of module i in the whole system.

[0082] During implementation, the threshold can be configured as needed. For example, when R is less than 0.8, the failure level of the model is considered to be intolerable. At this time, the model needs to be completely overturned and the data needs to be re-entered into the model for parameter fitting to obtain a new model with higher stability and accuracy.

[0083] In one embodiment, when R ≥ 0.8, it means that the accuracy of the model is acceptable, indicating that the model has not completely failed. At this time, the model can be fine-tuned to improve its accuracy. Therefore, when the prediction accuracy meets the preset conditions, the embodiment of the present application can adjust the parameter γ in the risk transmission model according to the data in the real scene. i Fine-tune, that is, Fine-tune. This parameter directly affects The value in the function, that is, whether the value is 1 or 0. The fine-tuning method used in this embodiment is also based on the best prediction accuracy, that is, by constructing the objective function By changing the parameters of the model fitting The value of L is minimized, and the threshold parameter at this time is And use it to replace the original parameter value This completes the fine-tuning update of the model.

[0084] After the model is fitted and corrected, it can be used for fault prediction of any data transmission system to avoid the risk of system crash due to excessive data transmission. Taking the customer data storage and scheduling system of a financial institution as an example, it has been mentioned above that it generally includes multiple components and modules such as data storage area, data cache area, calculation and analysis module, data transmission module, loading and unloading module, front-end display and editing module and security verification module. These modules are transmitting data to each other all the time during normal operation. Through the technical solution of this application, it is possible to predict whether each module in the system will fail in the future, that is, by estimating the value It is judged by whether it is 1. When the number of modules judged to be faulty in the system exceeds a certain proportion, or several interrelated modules are predicted to be faulty at the same time, an early warning can be issued to relevant personnel, and corresponding flow limiting measures can be taken for the current data transmission. Distributed computing or peak-shifting processing can be used to ensure the normal progress of data transmission, realize rapid processing of data analysis, and avoid data security accidents such as system memory overflow and downtime.

[0085] In summary, the technical solution of this application has the following beneficial effects:

[0086] (1) Combining the technical characteristics of neural networks and traditional statistical network models, a simulated combined network model that forms a risk module trigger mechanism is constructed to achieve dynamic risk management scheduling of each module system;

[0087] (2) Design a new risk transmission mechanism for the combined network model and a new model fitting method to quickly and accurately determine the final structure of the model, which greatly improves the adaptability and effectiveness of the model;

[0088] (3) Design a new model feedback and correction mechanism to provide intelligent correction and feedback to the model after it is put into use, replacing the drawback of traditional models that can only rely on manual operation to achieve system updates.

[0089] Figure 6 This is a schematic diagram of the structure of a fault prediction device for a data transmission system provided in an embodiment of the present application. Figure 6 As shown, the fault prediction device 60 includes:

[0090] The data transmission volume acquisition module 610 is used to acquire, at any target time, a first data transmission volume of data output from any target module to other modules, and a second data transmission volume of data input from other modules to the target module;

[0091] an excitation function processing module 620, for comparing the first data transmission amount with the corresponding first excitation threshold parameter, and comparing the second data transmission amount with the corresponding second excitation threshold parameter, respectively, using a preset excitation function, and generating a first output value and a second output value respectively according to the comparison results, wherein the excitation function is used to measure the load pressure of the target module according to the amount by which the data transmission amount exceeds the corresponding excitation threshold parameter;

[0092] A prediction module 630, configured to calculate an estimated total transmission amount of the target module at the target time according to the first output value and the second output value, compare the estimated total transmission amount with a risk threshold parameter, and predict whether the target module fails according to the comparison result;

[0093] The first excitation threshold parameter, the second excitation threshold parameter and the risk threshold parameter are determined by parameter fitting using sample data of data transmission volume and fault information of each module of the data transmission system in different time periods.

[0094] In some embodiments, the prediction module 630 includes:

[0095] an estimated transmission total amount calculation unit, configured to calculate the estimated transmission total amount of the target module at the target time according to the first output value and the second output value by using a pre-built risk transmission model;

[0096] a prediction unit, configured to determine a risk estimate value of 1 if the estimated total transmission amount is greater than the risk threshold parameter, and predict that the target module has failed;

[0097] The risk transmission model is expressed as:

[0098]

[0099] i represents any target module, n represents the total number of modules, t represents any target time, x ik,t represents the first data transmission volume, x ji,t represents the second data transmission amount, k and j represent other modules, γ i , τ k and θ i represent the risk threshold parameter, the first excitation threshold parameter and the second excitation threshold parameter respectively, it represents the risk estimate; sgn(x,γ) represents the sign function, when x>γ, sgn(x,γ)=1, otherwise it is 0; f(x ji,t ,θ i ) and f(x ik,t ,τ k ) represents the activation function, when xji,t Greater than θ i , x ik,t Greater than τ k , the excitation function is excited and generates the first output value and the second output value; α ij , β ik , i1 , i2 The data transmission volume and fault information sample data of each module of the data transmission system in different time periods are used to determine the data through parameter fitting.

[0100] In some embodiments, the excitation function processing module 620 includes:

[0101] A first output value generating unit is configured to generate the first output value by performing calculation according to the calculation method defined by the excitation function according to the amount by which the first data transmission amount exceeds the first excitation threshold parameter when the first data transmission amount is greater than the first excitation threshold parameter;

[0102] A second output value generating unit is used to generate the second output value by performing calculation according to the amount by which the second data transmission amount exceeds the second excitation threshold parameter and in accordance with the calculation method defined by the excitation function when the second data transmission amount is greater than the second excitation threshold parameter.

[0103] In some embodiments, the apparatus further comprises a parameter fitting module, wherein the parameter fitting module comprises:

[0104] A risk fitting estimate value calculation unit, used to input sample data of the data transmission volume of each module of the data transmission system in different time periods into the risk conduction model for fitting calculation to obtain a risk fitting estimate value;

[0105] An objective function calculation unit, used to calculate an objective function according to the risk fitting estimate and the fault information of each module in the sample data;

[0106] The parameter fitting unit is used to fit each parameter α in the risk transmission model. ij , β ik , i1 , i2 , τ k ,θ i , γ i The respective search spaces are searched according to the gradient descent method until the objective function meets the convergence condition, and α is determined. ij , β ik , i1 , i2 , τ k ,θ i , γ irespective values.

[0107] In some embodiments, the parameter fitting module further includes a parameter initial value selection unit, which is specifically used to:

[0108] In the process of searching according to the gradient descent method, for each parameter of the risk conduction model, the initial parameter value of the last failed fitting is taken as the center of the circle, the distance between the parameter value of the last iteration in the last failed fitting process and the center of the circle is taken as the radius r, and a new initial parameter value is selected at a position 2r away from the center of the circle for the next fitting in the direction of the largest gradient descent in the last failed fitting process.

[0109] In some embodiments, the device further includes a failure judgment module, and the failure judgment module includes:

[0110] A prediction accuracy calculation unit, used to calculate the prediction accuracy of the risk transmission model according to the proportion of the data transmission volume of each module in the data transmission system in the real scenario and the difference between the risk estimation value determined by the risk transmission model according to the data in the real scenario and the real risk value in the real scenario;

[0111] A failure judgment unit is used to judge whether the risk transmission model fails according to the prediction accuracy.

[0112] In some embodiments, the device further includes a fine-tuning module, and the fine-tuning module is specifically configured to:

[0113] When the prediction accuracy meets the preset conditions, the parameter γ in the risk conduction model is adjusted according to the data in the real scenario. i Make fine adjustments.

[0114] The fault prediction device of the data transmission system provided in the embodiment of the present application can be used to execute the technical solution of the fault prediction method of the data transmission system in the above embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0115] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the excitation function processing module 620 can be a separately established processing element, or it can be integrated in a chip of the above device for implementation. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a processing element of the above device. The function of the above excitation function processing module 620. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0116] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 7 As shown, the electronic device may include: a transceiver 121 , a processor 122 , and a memory 123 .

[0117] The processor 122 executes the computer execution instructions stored in the memory, so that the processor 122 executes the scheme in the above embodiment. The processor 122 can be a general-purpose processor, including a central processing unit CPU, a network processor (network processor, NP), etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0118] The memory 123 is connected to the processor 122 via a system bus and completes communication between them. The memory 123 is used to store computer program instructions.

[0119] The transceiver 121 may be used to obtain tasks to be executed and configuration information of the tasks to be executed.

[0120] The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The transceiver is used to realize the communication between the database access device and other computers (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory.

[0121] The electronic device provided in the embodiment of the present application may be the terminal device of the above embodiment.

[0122] An embodiment of the present application further provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes the technical solution of the fault prediction method of the data transmission system of the above embodiment.

[0123] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, it can implement the technical solution of the fault prediction method of the data transmission system in the above embodiment.

[0124] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present application, and the programming language includes an object-oriented programming language, such as Java, Smalltalk, C++, and also includes a conventional procedural programming language, such as "C" language or similar programming language. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0125] Note that the above are only preferred embodiments of the present application and the technical principles used. Those skilled in the art will understand that the present application is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A method for predicting a fault in a data transmission system, wherein the data transmission system comprises a plurality of modules, characterized in that: The method comprises: Acquire a first data transmission volume of data output from any target module to other modules, and a second data transmission volume of data input from other modules to the target module at any target time; Using a preset excitation function, respectively comparing the first data transmission amount with a corresponding first excitation threshold parameter, and comparing the second data transmission amount with a corresponding second excitation threshold parameter, and respectively generating a first output value and a second output value according to the comparison results, wherein the excitation function is used to measure the load pressure of the target module according to the amount by which the data transmission amount exceeds the corresponding excitation threshold parameter; Calculating the estimated total transmission amount of the target module at the target time according to the first output value and the second output value, and comparing the estimated total transmission amount with a risk threshold parameter, and predicting whether the target module fails according to the comparison result; The first excitation threshold parameter, the second excitation threshold parameter and the risk threshold parameter are determined by parameter fitting using sample data of data transmission volume and fault information of each module of the data transmission system in different time periods.

2. The method according to claim 1, characterized in that: The step of calculating the estimated total transmission amount of the target module at the target time according to the first output value and the second output value, comparing the estimated total transmission amount with a risk threshold parameter, and predicting whether the target module fails according to the comparison result includes: Using a pre-built risk transmission model, the estimated total transmission volume of the target module at the target time is calculated according to the first output value and the second output value; If the estimated total transmission amount is greater than the risk threshold parameter, determining the risk estimate value to be 1, and predicting that the target module fails; The risk transmission model is expressed as: i represents any target module, n represents the total number of modules, t represents any target time, x ik,t represents the first data transmission volume, x ji,t represents the second data transmission amount, k and j represent other modules, γ i , τ k and θ i represent the risk threshold parameter, the first excitation threshold parameter and the second excitation threshold parameter respectively, and y it represents the risk estimate; sgn(x,γ) represents the sign function, when x>γ, sgn(x,γ)=1, otherwise it is 0; f(x ji,t ,θ i ) and f(x ik,t ,τ k ) represents the activation function, when x ji,t Greater than θ i , x ik,t Greater than τ k , the excitation function is excited and generates the first output value and the second output value; α ij , β ik , i1 , i2 The data transmission volume and fault information sample data of each module of the data transmission system in different time periods are used to determine the data through parameter fitting.

3. The method according to claim 1, characterized in that The step of generating a first output value and a second output value according to the comparison result comprises: When the first data transmission amount is greater than the first excitation threshold parameter, the first output value is generated by performing calculation according to the calculation method defined by the excitation function according to the amount by which the first data transmission amount exceeds the first excitation threshold parameter; When the second data transmission amount is greater than the second excitation threshold parameter, the second output value is generated by performing calculation according to the amount by which the second data transmission amount exceeds the second excitation threshold parameter in accordance with the calculation method defined by the excitation function.

4. The method according to claim 2, characterized in that: The parameter fitting process includes: Inputting sample data of the data transmission volume of each module of the data transmission system in different time periods into the risk transmission model for fitting calculation to obtain a risk fitting estimate; Calculating an objective function according to the risk fitting estimate and the fault information of each module in the sample data; In the risk transmission model, each parameter α ij , β ik , i1 , i2 , τ k ,θ i , γ i The respective search spaces are searched according to the gradient descent method until the objective function meets the convergence condition, and α is determined. ij , β ik , i1 , i2 , τ k ,θ i , γ i respective values.

5. The method according to claim 4, characterized in that Also includes: In the process of searching according to the gradient descent method, for each parameter of the risk conduction model, the initial parameter value of the last failed fitting is taken as the center of the circle, the distance between the parameter value of the last iteration in the last failed fitting process and the center of the circle is taken as the radius r, and a new initial parameter value is selected at a position 2r away from the center of the circle for the next fitting in the direction of the largest gradient descent in the last failed fitting process.

6. The method according to claim 2, characterized in that Also includes: Calculate the prediction accuracy of the risk transmission model according to the proportion of the data transmission volume of each module in the data transmission system in the real scenario and the difference between the risk estimation value determined by the risk transmission model according to the data in the real scenario and the real risk value in the real scenario; Whether the risk transmission model fails is determined based on the prediction accuracy.

7. The method according to claim 6, characterized in that Also includes: When the prediction accuracy meets the preset conditions, the parameter γ in the risk conduction model is adjusted according to the data in the real scenario. i Make fine adjustments.

8. A fault prediction device for a data transmission system, the data transmission system comprising a plurality of modules, characterized in that: The device comprises: A data transmission volume acquisition module, used to acquire, at any target moment, a first data transmission volume of data output from any target module to other modules, and a second data transmission volume of data input from other modules to the target module; an excitation function processing module, used to compare the first data transmission amount with the corresponding first excitation threshold parameter, and compare the second data transmission amount with the corresponding second excitation threshold parameter using a preset excitation function, and to generate a first output value and a second output value according to the comparison results, respectively, wherein the excitation function is used to measure the load pressure of the target module according to the amount by which the data transmission amount exceeds the corresponding excitation threshold parameter; a prediction module, configured to calculate an estimated total transmission amount of the target module at the target time according to the first output value and the second output value, compare the estimated total transmission amount with a risk threshold parameter, and predict whether the target module fails according to the comparison result; The first excitation threshold parameter, the second excitation threshold parameter and the risk threshold parameter are determined by parameter fitting using sample data of data transmission volume and fault information of each module of the data transmission system in different time periods.

9. The device according to claim 8, characterized in that The prediction module comprises: an estimated transmission total amount calculation unit, configured to calculate the estimated transmission total amount of the target module at the target time according to the first output value and the second output value by using a pre-built risk transmission model; a prediction unit, configured to determine a risk estimate value of 1 if the estimated total transmission amount is greater than the risk threshold parameter, and predict that the target module has failed; The risk transmission model is expressed as: i represents any target module, n represents the total number of modules, t represents any target time, x ik,t represents the first data transmission volume, x ji,t represents the second data transmission amount, k and j represent other modules, γ i , τ k and θ i represent the risk threshold parameter, the first excitation threshold parameter and the second excitation threshold parameter respectively, and y it represents the risk estimate; sgn(x,γ) represents the sign function, when x>γ, sgn(x,γ)=1, otherwise it is 0; f(x ji,t ,θ i ) and f(x ik,t ,τ k ) represents the activation function, when x ji,t Greater than θ i , x ik,t Greater than τ k , the excitation function is excited and generates the first output value and the second output value; α ij , β ik , i1 , i2 The data transmission volume and fault information sample data of each module of the data transmission system in different time periods are used to determine the data through parameter fitting.

10. The device according to claim 9, characterized in that It also includes a parameter fitting module, which includes: A risk fitting estimate value calculation unit, used to input sample data of the data transmission volume of each module of the data transmission system in different time periods into the risk conduction model for fitting calculation to obtain a risk fitting estimate value; An objective function calculation unit, used to calculate an objective function according to the risk fitting estimate and the fault information of each module in the sample data; The parameter fitting unit is used to fit each parameter α in the risk transmission model. ij , β ik , i1 , i2 , τ k ,θ i , γ i The respective search spaces are searched according to the gradient descent method until the objective function meets the convergence condition, and α is determined. ij , β ik , i1 , i2 , τ k ,θ i , γ i respective values.

11. The device according to claim 10, characterized in that The parameter fitting module also includes a parameter initial value selection unit, which is specifically used to: In the process of the parameter fitting unit searching according to the gradient descent method, for each parameter of the risk conduction model, the initial parameter value of the last failed fitting is taken as the center of the circle, and the distance between the parameter value of the last iteration in the last failed fitting process and the center of the circle is taken as the radius r. According to the direction of the largest gradient descent in the last failed fitting process, a new initial parameter value is selected at a position 2r away from the center of the circle for the next fitting.

12. The device according to claim 9, characterized in that It also includes a failure judgment module, which includes: A prediction accuracy calculation unit, used to calculate the prediction accuracy of the risk transmission model according to the proportion of the data transmission volume of each module in the data transmission system in the real scenario and the difference between the risk estimation value determined by the risk transmission model according to the data in the real scenario and the real risk value in the real scenario; A failure judgment unit is used to judge whether the risk transmission model fails according to the prediction accuracy.

13. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the fault prediction method for the data transmission system according to any one of claims 1 to 7.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the fault prediction method for a data transmission system according to any one of claims 1 to 7.

15. A computer program product, characterized in that The invention comprises a computer program, which, when executed by a processor, implements the fault prediction method for a data transmission system according to any one of claims 1 to 7.

Images