Method, apparatus and medium for failure prediction of data transmission system

Abstract

The application provides a fault prediction method, device and equipment of a data transmission system and a medium, relates to the field of data analysis, and particularly relates to the fields of big data analysis and artificial intelligence. The method comprises the following steps: acquiring, at any target moment, a first data transmission amount of data output by any target module to other modules and a second data transmission amount of data input by other modules to the target module; comparing the first data transmission amount with a corresponding first excitation threshold parameter and the second data transmission amount with a corresponding second excitation threshold parameter by using a preset excitation function, and generating a first output value and a second output value according to the comparison results; calculating an estimated total transmission amount of the target module at the target moment according to the first output value and the second output value, comparing the estimated total transmission amount with a risk threshold parameter, and predicting whether the target module will malfunction according to the comparison result. The application can effectively predict whether the system will malfunction in the future.

Description

Technical Field

[0001] This application relates to the field of data analysis technology, particularly to the fields of big data analysis and artificial intelligence, and specifically to a method, apparatus, device, and medium for fault prediction of a data transmission system. Background Technology

[0002] Any system requires the combination and coordinated operation of various modules to function properly. The network formed by these modules constitutes the basic layout of the system. Each module in the network carries the risk of failure, which in turn poses a security risk to the normal operation of the entire network system.

[0003] In the financial sector, data storage and scheduling systems, such as financial data storage systems, are examples of such data transmission systems. They consist of multiple components or modules that are interconnected and frequently exchange data, forming a data transmission network. Therefore, the ability to comprehensively predict and assess the systemic risks present in these data transmission systems is crucial in the financial field. Summary of the Invention

[0004] This application provides a method, apparatus, device, and medium for predicting faults in a data transmission system, in order to solve the problem that the prior art cannot comprehensively assess and predict faults in a data transmission system.

[0005] In a first aspect, this application provides a fault prediction method for a data transmission system, the data transmission system comprising multiple modules, the method comprising:

[0006] At any given target time, obtain the first data transmission volume of any target module outputting data to other modules, and the second data transmission volume of other modules inputting data to the target module;

[0007] Using a preset activation function, the first data transmission amount is compared with the corresponding first activation threshold parameter, and the second data transmission amount is compared with the corresponding second activation threshold parameter. Based on the comparison results, a first output value and a second output value are generated respectively. The activation function is used to measure the load pressure of the target module based on the amount by which the data transmission amount exceeds the corresponding activation threshold parameter.

[0008] The estimated total transmission volume of the target module at the target time is calculated based on the first output value and the second output value, and the estimated total transmission volume is compared with the risk threshold parameter. Based on the comparison result, it is predicted whether the target module will fail.

[0009] The first excitation threshold parameter, the second excitation threshold parameter, and the risk threshold parameter are determined by parameter fitting using sample data of data transmission volume and fault information of each module of the data transmission system at different time periods.

[0010] Secondly, this application provides a fault prediction device for a data transmission system, the data transmission system comprising multiple modules, the device comprising:

[0011] The data transmission volume acquisition module is used to acquire, at any target time, a first data transmission volume of data output by any target module to other modules, and a second data transmission volume of data input by other modules to the target module;

[0012] The activation function processing module is used to compare the first data transmission amount with the corresponding first activation threshold parameter and the second data transmission amount with the corresponding second activation threshold parameter using a preset activation function, and generate a first output value and a second output value according to the comparison results. The activation function is used to measure the load pressure of the target module based on the amount by which the data transmission amount exceeds the corresponding activation threshold parameter.

[0013] The prediction module is used to calculate the estimated total transmission volume of the target module at the target time based on the first output value and the second output value, compare the estimated total transmission volume with the risk threshold parameter, and predict whether the target module will fail based on the comparison result.

[0014] The first excitation threshold parameter, the second excitation threshold parameter, and the risk threshold parameter are determined by parameter fitting using sample data of data transmission volume and fault information of each module of the data transmission system at different time periods.

[0015] Thirdly, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a fault prediction method for a data transmission system as described in any of the embodiments of this application.

[0016] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a fault prediction method for a data transmission system as described in any of the embodiments of this application.

[0017] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements a fault prediction method for a data transmission system as described in any of the embodiments of this application.

[0018] The fault prediction method, apparatus, device, and medium for a data transmission system provided in this application are based on a first data transmission volume output by any target module to other modules in the data transmission system, and a second data transmission volume input by other modules to the target module. Using a preset excitation function, these two volumes are compared with a first excitation threshold parameter and a second excitation threshold parameter, respectively, and a first output value and a second output value are generated based on the comparison results. Then, the estimated total transmission volume of the target module is calculated based on the first and second output values, and finally compared with a risk threshold parameter. The comparison result is used to predict whether the target module will fail. Furthermore, the first excitation threshold parameter, the second excitation threshold parameter, and the risk threshold parameter are determined by parameter fitting using sample data of data transmission volume and fault information of each module in the data transmission system at different time periods. It can be seen that this application, by designing an excitation function and fitting the relevant parameters, measures the load pressure of the target module based on the amount by which the data transmission volume exceeds the corresponding excitation threshold parameter, calculates the estimated total transmission volume, and achieves fault prediction based on the comparison result of the estimated total transmission volume and the risk threshold parameter. Therefore, this application can predict whether each module in the data transmission system will fail in the future from an overall perspective, and has good adaptability and effectiveness. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A flowchart illustrating a fault prediction method for a data transmission system provided in this application embodiment;

[0021] Figure 2 A schematic diagram illustrating inter-module data transmission provided in an embodiment of this application;

[0022] Figure 3 A flowchart illustrating the method for predicting whether a target module has failed, provided in an embodiment of this application.

[0023] Figure 4 A flowchart illustrating parameter fitting in the fault prediction method provided in this application embodiment;

[0024] Figure 5 A flowchart for determining whether a risk transmission model has failed in the fault prediction method provided in this application embodiment;

[0025] Figure 6A schematic diagram of the structure of a fault prediction device for a data transmission system provided in an embodiment of this application;

[0026] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0027] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0028] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present application are shown in the drawings, not the entire structure.

[0029] It should be noted that similar reference numerals and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, terms such as "first" and "second" are used only to distinguish descriptions and should not be construed as indicating or implying relative importance. The acquisition, storage, use, and processing of data in the technical solution of this application all comply with relevant national laws and regulations. It should be noted that in the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solution of this application, and do not imply that the applicant has already used or necessarily used the relevant content of that solution.

[0030] Figure 1 This is a flowchart illustrating a fault prediction method for a data transmission system provided in an embodiment of this application. This embodiment is applicable to predicting whether individual modules and the system as a whole will fail in the future. This method can be executed by a fault prediction device for the data transmission system. This device can be implemented in software and / or hardware, preferably configured in an electronic device, such as a computer or server. The data transmission system includes multiple modules. For example... Figure 1 As shown, the method specifically includes:

[0031] S101. Obtain the first data transmission amount of any target module outputting data to other modules at any target time, and the second data transmission amount of other modules inputting data to the target module.

[0032] S102. Using a preset excitation function, the first data transmission amount is compared with the corresponding first excitation threshold parameter, and the second data transmission amount is compared with the corresponding second excitation threshold parameter. Based on the comparison results, the first output value and the second output value are generated respectively.

[0033] The excitation function is used to measure the load pressure of the target module based on the amount by which the data transmission volume exceeds the corresponding excitation threshold parameter.

[0034] S103. Calculate the estimated total transmission volume of the target module at the target time based on the first output value and the second output value, compare the estimated total transmission volume with the risk threshold parameter, and predict whether the target module will fail based on the comparison result.

[0035] Among them, the first excitation threshold parameter, the second excitation threshold parameter, and the risk threshold parameter are determined by parameter fitting using sample data of data transmission volume and fault information of each module of the data transmission system at different time periods.

[0036] Specifically, a data transmission system is composed of multiple modules, each responsible for a specific function. The combination of these modules makes the system a complete tool capable of achieving specific goals. Taking a customer data storage and scheduling system commonly used in financial enterprises as an example, it typically includes multiple modules such as a data storage area, a data cache area, a calculation and analysis module, a data transmission module, a data loading and unloading module, a front-end display and editing module, and a security verification module. These modules can be labeled as M1, M2, M3, M4, ..., M... n , where n is the total number of modules. Figure 2 This is a schematic diagram illustrating inter-module data transmission provided in an embodiment of this application. As shown in the figure, these modules are connected to each other via connecting lines, which indicate data transmission between the modules. Together, they form a data transmission network to maintain the normal operation of the enterprise.

[0037] This application is based on the first data transmission volume of each module outputting data to other modules and the second data transmission volume of other modules inputting data to the module, and uses the constructed model to predict whether the target module will fail in the future, thereby realizing failure prediction and risk management of the entire system.

[0038] In the constructed model, the load pressure of the target module is measured by the amount by which the data transmission volume exceeds the corresponding trigger threshold parameter through an activation function. Specifically, the first data transmission volume is compared with the corresponding first trigger threshold parameter, and the second data transmission volume is compared with the corresponding second trigger threshold parameter. Based on the comparison results, a first output value and a second output value are generated respectively. The first and second output values ​​can then be used to further calculate the estimated total transmission volume of the target module at the target time. If the estimated total transmission volume is greater than the risk threshold parameter, it can be predicted that the target module may fail in the future.

[0039] Specifically, generating a first output value and a second output value based on the comparison result can include: when the first data transmission amount is greater than the first excitation threshold parameter, calculating according to the calculation method defined by the excitation function based on the amount by which the first data transmission amount exceeds the first excitation threshold parameter, and generating a first output value; when the second data transmission amount is greater than the second excitation threshold parameter, calculating according to the calculation method defined by the excitation function based on the amount by which the second data transmission amount exceeds the second excitation threshold parameter, and generating a second output value.

[0040] In other words, the excitation function can be defined with a specific calculation method to calculate the first output value or the second output value according to the defined calculation method when the first data transmission amount is greater than the first excitation threshold parameter or the second data transmission amount is greater than the second excitation threshold parameter. This application does not limit the calculation method defined for the excitation function; it only needs to satisfy the above-mentioned excitation structure characteristics.

[0041] Meanwhile, in this embodiment, the first activation threshold parameter, the second activation threshold parameter, and the risk threshold parameter are determined by parameter fitting using sample data of data transmission volume and fault information of each module of the data transmission system at different time periods. For example, the data transmission volume of each module in the system at multiple different time periods and the corresponding information on whether the module will fail in the future can be obtained, and sample data can be constructed accordingly. The above parameters are then fitted using the sample data. In this way, the above parameters determined by fitting are used in the model to achieve the goal of measuring the load pressure of the target module in a real-world scenario based on the amount by which the data transmission volume exceeds the corresponding activation threshold parameter. Based on the comparison result of the calculated estimated total transmission volume and the risk threshold parameter, the goal of predicting whether the target module may fail in the future can be achieved.

[0042] The technical solution of this application, through designing an excitation function and fitting and determining the parameters involved in the model, measures the load pressure of the target module based on the amount by which the data transmission volume exceeds the corresponding excitation threshold parameter. An estimated total transmission volume is calculated, and fault prediction is achieved based on the comparison between the estimated total transmission volume and the risk threshold parameter. Therefore, this application can predict whether each module in the data transmission system will fail in the future, and has good adaptability and effectiveness.

[0043] Figure 3 This is a flowchart illustrating the prediction of whether a target module malfunctions in the fault prediction method provided in this application embodiment. For example... Figure 3 As shown, the process includes:

[0044] S301. Using a pre-built risk transmission model, calculate the estimated total transmission amount of the target module at the target time based on the first output value and the second output value.

[0045] S302. If the estimated total transmission volume is greater than the risk threshold parameter, the risk estimate is set to 1, and the target module is predicted to fail.

[0046] Specifically, in order to improve the sensitivity and versatility of the model, this application combines statistical network models and deep learning neural network algorithm mechanisms to construct a biomimetic risk transmission model similar to the triggering mechanism of neurons.

[0047] First, for n modules in a data transmission system, the data transmission between modules can be represented in matrix form, as follows:

[0048]

[0049] Where, x ij,t This represents the amount of data transmitted from module i to module j at time t, for {x} ii,t If i = 1, 2, ..., n, then it represents the case where module i transmits data to itself, which can be uniformly set as x. ii,t =0, i=1,2,…,n. Furthermore, any two modules can have unidirectional or bidirectional data transmission. If there is no data transmission between two modules, then their x is also 0.

[0050] The constructed risk transmission model is represented as follows:

[0051]

[0052] Where i represents any target module, n represents the total number of modules, t represents any target time, and x ik,t x represents the first data transfer amount from any target module i to other module k. ji,tThis represents the second data transfer amount from other module j to target module i, where k and j both represent other modules, γ i τ k and θ i These represent the risk threshold parameter, the first excitation threshold parameter, and the second excitation threshold parameter, respectively.

[0053] y it This represents the risk estimate, i.e., whether module i has failed, when y it A value of 0 indicates that no fault has occurred. it A value of 1 indicates a fault has occurred. sgn(x,γ) represents the sign function; when x > γ, sgn(x,γ) = 1, otherwise it is 0. f(x ji,t ,θ i ) and f(x ik,t ,τ k ) represents the activation function, when x ji,t Greater than θ i x ik,t Greater than τ k The activation function is activated, producing a first output value and a second output value. α ij β ik ρ i1 ρ i2 It is determined by parameter fitting using sample data of data transmission volume and fault information of each module of the data transmission system at different time periods.

[0054] The structure of the activation function is designed to mimic the neuron structure of a neural network model, and this application does not impose fixed requirements on the activation function f(x). ik,t ,τ k The same applies. k and θ i The threshold parameters are all determined through model fitting. The principle of activation function design is that each module i has a capacity limit for data transmission. Whether it is data input from the outside or data output to the outside, exceeding a certain amount will put load pressure on the module. The more it exceeds the limit, the greater the load pressure will be, and the more likely the module i will fail.

[0055] In one implementation, it can be represented as:

[0056]

[0057] In another implementation, it can also be expressed as:

[0058]

[0059] in, It is an indicator function, when (x ji,t-θ i When )>0, the indicator function is 1, otherwise it is 0.

[0060] Furthermore, regarding the risk transmission model in this application, it should be noted that ρ i1 and ρ i2 These are weight parameters, determined through model fitting. The only constraint is: ρ i1 +ρ i2 =1, its existence is significant because: for and These two logic transformation modules are weighted and combined to improve the model's adaptability.

[0061] Figure 4 This is a flowchart illustrating parameter fitting in the fault prediction method provided in this application embodiment. For example... Figure 4 As shown, the process includes:

[0062] S401. Input the sample data of data transmission volume of each module of the data transmission system at different time periods into the risk transmission model for fitting calculation to obtain the risk fitting estimate.

[0063] S402. Calculate the objective function based on the risk fitting estimate and the fault information of each module in the sample data.

[0064] S403, In the risk transmission model, the parameters α ij β ik ρ i1 ρ i2 τ k θ i γ i Each search space is searched using gradient descent until the objective function satisfies the convergence condition, at which point α is determined. ij β ik ρ i1 ρ i2 τ k θ i γ i Their respective values.

[0065] The sample data can be generally denoted as G={(x ij,t ,y it |i=1,…,n;j=1,…,n} t∈T T represents the set of time spans during which the sample data was collected, and each time t corresponds to {(x ij,t ,y it A sample data point is defined as} | i = 1, ..., n; j = 1, ..., n.

[0066] The sample data is input into the risk transmission model for fitting calculation, and the risk fitting estimate is obtained. Then, the objective function is calculated based on the risk fitting estimate and the fault information of each module in the sample data.

[0067] The objective function is set as mean squared error, and is expressed as:

[0068]

[0069] The parameters that minimize MSE are taken as the final parameter estimates, i.e., parameter α in the risk transmission model. ij β ik ρ i1 ρ i2 τ k θ i γ i The fitting results. To achieve the goal of minimizing the MSE, the embodiments of this application adjust the parameters α of the risk transmission model. ij β ik ρ i1 ρ i2 τ k θ i γ i Each model has its own search space, which is searched using gradient descent. Due to the large number of parameters in the model, reaching 4n... 2 +2n+1, as long as n is greater than 5, the parameter dimension will exceed 100. Therefore, in this embodiment, the gradient descent method is modified to a certain extent, replacing it with a local optimum that is closer to the global optimum to improve iteration efficiency. That is, during the search process according to the gradient descent method, for each parameter of the risk transmission model, the initial value of the parameter that failed the previous fitting is taken as the center, and the distance between the parameter value of the last iteration in the previous fitting failure and the center is taken as the radius r. Following the direction of the largest gradient descent amplitude in the previous fitting failure, a new initial parameter value is selected at a position 2r away from the center for the next fitting.

[0070] Specifically, for those caused by The constructed space requires all parameters {α} 11 ,…,α nn ,β 11 ,…,β nn ,ρ 11 ,…,ρ n1 ,ρ 12 ,…,ρ n2 ,θ1,…,θ n ,τ1,…,τ n ,γ1,…,γ nInitial values ​​are assigned, and calculations are performed using the conventional gradient descent method. Starting from the initial parameter values, each parameter in the parameter set is adjusted according to the gradient of the objective function MSE value, and the current parameter value is shifted by a fixed step size in the direction of gradient descent. The fixed step size can be preset, for example, to 0.1. Then, following the direction of gradient descent, each unit shift in this direction completes one parameter iteration until a local optimum of MSE is found. This is the point where the gradient is 0, which is also the parameter combination point where the MSE reaches the minimum value in the region. If a local optimum is not found after many parameter iterations (e.g., 3000 or 5000 times), then the parameter set of the model is considered non-convergent, meaning that no parameter combination that makes the MSE reach the minimum value in the region can be found. In this case, the initial parameter value selection can be considered a failure, and a new initial parameter value needs to be selected, and parameter iteration can be performed again. If the previously selected initial parameter value did not converge, then it proves that the selected initial value is too far from the optimum, and points within a certain region are not suitable as initial values ​​for parameter fitting. Therefore, in this embodiment, the initial parameter value of the previous failed parameter is the center P, and the distance between the parameter value when the parameter fitting failed (i.e., the last iteration) and the center of the circle is the radius r. Then, when the initial parameter value is selected again, it is selected in the direction of the largest gradient descent during the previous failed parameter fitting process, and a new initial parameter value is selected at a position 2r away from the center of the circle to start the fitting iteration.

[0071] For example, suppose the model now only has two parameters {α, β} to fit. The initial parameter combination selected in the last iteration was {α = 1, β = 1}, and the maximum number of iterations was set to 5000. After 5000 gradient descent iterations, the result of the last iteration, the 5000th iteration, is {α = 4, β = 5}. At this point, the model has still not converged, indicating that the initial parameter selection {α = 1, β = 1} was a failure, and a new iteration needs to be started with a new set of initial parameters {α, β}. The distance between {α = 1, β = 1} and {α = 4, β = 5} is... This means that there will be no optimal solution for the model parameters within a region centered at P with radius r = 5, where {α = 1, β = 1}. Otherwise, the model would converge, and this region can be labeled X. To prevent falling into region X again during the next iteration of the model parameters, new initial values ​​for the model parameters can be selected at a position 2r away from the center P. However, the points satisfying this condition form a circle 2r away from the center P. Therefore, selecting the initial value point on this circle becomes crucial. In this application, the direction of the maximum gradient descent in the previous failed parameter iteration is considered the direction where the optimal solution is most likely to occur. Assuming the vector direction of the maximum gradient descent in the previous failed iteration is d = {0.6, 0.8}, then the selection point for the new initial parameter values ​​should be expressed as:

[0072] P+d*2r={1,1}+{0.6*2r,0.8*2r}={1+0.6*10,1+0.8*10}={7,9}

[0073] Therefore, {α=7,β=9} becomes the initial value for the next iteration, which is much more efficient than randomly selecting an initial value for a new iteration.

[0074] Figure 5 This is a flowchart illustrating the process of determining whether a risk transmission model has failed in the fault prediction method provided in this application embodiment. For example... Figure 5 As shown, the process includes:

[0075] S501. Based on the proportion of data transmission volume of each module in the data transmission system in the real scenario, and the difference between the risk estimate determined by the risk transmission model based on the data in the real scenario and the actual risk value in the real scenario, calculate the prediction accuracy of the risk transmission model.

[0076] S502. Determine whether the risk transmission model has failed based on the prediction accuracy.

[0077] Based on the above embodiments, after the model is fitted, to truly achieve long-term optimization of the model's performance, it is necessary to dynamically adjust the model in real time based on feedback from actual usage results. For the risk transmission model described in the above embodiments, it is assumed that...

[0078] This is the result of model fitting parameters. In this embodiment, the fitted parameters are used for risk analysis with new data from real-world scenarios, and then corrected based on feedback from real-world data trials. The difference between the risk estimate determined by the risk transmission model based on data from real-world scenarios and the actual risk value in real-world scenarios is compared. Combined with the proportion of data transmission volume of each module in the data transmission system in real-world scenarios, the prediction accuracy of the risk transmission model is calculated. The prediction accuracy is used to determine whether the risk transmission model has failed.

[0079] Specifically, assuming the data in the real-world scenario is labeled as T * It is the collection of all real-world scenario data over a given time span. G * The input risk transmission model is then modified to determine whether the model has failed. Before determining failure, a quantitative system for measuring the degree of failure needs to be constructed. It can be calculated That is, by Estimated The estimated value. Next, compare it with the actual value. Compared with the estimated value The difference in accuracy is used to determine the accuracy of the prediction; lower accuracy indicates a higher degree of model failure. However, since the importance of each module varies—some modules transmit large amounts of data while others transmit small amounts—it is necessary to assign different weights to each module and use these weights as a basis for measuring model accuracy. Therefore, the following formula for calculating accuracy is constructed:

[0080]

[0081] in It is an indicator function, when hour, Conversely, it is 0. It is the total amount of data input and output by module i. This represents the proportion of data transmission volume of module i in the entire system.

[0082] In implementation, thresholds can be configured as needed. For example, when R is less than 0.8, the failure of the model is considered to be intolerable. In this case, the model needs to be completely scrapped and rebuilt. The data is then re-inputted into the model for parameter fitting to obtain a new model with higher stability and accuracy.

[0083] In one implementation, when R ≥ 0.8, it indicates that the model's accuracy is acceptable, meaning the model has not completely failed. In this case, the model can be fine-tuned to improve its accuracy. Therefore, in this embodiment, when the prediction accuracy meets a preset condition, the parameter γ in the risk transmission model can be adjusted based on data from real-world scenarios. i To make fine adjustments, that is, to Fine-tune this parameter. This parameter directly affects... The value in the function, i.e., whether it is 1 or 0. The fine-tuning method used in this embodiment is also based on the optimal prediction accuracy, that is, by constructing the objective function. By changing the parameters fitted by the model The value of makes L reach its minimum, and the threshold parameter at this point is denoted as . And use it to replace the original parameter value This allows for fine-tuning and updating of the model.

[0084] Once the model is fitted and corrected, it can be used for fault prediction in any data transmission system, avoiding the risk of system crash due to excessive data transmission. Taking a financial institution's customer data storage and scheduling system as an example, as mentioned above, it generally includes multiple components and modules such as a data storage area, a data cache area, a calculation and analysis module, a data transmission module, a data loading and unloading module, a front-end display and editing module, and a security verification module. These modules are constantly transmitting data to each other during normal operation. The technical solution of this application can predict whether each module in the system will fail in the future, i.e., by estimating... The system uses a value of 1 to determine whether a failure occurs. When the number of modules identified as faulty in the system exceeds a certain proportion, or when several interconnected modules are simultaneously predicted to fail, an early warning can be issued to relevant personnel. Corresponding rate-limiting measures can then be implemented for current data transmission. Distributed computing or off-peak processing can be used to ensure the normal operation of data transmission, enabling rapid data analysis and preventing data security incidents such as system memory overflow leading to system crashes.

[0085] In summary, the technical solution of this application has the following beneficial effects:

[0086] (1) Combining the characteristics of neural networks and traditional statistical network models, a simulated combined network model is constructed to form a risk module triggering mechanism, so as to realize the dynamic risk management scheduling of each module system;

[0087] (2) A novel risk transmission mechanism was designed for the combined network model, and a novel model fitting method was designed to quickly and accurately determine the final structure of the model, which greatly improved the adaptability and effectiveness of the model.

[0088] (3) Design a brand-new model feedback correction mechanism to provide intelligent correction feedback after the model is put into use, replacing the shortcomings of traditional models that can only rely on manual operation to update the system.

[0089] Figure 6 This is a schematic diagram of the structure of a fault prediction device for a data transmission system provided in an embodiment of this application. Figure 6 As shown, the fault prediction device 60 includes:

[0090] The data transmission volume acquisition module 610 is used to acquire, at any target time, a first data transmission volume of data output by any target module to other modules, and a second data transmission volume of data input by other modules to the target module;

[0091] The activation function processing module 620 is used to compare the first data transmission amount with the corresponding first activation threshold parameter and the second data transmission amount with the corresponding second activation threshold parameter using a preset activation function, and generate a first output value and a second output value according to the comparison results. The activation function is used to measure the load pressure of the target module based on the amount by which the data transmission amount exceeds the corresponding activation threshold parameter.

[0092] The prediction module 630 is used to calculate the estimated total transmission volume of the target module at the target time based on the first output value and the second output value, compare the estimated total transmission volume with the risk threshold parameter, and predict whether the target module will fail based on the comparison result.

[0093] The first excitation threshold parameter, the second excitation threshold parameter, and the risk threshold parameter are determined by parameter fitting using sample data of data transmission volume and fault information of each module of the data transmission system at different time periods.

[0094] In some embodiments, the prediction module 630 includes:

[0095] The estimated total transmission volume calculation unit is used to calculate the estimated total transmission volume of the target module at the target time based on the first output value and the second output value using a pre-built risk transmission model.

[0096] The prediction unit is used to determine the risk estimate value as 1 and predict that the target module will fail if the estimated total transmission volume is greater than the risk threshold parameter.

[0097] The risk transmission model is expressed as follows:

[0098]

[0099] i represents any target module, n represents the total number of modules, t represents any target time, and x ik,t x represents the first data transmission volume. ji,t This represents the second data transmission volume, where k and j both represent other modules, and γ i τ k and θ i The risk threshold parameter, the first activation threshold parameter, and the second activation threshold parameter are respectively represented by y. it Represents the risk estimate; sgn(x,γ) represents the sign function, which is 1 when x > γ and 0 otherwise; f(x ji,t ,θ i ) and f(x ik,t ,τ k ) represents the activation function, when xji,t Greater than θ i x ik,t Greater than τ k The activation function is activated and produces the first output value and the second output value; α ij β ik ρ i1 ρ i2 It is determined by parameter fitting using sample data of data transmission volume and fault information of each module of the data transmission system at different time periods.

[0100] In some embodiments, the excitation function processing module 620 includes:

[0101] The first output value generation unit is used to generate the first output value by calculating according to the calculation method defined by the excitation function based on the amount by which the first data transmission amount exceeds the first excitation threshold parameter when the first data transmission amount is greater than the first excitation threshold parameter.

[0102] The second output value generation unit is used to generate the second output value by calculating according to the calculation method defined by the excitation function based on the amount by which the second data transmission amount exceeds the second excitation threshold parameter when the second data transmission amount is greater than the second excitation threshold parameter.

[0103] In some embodiments, the apparatus further includes a parameter fitting module, the parameter fitting module comprising:

[0104] The risk fitting estimate calculation unit is used to input sample data of the data transmission volume of each module of the data transmission system at different time periods into the risk transmission model for fitting calculation to obtain the risk fitting estimate.

[0105] The objective function calculation unit is used to calculate the objective function based on the risk fitting estimate and the fault information of each module in the sample data;

[0106] Parameter fitting unit, used for fitting each parameter α in the risk transmission model ij β ik ρ i1 ρ i2 τ k θ i γ i Each search space is searched using gradient descent until the objective function satisfies the convergence condition, at which point α is determined. ij β ik ρ i1 ρ i2 τ k θ i γ iTheir respective values.

[0107] In some embodiments, the parameter fitting module further includes a parameter initial value selection unit, specifically used for:

[0108] During the search process using the gradient descent method, for each parameter of the risk transmission model, the initial value of the parameter that failed the previous fitting is taken as the center, and the distance between the parameter value of the last iteration in the previous fitting failure and the center is taken as the radius r. Following the direction of the largest gradient descent amplitude in the previous fitting failure, a new initial parameter value is selected at a position 2r away from the center for the next fitting.

[0109] In some embodiments, the apparatus further includes a failure determination module, the failure determination module comprising:

[0110] The prediction accuracy calculation unit is used to calculate the prediction accuracy of the risk transmission model based on the proportion of data transmission volume of each module in the data transmission system in the real scenario, and the difference between the risk estimate determined by the risk transmission model based on the data in the real scenario and the actual risk value in the real scenario.

[0111] The failure determination unit is used to determine whether the risk transmission model has failed based on the prediction accuracy.

[0112] In some embodiments, the device further includes a fine-tuning module, which is specifically used for:

[0113] When the prediction accuracy meets the preset conditions, the parameter γ in the risk transmission model is adjusted based on the data from the real scenario. i Make minor adjustments.

[0114] The fault prediction device for the data transmission system provided in this application embodiment can be used to execute the technical solution of the fault prediction method for the data transmission system in the above embodiment. Its implementation principle and technical effect are similar, and will not be described again here.

[0115] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing element calls; they can be fully implemented in hardware; or some modules can be implemented by processing element calls to software, while others are implemented in hardware. For example, the excitation function processing module 620 can be a separate processing element, or it can be integrated into a chip in the above device. Alternatively, it can be stored as program code in the memory of the above device, and its function can be called and executed by a processing element of the device. The implementation of other modules is similar. Moreover, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed through the integrated logic circuits in the hardware of the processor element or through software instructions.

[0116] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 7 As shown, the electronic device may include: transceiver 121, processor 122, and memory 123.

[0117] Processor 122 executes computer execution instructions stored in memory, causing processor 122 to perform the scheme in the above embodiments. Processor 122 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0118] The memory 123 is connected to the processor 122 via the system bus and completes communication between them. The memory 123 is used to store computer program instructions.

[0119] Transceiver 121 can be used to obtain the task to be run and its configuration information.

[0120] The system bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. Transceivers are used to enable communication between database access devices and other computers (e.g., clients, read-write libraries, and read-only libraries). Memory may include random access memory (RAM) and may also include non-volatile memory.

[0121] The electronic device provided in this application embodiment can be the terminal device described in the above embodiments.

[0122] This application also provides a computer-readable storage medium storing computer instructions that, when executed on a computer, cause the computer to perform the fault prediction method of the data transmission system described above.

[0123] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When the at least one processor executes the computer program, it can implement the technical solution of the fault prediction method of the data transmission system in the above embodiments.

[0124] In the implementation of the computer program product, computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0125] Note that the above description is merely a preferred embodiment and the technical principles employed in this application. Those skilled in the art will understand that this application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments. Many other equivalent embodiments may be included without departing from the concept of this application, and the scope of this application is determined by the scope of the appended claims.

Claims

1. A fault prediction method for a data transmission system, the data transmission system comprising multiple modules, characterized in that, The method includes: At any given target time, obtain the first data transmission volume of any target module outputting data to other modules, and the second data transmission volume of other modules inputting data to the target module; Using a preset activation function, the first data transmission amount is compared with the corresponding first activation threshold parameter, and the second data transmission amount is compared with the corresponding second activation threshold parameter. Based on the comparison results, a first output value and a second output value are generated respectively. The activation function is used to measure the load pressure of the target module based on the amount by which the data transmission amount exceeds the corresponding activation threshold parameter. The estimated total transmission volume of the target module at the target time is calculated based on the first output value and the second output value, and the estimated total transmission volume is compared with the risk threshold parameter. Based on the comparison result, it is predicted whether the target module will fail. The first excitation threshold parameter, the second excitation threshold parameter, and the risk threshold parameter are determined by parameter fitting using sample data of data transmission volume and fault information of each module of the data transmission system at different time periods.

2. The method according to claim 1, characterized in that, The step of calculating the estimated total transmission volume of the target module at the target time based on the first output value and the second output value, comparing the estimated total transmission volume with a risk threshold parameter, and predicting whether the target module has failed based on the comparison result includes: Using a pre-built risk transmission model, the estimated total transmission volume of the target module at the target time is calculated based on the first output value and the second output value; If the estimated total transmission volume is greater than the risk threshold parameter, then the risk estimate value is determined to be 1, and the target module is predicted to fail. The risk transmission model is expressed as follows: i represents any target module, n represents the total number of modules, t represents any target time, and x ik,t x represents the first data transmission volume. ji,t This represents the second data transmission volume, where k and j both represent other modules, and γ i τ k and θ i The risk threshold parameter, the first activation threshold parameter, and the second activation threshold parameter are respectively represented by y. it Represents the risk estimate; sgn(x,γ) represents the sign function, which is 1 when x > γ and 0 otherwise; f(x ji,t ,θ i ) and f(x ik,t ,τ k ) represents the activation function, when x ji,t Greater than θ i x ik,t Greater than τ k The activation function is activated and produces the first output value and the second output value; α ij β ik ρ i1 ρ i2 It is determined by parameter fitting using sample data of data transmission volume and fault information of each module of the data transmission system at different time periods.

3. The method according to claim 1, characterized in that, The step of generating a first output value and a second output value based on the comparison result includes: When the first data transmission amount is greater than the first excitation threshold parameter, the first output value is generated according to the calculation method defined by the excitation function based on the amount by which the first data transmission amount exceeds the first excitation threshold parameter. When the second data transmission amount is greater than the second excitation threshold parameter, the second output value is generated according to the calculation method defined by the excitation function based on the amount by which the second data transmission amount exceeds the second excitation threshold parameter.

4. The method according to claim 2, characterized in that, The parameter fitting process includes: The sample data of the data transmission volume of each module of the data transmission system at different time periods are input into the risk transmission model for fitting calculation to obtain the risk fitting estimate. The objective function is calculated based on the risk fitting estimate and the fault information of each module in the sample data; In the risk transmission model, each parameter α ij β ik ρ i1 ρ i2 τ k θ i γ i Each search space is searched using gradient descent until the objective function satisfies the convergence condition, at which point α is determined. ij β ik ρ i1 ρ i2 τ k θ i γ i Their respective values.

5. The method according to claim 4, characterized in that, Also includes: During the search process using the gradient descent method, for each parameter of the risk transmission model, the initial value of the parameter that failed the previous fitting is taken as the center, and the distance between the parameter value of the last iteration in the previous fitting failure and the center is taken as the radius r. Following the direction of the largest gradient descent amplitude in the previous fitting failure, a new initial parameter value is selected at a position 2r away from the center for the next fitting.

6. The method according to claim 2, characterized in that, Also includes: The prediction accuracy of the risk transmission model is calculated based on the proportion of data transmission volume of each module in the data transmission system in the real scenario, and the difference between the risk estimate determined by the risk transmission model based on the data in the real scenario and the actual risk value in the real scenario. The risk transmission model is determined to be invalid based on the prediction accuracy.

7. The method according to claim 6, characterized in that, Also includes: When the prediction accuracy meets the preset conditions, the parameter γ in the risk transmission model is adjusted based on the data from the real scenario. i Make minor adjustments.

8. A fault prediction device for a data transmission system, the data transmission system comprising multiple modules, characterized in that, The device includes: The data transmission volume acquisition module is used to acquire, at any target time, a first data transmission volume of data output by any target module to other modules, and a second data transmission volume of data input by other modules to the target module; The activation function processing module is used to compare the first data transmission amount with the corresponding first activation threshold parameter and the second data transmission amount with the corresponding second activation threshold parameter using a preset activation function, and generate a first output value and a second output value according to the comparison results. The activation function is used to measure the load pressure of the target module based on the amount by which the data transmission amount exceeds the corresponding activation threshold parameter. The prediction module is used to calculate the estimated total transmission volume of the target module at the target time based on the first output value and the second output value, compare the estimated total transmission volume with the risk threshold parameter, and predict whether the target module will fail based on the comparison result. The first excitation threshold parameter, the second excitation threshold parameter, and the risk threshold parameter are determined by parameter fitting using sample data of data transmission volume and fault information of each module of the data transmission system at different time periods.

9. The apparatus according to claim 8, characterized in that, The prediction module includes: The estimated total transmission volume calculation unit is used to calculate the estimated total transmission volume of the target module at the target time based on the first output value and the second output value using a pre-built risk transmission model. The prediction unit is used to determine the risk estimate value as 1 and predict that the target module will fail if the estimated total transmission volume is greater than the risk threshold parameter. The risk transmission model is expressed as follows: i represents any target module, n represents the total number of modules, t represents any target time, and x ik,t x represents the first data transmission volume. ji,t This represents the second data transmission volume, where k and j both represent other modules, and γ i τ k and θ i The risk threshold parameter, the first activation threshold parameter, and the second activation threshold parameter are respectively represented by y. it Represents the risk estimate; sgn(x,γ) represents the sign function, which is 1 when x > γ and 0 otherwise; f(x ji,t ,θ i ) and f(x ik,t ,τ k ) represents the activation function, when x ji,t Greater than θ i x ik,t Greater than τ k The activation function is activated and produces the first output value and the second output value; α ij β ik ρ i1 ρ i2 It is determined by parameter fitting using sample data of data transmission volume and fault information of each module of the data transmission system at different time periods.

10. The apparatus according to claim 9, characterized in that, It also includes a parameter fitting module, which includes: The risk fitting estimate calculation unit is used to input sample data of the data transmission volume of each module of the data transmission system at different time periods into the risk transmission model for fitting calculation to obtain the risk fitting estimate. The objective function calculation unit is used to calculate the objective function based on the risk fitting estimate and the fault information of each module in the sample data; Parameter fitting unit, used for fitting each parameter α in the risk transmission model ij β ik ρ i1 ρ i2 τ k θ i γ i Each search space is searched using gradient descent until the objective function satisfies the convergence condition, at which point α is determined. ij β ik ρ i1 ρ i2 τ k θ i γ i Their respective values.

11. The apparatus according to claim 10, characterized in that, The parameter fitting module further includes a parameter initial value selection unit, specifically used for: During the search process of the parameter fitting unit using the gradient descent method, for each parameter of the risk transmission model, the initial value of the parameter that failed the previous fitting is taken as the center, and the distance between the parameter value of the last iteration in the previous fitting failure and the center is taken as the radius r. Following the direction of the largest gradient descent amplitude in the previous fitting failure, a new initial value of the parameter is selected at a position 2r away from the center for the next fitting.

12. The apparatus according to claim 9, characterized in that, It also includes a failure detection module, which includes: The prediction accuracy calculation unit is used to calculate the prediction accuracy of the risk transmission model based on the proportion of data transmission volume of each module in the data transmission system in the real scenario, and the difference between the risk estimate determined by the risk transmission model based on the data in the real scenario and the actual risk value in the real scenario. The failure determination unit is used to determine whether the risk transmission model has failed based on the prediction accuracy.

13. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the fault prediction method for the data transmission system as described in any one of claims 1-7.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the fault prediction method for the data transmission system as described in any one of claims 1-7.

15. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the fault prediction method for the data transmission system according to any one of claims 1-7.

Images