System risk monitoring method and device, equipment and medium

Abstract

The invention provides a system risk monitoring method and device, equipment and a medium, and relates to the field of data analysis, in particular to the field of big data analysis and artificial intelligence. Comprising the following steps: acquiring a duration in which a resource occupation amount continuously exceeds a proportion threshold of a total resource amount from historical sample data of system operation; calculating a system accumulated risk value of each period according to the duration, and obtaining a record of whether system crash occurs in each period; dividing an interval formed from the minimum value to the maximum value in the accumulated risk values of the system into a plurality of risk levels; obtaining the current duration when the current resource occupation amount of the system continuously exceeds the proportion threshold value of the total resource amount, calculating the current system accumulated risk value, and determining the current risk level to which the current system accumulated risk value belongs; determining the proportion of the number of system crashes generated when the system reaches the current risk level in the total number of system crashes corresponding to the plurality of risk levels, and taking the proportion as the current cumulative crash probability; and in response to the current cumulative collapse probability being greater than a preset threshold, executing a load reduction strategy.

Description

Technical Field

[0001] The present application relates to the field of data analysis technology, in particular to the field of big data analysis and artificial intelligence, and specifically to a system risk monitoring method, device, equipment and medium. Background Art

[0002] A complete system is operated by the coordinated operation of various components. The operation of each component consumes system resources such as disk space, CPU load, and memory. So when the total amount of system resources is fixed, how to schedule the resource allocation of each component to achieve balanced and stable overall performance and prevent systemic risks becomes the key to the system's self-maintenance stability mechanism. Summary of the invention

[0003] The present application provides a system risk monitoring method, device, equipment and medium to solve the problem of how to prevent system risks and maintain system stability in the prior art.

[0004] In a first aspect, the present application provides a system risk monitoring method, comprising:

[0005] Obtaining, from historical sample data of system operation, the duration during which the resource usage of the system continuously exceeds a threshold value of the proportion of the total system resources;

[0006] Taking time T as a period, calculating the system cumulative risk value of each period within N periods according to the duration, and obtaining a record of whether a system crash occurs within each period;

[0007] According to the records, the interval from the minimum value to the maximum value of the system cumulative risk value of each period is divided into a plurality of risk levels;

[0008] According to the current real-time operation data of the system, obtaining the current duration of the current resource usage of the system in the current cycle continuously exceeding the ratio threshold of the total system resources;

[0009] Calculate the current system cumulative risk value according to the current duration, and determine the current risk level to which the current system cumulative risk value belongs from the multiple risk levels;

[0010] Determine, according to the records, the proportion of the number of system crashes generated when the system reaches the current risk level to the total number of system crashes corresponding to the multiple risk levels, and use the proportion as the current cumulative crash probability;

[0011] In response to the current cumulative crash probability being greater than a preset threshold, a preset load reduction strategy is executed for the system.

[0012] In a second aspect, the present application provides a system risk monitoring device, comprising:

[0013] A first acquisition module is used to obtain, from historical sample data of system operation, a duration during which the resource usage of the system continuously exceeds a threshold value of the proportion of the total system resources;

[0014] A risk value calculation module is used to calculate the system cumulative risk value of each period within N periods according to the time length, taking time T as a period, and obtaining a record of whether a system crash occurs in each period;

[0015] A level classification module, used for classifying the interval from the minimum value to the maximum value of the system cumulative risk value of each period into a plurality of risk levels according to the records;

[0016] A second acquisition module is used to acquire, based on the current real-time operation data of the system, the current duration during which the current resource occupancy of the system in the current cycle continues to exceed the ratio threshold of the total system resources;

[0017] a level determination module, configured to calculate a current system cumulative risk value according to the current duration, and determine a current risk level to which the current system cumulative risk value belongs from among the multiple risk levels;

[0018] a cumulative crash probability calculation module, configured to determine, based on the records, a ratio of the number of system crashes generated when the system reaches the current risk level to the total number of system crashes corresponding to the multiple risk levels, and use the ratio as the current cumulative crash probability;

[0019] The scheduling module is used for executing a preset load reduction strategy for the system in response to the current cumulative crash probability being greater than a preset threshold.

[0020] In a third aspect, the present application also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a system risk monitoring method as described in any one of the embodiments of the present application is implemented.

[0021] In a fourth aspect, the present application further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, a system risk monitoring method as described in any one of the embodiments of the present application is implemented.

[0022] In a fifth aspect, the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements a system risk monitoring method as described in any one of the embodiments of the present application.

[0023] The system risk monitoring method, device, equipment and medium provided by the present application calculate the system cumulative risk value of each cycle by using the duration of the system resource occupation continuously exceeding the ratio threshold of the total system resource, and divide the interval from the minimum value to the maximum value in the system cumulative risk value of each cycle into multiple risk levels according to the record of whether the system crash occurs in each cycle. During the current real-time operation of the system, the current system cumulative risk value is calculated according to the real-time operation data of the system, and the current risk level to which it belongs is determined, and then the proportion of the number of system crashes generated when the system reaches the current risk level in the total number of system crashes corresponding to multiple risk levels is determined as the current cumulative crash probability. Finally, the current cumulative crash probability is compared with the preset threshold. When the current cumulative crash probability is greater than the preset threshold, it indicates that the system has a crash risk, and a pre-set load reduction strategy can be executed. Thus, for the situation where the system is prone to the risk of system crash due to excessive resource load, more accurate monitoring is achieved intelligently, and timely warnings and corresponding load reduction measures are taken, thereby improving the stability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 A flowchart of a system risk monitoring method provided in an embodiment of the present application;

[0026] Figure 2a A flow chart of dividing risk levels in the system risk monitoring method provided in an embodiment of the present application;

[0027] Figure 2b This is an example of a histogram of the number and interval distribution of system crashes in an embodiment of the present application;

[0028] Figure 3 A flow chart for evaluating risk levels in a system risk monitoring method provided in an embodiment of the present application;

[0029] Figure 4 A schematic diagram of the structure of a system risk monitoring device provided in an embodiment of the present application;

[0030] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0031] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0032] The present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for ease of description, only the parts related to the present application, rather than all structures, are shown in the accompanying drawings.

[0033] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance. The acquisition, storage, use, processing, etc. of data in the technical solution of this application comply with the relevant provisions of national laws and regulations. It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. The purpose is only to illustrate the feasibility of the implementation of the technical solution of this application, but it does not mean that the applicant has or must use the relevant content of the solution.

[0034] Figure 1 A flowchart of a system risk monitoring method provided in an embodiment of the present application. This embodiment can be applied to monitor the situation where the system is prone to system crash risk due to excessive resource load. The method can be executed by a system risk monitoring device, which can be implemented in software and / or hardware, preferably configured in an electronic device, such as a computer device or a server. Figure 1 As shown, the method specifically includes:

[0035] S101. Obtain, from historical sample data of system operation, a duration during which the system resource usage continues to exceed a threshold value of the proportion of the total system resources.

[0036] S102 , taking time T as a period, calculating the system cumulative risk value of each period within N periods according to the duration, and obtaining a record of whether a system crash occurs in each period.

[0037] S103. According to the records, the interval from the minimum value to the maximum value in the system cumulative risk value of each period is divided into a plurality of risk levels.

[0038] S104. According to the current real-time operation data of the system, obtain the current duration during which the current resource usage of the system in the current cycle continues to exceed the proportion threshold of the total system resources.

[0039] S105: Calculate the current system cumulative risk value according to the current duration, and determine the current risk level to which the current system cumulative risk value belongs from multiple risk levels.

[0040] S106. Determine, based on the records, the ratio of the number of system crashes generated when the system reaches the current risk level to the total number of system crashes corresponding to the multiple risk levels, and use the ratio as the current cumulative crash probability.

[0041] S107 . In response to the current cumulative crash probability being greater than a preset threshold, executing a preset load reduction strategy for the system.

[0042] The system consists of different modules. For example, the modules of the system can be marked as S1, S2, ..., S n , n is the number of modules. The system is operated by these n modules, and the resource usage of each module at time t can be marked as R 1,t , R 2,t , …, R n,t ,t=1,2,3,…,T. The total system resources are marked as R, and the system resource usage is the sum of the resource usage of each module, which is marked as R t =R 1,t +R 2,t +…+R n,t .

[0043] In the embodiment of the present application, the relationship between the system resource usage and the total system resources is used to quantify the system cumulative risk value. For example, when the system resource usage exceeds a certain percentage threshold of the total system resources, it indicates that the system is at risk of crashing. The percentage threshold can be marked as δ. That is, when R t >δ*R, there may be a risk of system crash, and this risk will increase with R t The continuous increase in the proportion threshold, that is, the long-term resource occupation exceeds the proportion threshold of the total amount, will cause the risk of system crash to increase rapidly. Therefore, in the embodiment of the present application, the duration of the system resource occupation continuously exceeding the proportion threshold of the total system resources is obtained, and then the system cumulative risk value is calculated based on the duration.

[0044] Specifically, in step S102, the system cumulative risk value of each period in N periods may be calculated based on the maximum value of the duration and the number of times the duration is greater than the time threshold. The system cumulative risk value is marked as H T , and is expressed by the following formula:

[0045]

[0046] Among them, R represents the total amount of system resources, δ represents the ratio threshold, and R t Indicates the system resource usage, Δt max Represents R t The maximum duration of exceeding δ*R, η represents the time threshold, N T,η Indicates that within T time R t The number of times that the duration of exceeding δ*R is greater than η occurs, is the indicative function, when R t When >δ*R, on the contrary

[0047] Exemplarily, δ is a decimal in the range of 0-1, and can take a value in the range of 0.9-0.99.

[0048] For the system, within time T, R t The more times the duration of δ*R exceeds η, the greater the continuous pressure on the system. Therefore, in this embodiment, the system cumulative risk value H T With R t The maximum duration Δt that exceeds δ*R max is proportional to R t The number of times N that the duration of δ*R exceeds η occurs T,η is proportional to. As for the integral part It is an exponential integral, indicating the system's cumulative risk value H T With R t > The longer the duration of δ*R is, the more exponentially it grows. Therefore, the accumulated risk value H can be more accurately calculated through the system. T To measure the growth characteristics of system load risk in reality.

[0049] Furthermore, the ratio threshold δ and the time threshold η can be determined as follows:

[0050] The system cumulative risk value of each period is recorded as H iT , record it as y i , where i is a natural number from 1 to N. i =1 indicates a system crash, y i = 0 means no system crash occurs. Construct target sample {H iT ,y i}, use the target sample to estimate the reasonable values ​​of the ratio threshold δ and the time threshold η. In addition, the ratio threshold δ and the time threshold η should ensure that the system cumulative risk value H iT With y iHighly positive correlation.

[0051] Get H iT The minimum value H in min and the maximum value H max , calculate H iT With H min The first difference of max With H min The second difference of , and calculate the ratio of the first difference to the second difference, recorded as Coming soon iT The data is normalized to the range of 0-1 and is expressed as:

[0052]

[0053] The estimated values ​​of δ and η are taken as the values ​​that minimize the following formula:

[0054]

[0055] in, Reaching the minimum means With y i The closest, due to With H iT Completely positively correlated, so this indicates that H iT With y i There is a high positive correlation, and the selected values ​​of the ratio threshold δ and time threshold η are the optimal parameter estimates.

[0056] Therefore, in the embodiment of the present application, the parameter estimation values ​​of the proportion threshold δ and the time threshold η are determined based on historical data, which is more accurate and effective than the manually set method. At the same time, the parameter estimation values ​​of the proportion threshold δ and the time threshold η can also be updated based on the new data generated by the system in real time, thereby further ensuring the effectiveness of the thresholds and providing a basis for the subsequent risk level division.

[0057] In step S103, the interval from the minimum value to the maximum value in the system cumulative risk value of each period is divided into multiple risk levels according to the record of whether the system crash occurs in each period. After the risk level is divided, when monitoring the system risk, the real-time system cumulative risk value of the system in a period of time can be calculated first, and then the risk level to which it belongs can be determined according to its size, and then whether the system has the risk of crashing can be further measured.

[0058] Figure 2a A flow chart of dividing risk levels in the system risk monitoring method provided in an embodiment of the present application. In this embodiment, the process of dividing risk levels may include:

[0059] S201. Divide the interval from the minimum value to the maximum value in the system cumulative risk value of each period into multiple sub-intervals.

[0060] S202: taking the system cumulative risk value of each period and its corresponding record as a target sample, and dividing multiple sub-intervals into multiple risk levels according to the sample quantity of the target sample in each sub-interval and the number of system crashes determined based on the records.

[0061] The symbols in the above embodiment are used, H iT The minimum value in H min , the maximum value is H max , H min =min{H iT ,i=1,2,…,N},H ma =max{H iT ,i=1,2,…,N}. min ,H max ] is discretized and divided into N0 subintervals. For example, represents rounding down, for example, Ensure that each subinterval contains 10 target samples on average to avoid the situation where a large number of subinterval target samples are 0. The present application embodiment does not impose any restrictions on the number of subintervals. Finally, the result of the subinterval division can be marked as: h i =(H min +(i-1)*Δ,H min +i*Δ],i=1,2,…,N0, where, And in particular, h0=(-∞,H min ],

[0062] Next, the system cumulative risk value of each cycle and its corresponding records are used as a target sample, and multiple sub-intervals are divided into multiple risk levels according to the sample number of the target sample in each sub-interval and the number of system crashes determined based on the records. For example, each risk level can contain multiple sub-intervals, and the total number of system crashes in the sub-intervals contained in different risk levels is different. The higher the risk level, the greater the proportion of the total number of system crashes in the sample number of the risk level.

[0063] In summary, the key points of risk classification are: the span of each risk level should be close (except for the lowest and highest two levels); the total number of target samples of each risk level after classification should show an exponential decay characteristic, that is, the number of target samples of low risk levels should be large, and the higher the risk level, the smaller the number of target samples; the higher the risk level, the higher the system collapse risk of each sub-interval is similar to the density d iThe bigger, Also bigger.

[0064] Among them, in the sub-interval h i Based on the i The target number of samples is marked as k i , mark the number of system crashes that occurred in the target sample in the subinterval as m i , that is, y in the target sample i The total number of 1. The risk of system collapse is similar to the density d i and They are expressed by the following formulas respectively:

[0065]

[0066] Where N0 represents the number of subintervals; Δ represents H max and H min The ratio of the difference between N0 and N1. Therefore This setting is because and The corresponding subintervals are represented as h0=(-∞,H min ]、 The span of these two subintervals is positive infinity + ∞, so it is reasonable that the density is 0.

[0067] In one implementation, risk levels may be divided as follows:

[0068] Based on the above information, statistics are performed to obtain a histogram of the number of system crashes and the distribution of sub-intervals. In the histogram, the horizontal axis represents the division of sub-intervals, and the vertical axis represents the number of system crashes in each sub-interval. Figure 2b This is an example of a histogram of the number and interval distribution of system crashes in an embodiment of the present application.

[0069] According to actual needs, it is assumed that there are 7 pre-divided risk levels, namely, extremely low risk, very low risk, low risk, medium risk, high risk, very high risk and extremely high risk. First, observe the rightmost sub-interval in the bar chart. If the number of system crashes If it is 0, the sub-intervals are directly merged. if Number of system crashes If it is also 0, continue to merge sub-intervals Until the number of system crashes is not 0 when a sub-interval is merged. The number of sub-intervals merged at this time is marked as n1, that is, the sub-interval They are merged into the highest risk level, level 7, and the risk level is set to extremely high.

[0070] It should be noted here that Figure 2b In the histogram shown, the rightmost subinterval h 16 The number of system crashes m 16 is not 0, so in Figure 2b In the example, according to the above division method, h 16 As the highest risk level. Figure 2b This is just an example. During the operation of the system, due to the differences in the collected samples, it is possible that the number of system crashes in the rightmost sub-interval is 0. For example, this may happen when the total number of samples in the rightmost sub-interval is very small. However, a small total number of samples does not mean that the risk level of the sub-interval will be low. Therefore, in addition to dividing the highest level in the above manner, the embodiment of the present application also performs a similar density processing on the number of system crashes, that is, in the divided risk level, the higher the risk level, the higher the system crash risk of each sub-interval. i and The larger the risk level, the more accurate the classification is. At the same time, the embodiment of the present application also evaluates the effectiveness of the risk levels that have been divided based on the average system crash risk similarity density of each risk level to further ensure the effectiveness of the classification. The specific implementation method of the evaluation will be described in detail below.

[0071] After determining the highest level, the next step is to determine the other risk levels in turn. Specifically, analyze the remaining sub-intervals, and to ensure that the span of the sub-intervals of the main levels is not too different, each level from level 6 to level 2 will be covered in turn. sub-intervals, that is, the sixth level includes sub-intervals The fifth level includes sub-intervals The fourth level includes sub-intervals The third level includes sub-intervals The second level includes subintervals The corresponding setting levels from level 6 to level 2 are: very high risk, high risk, medium risk, low risk and very low risk. It constitutes the first level and is set at the level of: extremely low risk.

[0072] Calculate the average system crash risk similarity density for each risk level above Respectively expressed as:

[0073]

[0074]

[0075] Therefore, the interval from the minimum to the maximum value of the system cumulative risk value in each cycle is divided into seven risk levels, which reflect the risk of the system's working state from high to low, and will be used to help users realize the intelligent scheduling of the system and reduce the system safety risk. It will be used to evaluate the effectiveness of the above risk classification model.

[0076] With the risk level, the system can be monitored in real time. First, based on the current real-time operation data of the system, the current duration of the current resource usage of the system in the current cycle that continues to exceed the threshold value of the proportion of the total system resources is obtained. Then, the current system cumulative risk value is calculated based on the current duration, and the current risk level to which the current system cumulative risk value belongs is determined from multiple risk levels. Among them, the method for calculating the current system cumulative risk value is the same as the method described in the above embodiment, which will not be repeated here. Since each risk level includes multiple sub-intervals, and each sub-interval includes the system cumulative risk values ​​of multiple cycles, the position of the current system cumulative risk value can be determined by comparing the size of the value, thereby determining the current risk level of the system.

[0077] In step S106, based on the record of whether a system crash occurs in each cycle, the ratio of the number of system crashes generated when the system reaches the current risk level to the total number of system crashes corresponding to multiple risk levels is determined, and the ratio is used as the current cumulative crash probability. Specifically, based on the record, the number of system crashes at each risk level is first obtained, and their sum is calculated. Then, the sum of the number of system crashes at the current risk level and each risk level lower than the current risk level in the multiple risk levels is calculated, and the ratio of the result to the sum is calculated, and the ratio is used as the current cumulative crash probability. The current cumulative crash probability ρ i It can be expressed as:

[0078]

[0079] Among them, in the example of dividing seven risk levels, if the current risk level is i, the current cumulative crash probability is ρ i , i is a natural number from 1 to 7. The number of system crashes at each risk level can be marked as M1, M2, ..., M7. Therefore, Indicates the total number of system crashes corresponding to each risk level. Represents the sum of the number of system crashes for the current risk level and each risk level lower than the current risk level.

[0080] By the cumulative collapse probability ρ iTo characterize the size of the system risk, the higher the cumulative probability of collapse, the greater the system risk. At the same time, the threshold θ can be configured according to the needs. The value of θ is between 0-1, for example, a value between 0.8-0.95 can be selected. i >θ, indicating that the standard for reducing the load on the system has been reached, and thus the system executes the pre-set load reduction strategy.

[0081] Among them, the strategy of reducing the load can be to suspend the task with the smallest load in the system until the risk level of the system drops to the target level, and the target cumulative crash probability corresponding to the target level is less than or equal to the preset threshold value θ. In this way, it can prevent the system from over-reducing the load, resulting in too low a system load, and avoid a large amount of idle computing resources and waste of resources.

[0082] In one embodiment, based on the above-mentioned embodiment, the method of the embodiment of the present application further includes: using the current real-time operation data of the system to update the historical sample data, and re-determining multiple risk levels based on the updated historical sample data, and then continuing to monitor the risk of the system according to the re-determined multiple risk levels. That is to say, the model and method for dividing risk levels can be continuously updated in the embodiment of the present application. When the system generates new data, the characteristics of the new data may change the levels divided according to the historical data. Therefore, timely updating of historical data and updating of risk levels can maintain the effectiveness of system risk monitoring. For example, based on the updated historical data, the optimal parameter values ​​of the proportion threshold δ and the time threshold η are re-obtained, and then the system cumulative risk value of each cycle is calculated based on the optimal parameter value, and then the interval from the minimum value to the maximum value in the system cumulative risk value of each cycle is divided into multiple risk levels according to the record of whether the system crash occurs in each cycle.

[0083] In addition, the average system crash risk similar density can be used and the cumulative collapse probability ρ i To evaluate the rationality and effectiveness of the above classification. Figure 3 This is a flow chart for evaluating the risk level in the system risk monitoring method provided in the embodiment of the present application. Figure 3 As shown, the following steps are included:

[0084] S301. Calculate the average system crash risk similarity density and cumulative crash probability for each risk level.

[0085] S302: If the average system crash risk similarity density and cumulative crash probability of each risk level are both highly positively correlated with the corresponding risk level, and the average system crash risk similarity density is positively correlated with the cumulative crash probability, then it is determined that the division of multiple risk levels is effective.

[0086] First, if The high positive correlation with the corresponding level indicates that it is not likely to have a situation where the average system crash risk of the high risk level is similar to a lower density, otherwise it would be contrary to the basic facts. i If the system crash probability is highly positively correlated with the corresponding level, it means that it is not easy for the high risk level to have a lower cumulative crash probability, otherwise it would be contrary to the basic facts. In turn, the effectiveness of the risk level division can be evaluated by evaluating whether the average system crash risk similarity density and cumulative crash probability of each risk level are highly positively correlated with the corresponding risk level. At the same time, and ρ i Whether it is highly positively correlated can also be used to evaluate whether the risk level classification is effective.

[0087] Combined with the above and ρ i Only when all three positive correlation evaluation algorithms pass the positive correlation evaluation can the risk level classification model be considered to be still stable and effective. Otherwise, it means that the classification result of the model is problematic and the model should be systematically checked or rebuilt based on updated historical data.

[0088] In one embodiment, the following method can be used to evaluate Whether the corresponding level height is positively correlated, still taking the above embodiment in which the risk level includes seven levels as an example, the formula is expressed as:

[0089]

[0090] in, express exist In order of size, In the assumption When independent of the ordinal variable i The theoretical average value of E(·) is the expected function. At this time, if the ordinal variables i and If we consider X and Y as distribution variables with average values ​​between 1 and 7, then it is not difficult to prove that when the two are independent,

[0091] It should be noted here that in the above In the calculation formula, for and the actual difference of the ordinal variable i, which is compared with The ratio operation is performed to compare the actual difference value with the " How big is the difference between the theoretical value and the case where the ordinal variable i is completely independent?

[0092] Obviously, when When it is completely positively correlated with the ordinal variable i, When When and the ordinal variable i are independent, It is generally believed that if the risk level is divided reasonably, [ There are only one or two inconsistencies with i, and there should be a small error in the adjacent levels, so The value of is generally not more than 5% under reasonable circumstances, and 5% is also the critical point of a small probability event. If it exceeds 5%, it is considered abnormal. When It is highly positively correlated with its corresponding level.

[0093] Accordingly, ρ can be evaluated as follows i Whether the corresponding level height is positively correlated, still taking the above embodiment in which the risk level includes seven levels as an example, the formula is expressed as:

[0094]

[0095] Based on the above similar description, when col ρ <5% can be considered i It is highly positively correlated with its corresponding level.

[0096] Compare and ρ i Whether it is highly positively correlated, we can use the Spearman rank correlation coefficient to calculate and ρ i The rank correlation coefficient is used for hypothesis testing. If the hypothesis test passes, it can be considered that the two are positively correlated.

[0097] It should be noted that the embodiments of the present application do not impose any limitation on the positive correlation evaluation method adopted, and the method described above is only preferred.

[0098] The system risk monitoring method, device, equipment and medium provided by the present application calculate the system cumulative risk value of each cycle by using the duration of the system resource occupation continuously exceeding the ratio threshold of the total system resource, and divide the interval from the minimum value to the maximum value in the system cumulative risk value of each cycle into multiple risk levels according to the record of whether the system crash occurs in each cycle. During the current real-time operation of the system, the current system cumulative risk value is calculated according to the real-time operation data of the system, and the current risk level to which it belongs is determined, and then the proportion of the number of system crashes generated when the system reaches the current risk level in the total number of system crashes corresponding to multiple risk levels is determined as the current cumulative crash probability. Finally, the current cumulative crash probability is compared with the preset threshold. When the current cumulative crash probability is greater than the preset threshold, it indicates that the system has a crash risk, and a pre-set load reduction strategy can be executed. Thus, for the situation where the system is prone to the risk of system crash due to excessive resource load, more accurate monitoring is achieved intelligently, and timely warnings and corresponding load reduction measures are taken, thereby improving the stability of the system.

[0099] In the existing technical methods, it is generally entirely dependent on the personal experience of the system operator to evaluate whether the system load pressure can be tolerated, and it is basically impossible to automatically and intelligently schedule the system tasks. Once the task load is too heavy, it will lead to system crash. Therefore, there is a need for a complete set of evaluation methods that can effectively perform system scheduling and configuration work without human intervention. At the same time, it is also necessary to systematically evaluate the method and be able to detect the failure of the method in time so as to reconstruct or improve the current model.

[0100] In the embodiment of the present application, the system is prone to the risk of system crash due to excessive resource load. It does not simply use a certain operating parameter of the system as a monitoring target and a basis for measuring whether it is necessary to implement a load reduction strategy for the system, but proposes to construct a risk measurement H T The method is used, and on this basis, an intelligent risk grading method is designed. Based on the grading results, a system risk intelligent scheduling method is created to select and close some tasks according to the load pressure in the actual working scenario, and finally form an intelligent risk control mechanism. At the same time, the embodiment of the present application also introduces a real-time update evaluation method of the model to ensure the effectiveness of the risk scheduling model. When the effectiveness of the evaluation model fails, the model is reconstructed to avoid the continued use of the invalid model.

[0101] Figure 4 This is a schematic diagram of the structure of a system risk monitoring device provided in an embodiment of the present application. Figure 4 As shown, the system risk monitoring device 40 includes:

[0102] The first acquisition module 401 is used to obtain, from historical sample data of system operation, a duration during which the resource usage of the system continuously exceeds a threshold value of the proportion of the total system resources.

[0103] The risk value calculation module 402 is used to calculate the system cumulative risk value of each period in N periods according to the time length, taking time T as a period, and obtain a record of whether a system crash occurs in each period.

[0104] The level classification module 403 is used to classify the interval from the minimum value to the maximum value of the system cumulative risk value of each period into multiple risk levels according to the records.

[0105] The second acquisition module 404 is used to acquire, based on the current real-time operation data of the system, the current duration during which the current resource occupancy of the system in the current cycle continues to exceed the ratio threshold of the total system resources.

[0106] The level determination module 405 is used to calculate the current system cumulative risk value according to the current duration, and determine the current risk level to which the current system cumulative risk value belongs from the multiple risk levels.

[0107] The cumulative crash probability calculation module 406 is used to determine the proportion of the number of system crashes generated when the system reaches the current risk level to the total number of system crashes corresponding to the multiple risk levels based on the records, and use the proportion as the current cumulative crash probability.

[0108] The scheduling module 407 is used to execute a preset load reduction strategy for the system in response to the current cumulative crash probability being greater than a preset threshold.

[0109] In some embodiments, the risk value calculation module 402 is specifically used to:

[0110] The system cumulative risk value of each period in N periods is calculated according to the maximum value of the duration and the number of times the duration is greater than the time threshold.

[0111] In some embodiments, the system accumulates a risk value H T It is expressed by the following formula:

[0112]

[0113] Wherein, R represents the total amount of system resources, δ represents the ratio threshold, and R t represents the resource usage of the system, Δt max Represents R t The maximum duration of exceeding δ*R, η represents the time threshold, N T,η Indicates that within T time Rt The number of times that the duration of exceeding δ*R is greater than η occurs, is the indicative function, when R t When >δ*R, on the contrary

[0114] In some embodiments, the device further includes a parameter estimation module, wherein the parameter estimation module is specifically configured to:

[0115] The system cumulative risk value of each cycle is recorded as H iT , and record the record as y i , where i is a natural number from 1 to N;

[0116] Get H iT The minimum value H in min and the maximum value H max , calculate H iT With H min The first difference of max With H min The second difference of , and calculate the ratio of the first difference to the second difference, recorded as

[0117] The estimated values ​​of δ and η are taken as the values ​​that minimize the following formula:

[0118] In some embodiments, the grading module 403 includes:

[0119] A sub-interval division unit, used for dividing the interval from the minimum value to the maximum value of the system cumulative risk value of each period into a plurality of sub-intervals;

[0120] A level division unit is used to take the system cumulative risk value of each cycle and its corresponding record as a target sample, and divide the multiple sub-intervals into multiple risk levels according to the sample number of the target sample in each sub-interval and the number of system crashes determined based on the record.

[0121] In some embodiments, among the multiple risk levels, a higher risk level has a larger corresponding average system crash risk similarity density, wherein the average system crash risk similarity density is determined according to the system crash risk similarity density of each sub-interval included in the corresponding risk level, and the system crash risk similarity density d i It is expressed by the following formula:

[0122]

[0123] Among them, k uIndicates the number of samples of the target sample in each subinterval, m i represents the number of system crashes in each subinterval, N0 represents the number of subintervals; H max It represents the maximum value of the system cumulative risk value in each period, H min represents the minimum value of the system cumulative risk value in each period, Δ represents H max and H min The ratio of the difference to N0.

[0124] In some embodiments, the cumulative crash probability calculation module 406 includes:

[0125] A first calculation unit, configured to obtain the number of system crashes of each risk level according to the record, and calculate a first sum value thereof;

[0126] A second calculation unit, configured to calculate a sum of the number of system crashes of the current risk level and each risk level lower than the current risk level among the multiple risk levels as a second sum value;

[0127] The third calculation unit is used to calculate a ratio of the second sum value to the first sum value, and use the ratio as the current cumulative crash probability.

[0128] In some embodiments, the load reduction strategy includes:

[0129] The system is paused starting from the task with the lightest load in the system until the risk level of the system drops to a target level, and the target cumulative crash probability corresponding to the target level is less than or equal to the preset threshold.

[0130] In some embodiments, the apparatus further comprises an update module, wherein the update module comprises:

[0131] a level updating unit, configured to update the historical sample data using the current real-time operation data of the system, and to re-determine a plurality of risk levels based on the updated historical sample data;

[0132] The continuing monitoring unit is used to continue to perform risk monitoring on the system according to the re-determined multiple risk levels.

[0133] In some embodiments, the apparatus further comprises an evaluation module, the evaluation module comprising:

[0134] A fourth calculation unit is used to calculate the average system collapse risk similarity density and cumulative collapse probability for each risk level;

[0135] A correlation evaluation unit is used to determine that the division of the multiple risk levels is effective if the average system crash risk similarity density and the cumulative crash probability of each risk level are both highly positively correlated with the corresponding risk level, and the average system crash risk similarity density is positively correlated with the cumulative crash probability.

[0136] The system risk monitoring device provided in the embodiment of the present application can be used to execute the technical solution of the system risk monitoring method in the above embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0137] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the level division module 403 can be a separately established processing element, or it can be integrated in a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a processing element of the above device. The function of the above level division module 403. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0138] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown, the electronic device may include: a transceiver 121 , a processor 122 , and a memory 123 .

[0139] The processor 122 executes the computer execution instructions stored in the memory, so that the processor 122 executes the scheme in the above embodiment. The processor 122 can be a general-purpose processor, including a central processing unit CPU, a network processor (network processor, NP), etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0140] The memory 123 is connected to the processor 122 via a system bus and completes communication between them. The memory 123 is used to store computer program instructions.

[0141] The transceiver 121 may be used to obtain tasks to be executed and configuration information of the tasks to be executed.

[0142] The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The transceiver is used to realize the communication between the database access device and other computers (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory.

[0143] The electronic device provided in the embodiment of the present application may be the terminal device of the above embodiment.

[0144] An embodiment of the present application also provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes the technical solution of the system risk monitoring method of the above embodiment.

[0145] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, the technical solution of the system risk monitoring method in the above embodiment can be implemented.

[0146] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present application, and the programming language includes an object-oriented programming language, such as Java, Smalltalk, C++, and also includes a conventional procedural programming language, such as "C" language or similar programming language. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on the remote computer, or completely on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0147] Note that the above are only preferred embodiments of the present application and the technical principles used. Those skilled in the art will understand that the present application is not limited to the specific embodiments herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A system risk monitoring method, characterized in that: include: Obtaining, from historical sample data of system operation, the duration during which the resource usage of the system continuously exceeds a threshold value of the proportion of the total system resources; Taking time T as a period, calculating the system cumulative risk value of each period within N periods according to the duration, and obtaining a record of whether a system crash occurs within each period; According to the records, the interval from the minimum value to the maximum value of the system cumulative risk value of each period is divided into a plurality of risk levels; According to the current real-time operation data of the system, the current duration of the current resource usage of the system in the current cycle continuously exceeding the ratio threshold of the total system resources is obtained; Calculate the current system cumulative risk value according to the current duration, and determine the current risk level to which the current system cumulative risk value belongs from the multiple risk levels; Determine, according to the records, the proportion of the number of system crashes generated when the system reaches the current risk level to the total number of system crashes corresponding to the multiple risk levels, and use the proportion as the current cumulative crash probability; In response to the current cumulative crash probability being greater than a preset threshold, a preset load reduction strategy is executed for the system.

2. The method according to claim 1, characterized in that The calculating the system cumulative risk value of each period in N periods according to the duration includes: The system cumulative risk value of each period in N periods is calculated according to the maximum value of the duration and the number of times the duration is greater than the time threshold.

3. The method according to claim 2, characterized in that The system cumulative risk value H T It is expressed by the following formula: Wherein, R represents the total amount of system resources, δ represents the ratio threshold, and R t represents the resource usage of the system, Δt max Represents R t The maximum duration of exceeding δ*R, η represents the time threshold, N T,η Indicates that within T time R t The number of times that the duration of exceeding δ*R is greater than η occurs, is the indicative function, when R t >δ*R, on the contrary 4. The method according to claim 3, characterized in that Also includes: The system cumulative risk value of each cycle is recorded as H iT , and record the record as y i , where i is a natural number from 1 to N; Get H iT The minimum value H in min and the maximum value H max , calculate H iT With H min The first difference of max With H min The second difference of , and calculate the ratio of the first difference to the second difference, recorded as The estimated values ​​of δ and η are taken as the values ​​that minimize the following formula:

5. The method according to claim 1, characterized in that According to the records, the interval from the minimum value to the maximum value in the system cumulative risk value of each period is divided into multiple risk levels, including: Dividing the interval from the minimum value to the maximum value of the system cumulative risk value of each period into a plurality of sub-intervals; The system cumulative risk value of each cycle and the corresponding record are taken as a target sample, and the multiple sub-intervals are divided into multiple risk levels according to the sample quantity of the target sample in each sub-interval and the number of system crashes determined based on the records.

6. The method according to claim 5, characterized in that Among the multiple risk levels, a higher risk level has a larger corresponding average system crash risk similarity density, wherein the average system crash risk similarity density is determined according to the system crash risk similarity density of each sub-interval included in the corresponding risk level, and the system crash risk similarity density d i It is expressed by the following formula: Among them, k i Indicates the number of samples of the target sample in each subinterval, m i represents the number of system crashes in each subinterval, N0 represents the number of subintervals; H max It represents the maximum value of the system cumulative risk value in each period, H min represents the minimum value of the system cumulative risk value in each period, Δ represents H max and H min The ratio of the difference to N0.

7. The method according to claim 1, characterized in that Determining, according to the records, the proportion of the number of system crashes generated when the system reaches the current risk level to the total number of system crashes corresponding to the multiple risk levels, and using the proportion as the current cumulative crash probability, includes: According to the records, the number of system crashes at each risk level is obtained, and the first sum value thereof is calculated; Calculating the sum of the number of system crashes of the current risk level and each risk level lower than the current risk level among the multiple risk levels as a second sum value; A ratio of the second sum value to the first sum value is calculated, and the ratio is used as the current cumulative crash probability.

8. The method according to claim 1, characterized in that The load reduction strategies include: The task with the lightest load in the system is suspended starting from the task, until the risk level of the system drops to a target level, and the target cumulative crash probability corresponding to the target level is less than or equal to the preset threshold.

9. The method according to claim 1, characterized in that: Also includes: updating the historical sample data using the current real-time operation data of the system, and re-determining a plurality of risk levels based on the updated historical sample data; The system continues to be risk monitored according to the re-determined plurality of risk levels.

10. The method according to claim 6, characterized in that Also includes: Calculate the average system crash risk similarity density and cumulative crash probability for each risk level; If the average system crash risk similarity density and cumulative crash probability of each risk level are both highly positively correlated with the corresponding risk level, and the average system crash risk similarity density is positively correlated with the cumulative crash probability, then it is determined that the division of the multiple risk levels is effective.

11. A system risk monitoring device, characterized in that: include: A first acquisition module is used to obtain, from historical sample data of system operation, a duration during which the resource usage of the system continuously exceeds a threshold value of the proportion of the total system resources; A risk value calculation module is used to calculate the system cumulative risk value of each period within N periods according to the time length, taking time T as a period, and obtaining a record of whether a system crash occurs in each period; A level classification module, used for classifying the interval from the minimum value to the maximum value of the system cumulative risk value of each period into a plurality of risk levels according to the records; A second acquisition module is used to acquire, based on the current real-time operation data of the system, the current duration during which the current resource occupancy of the system in the current cycle continues to exceed the ratio threshold of the total system resources; a level determination module, configured to calculate a current system cumulative risk value according to the current duration, and determine a current risk level to which the current system cumulative risk value belongs from among the multiple risk levels; a cumulative crash probability calculation module, configured to determine, based on the records, a ratio of the number of system crashes generated when the system reaches the current risk level to the total number of system crashes corresponding to the multiple risk levels, and use the ratio as the current cumulative crash probability; The scheduling module is used for executing a preset load reduction strategy for the system in response to the current cumulative crash probability being greater than a preset threshold.

12. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the system risk monitoring method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the system risk monitoring method according to any one of claims 1 to 10.

14. A computer program product, characterized in that It comprises a computer program, which, when executed by a processor, implements the system risk monitoring method according to any one of claims 1 to 10.

Images