A network fault detection method and system based on data analysis

Abstract

This invention belongs to the field of data processing technology, specifically relating to a network fault detection method and system based on data analysis. The method includes: real-time acquisition and normalization of multiple operational status indicators of target nodes; construction of a node state vector sequence; construction of a first-order difference vector and a second-order difference vector based on the differences between vectors at adjacent time points in the sequence, serving as the state transition velocity vector and state transition acceleration vector, respectively; construction of a Gram determinant; acquisition of the trajectory torsion area; acquisition of a computing power oscillation index by combining the modal distribution of the state transition velocity vector; recursively constructing an adaptive basis envelope based on the computing power oscillation index and a preset resource recovery time window length; and comparison of the two to determine whether a network computing node has experienced a processing fault. This invention can effectively suppress false alarms caused by normal business surges, while amplifying underlying hidden fault characteristic signals to reduce missed alarms, thus comprehensively improving detection accuracy.

Description

Technical Field

[0001] This invention relates to the field of data processing technology. More specifically, this invention relates to a network fault detection method and system based on data analysis. Background Technology

[0002] With the application of distributed computing architecture, modern network systems carry dense concurrent business tasks. The underlying network computing nodes continuously generate massive and multidimensional state data during operation and interaction. Accurate anomaly detection of these high-concurrency, multidimensional operational state data is the key to ensuring the stable operation of network information processing systems.

[0003] Existing data analysis methods typically employ multidimensional sequence anomaly detection algorithms based on statistical distance. The system collects various operational status indicators of computing nodes in real time, combines them into multidimensional feature vectors at each sampling time, calculates the covariance matrix using the feature vector set from the historical normal phase, and then obtains the Mahalanobis distance between the current sampled feature vector and the historical distribution center. When the Mahalanobis distance is greater than a preset safety boundary threshold, the node is determined to be abnormal. This absolute spatial distance analysis method can effectively capture extreme value data that deviates from the normal distribution.

[0004] In complex real-world operating conditions, traditional algorithms based on static distance metrics have two significant drawbacks. First, normal peak concurrent business traffic can lead to a strong correlation and synchronous surge in multi-dimensional operational metrics of nodes, resulting in a significantly inflated Mahalanobis distance. This can easily misjudge normal business surges as anomalies, triggering false alarms. Second, pure static distance metrics fail to fully consider the evolutionary correlation of multi-dimensional states over time. When a system experiences memory leaks leading to frequent page swapping oscillations or deadlocks, the operational data exhibits high-frequency, irregular distortions in phase space, while the absolute values ​​do not exceed limits in the short term. Traditional algorithms lose their ability to perceive temporal trajectories, resulting in a very high risk of missing hidden faults. Summary of the Invention

[0005] To address the technical problems of existing static detection algorithms leading to false alarms in normal business operations and missed detections of hidden faults, this invention provides solutions in the following aspects.

[0006] In a first aspect, the present invention provides a network fault detection method based on data analysis, comprising: Multiple operational status indicators of the target node are collected in real time, and each indicator is normalized. A node state vector sequence is constructed based on the normalized indicators. The difference between the node state vectors at adjacent time points in the node state vector sequence is obtained, and first-order and second-order difference vectors are constructed sequentially as the state transition velocity vector and state transition acceleration vector at each sampling time, respectively. A Gram determinant is constructed based on the state transition velocity vector and state transition acceleration vector at each sampling time to obtain the trajectory torsion area at each sampling time. The computational power oscillation index at each sampling time is obtained by combining the trajectory torsion area at each sampling time with the modal distribution of the state transition velocity vector at each sampling time. An adaptive basis envelope is recursively constructed based on the computational power oscillation index at each sampling time and the preset resource reclamation time window length. The computational power oscillation index is compared with the adaptive basis envelope to determine whether the network computing node has experienced a processing failure.

[0007] This invention constructs a node state vector sequence by normalizing various operational status indicators and extracts the state transition velocity vector and state transition acceleration vector representing the dynamic evolution of the time series, thus breaking the limitation of traditional purely static distance metrics that sever the correlation of data evolution. This invention utilizes Gram determinant to extract the trajectory torsion area and combines it with the modal distribution of the state transition velocity vector to obtain the computing power oscillation index. It can effectively suppress the synchronous rise of indicators caused by normal concurrent business peaks by utilizing the geometric topological properties of multidimensional phase space, while amplifying high-frequency irregular distortion anomaly signals caused by hidden faults at the system's underlying level. This invention combines a preset resource recovery time window length to recursively construct an adaptive basis envelope, providing a flexible judgment boundary for the computing power oscillation index that can accommodate the periodic noise fluctuations of the system. This achieves in-depth mining of the evolution characteristics of multidimensional operational status data and effectively solves the problem that existing static detection algorithms are prone to false alarms of normal business surges and false alarms of hidden faults due to the difficulty in distinguishing between normal business surges and real system faults under complex working conditions.

[0008] Preferably, the step of sequentially constructing a first-order difference vector and a second-order difference vector, which serve as the state transition velocity vector and state transition acceleration vector at each sampling time, respectively, includes: performing element-wise subtraction on the node state vector at each sampling time and the node state vector at the previous sampling time to obtain the state transition velocity vector at each sampling time; and performing element-wise subtraction on the state transition velocity vector at each sampling time and the state transition velocity vector at the previous sampling time to obtain the state transition acceleration vector at each sampling time.

[0009] Preferably, obtaining the trajectory torsion area corresponding to each sampling time includes: calculating the sum of squares of the elements in each dimension of the state transition velocity vector as the velocity modulus feature; calculating the sum of squares of the elements in each dimension of the state transition acceleration vector as the acceleration modulus feature; calculating the sum of the products of the state transition velocity vector and the state transition acceleration vector in the corresponding dimensions, and squaring the sum of the products as the orthogonal cross feature; multiplying the velocity modulus feature and the acceleration modulus feature, subtracting the orthogonal cross feature, and then performing a square root operation to obtain the trajectory torsion area.

[0010] This invention calculates the product of the velocity modulus and the acceleration modulus, subtracts the orthogonal crossover feature, and then performs a square root operation to obtain the area of ​​the parallelogram spanned by the state transition velocity vector and the state transition acceleration vector in the multidimensional phase space. When the network computing node is under normal peak service, various resource indicators show synchronous and linear increases in the same direction, making the two state transition vectors highly parallel and the trajectory torsion area significantly reduced and close to zero. However, when the system experiences faults such as disordered preemption of underlying resources, the evolution trajectories of the two state transition vectors undergo drastic orthogonal deflection, causing the trajectory torsion area to expand nonlinearly. Thus, by utilizing the geometric features of the vector angle, the common mode amplification phenomenon caused by a sudden increase in a single normal network service is effectively removed, achieving accurate extraction of hidden fault features.

[0011] Preferably, obtaining the computing power oscillation index corresponding to each sampling moment includes: calculating the sum of the absolute values ​​of each dimension in the state transition velocity vector, dividing the sum of the absolute values ​​by the square root of the number of running state indicators to obtain the normalized state offset amplitude corresponding to each sampling moment; mapping the normalized state offset amplitude to a nonlinear amplification factor using an exponential function with the natural constant as the base, and using the nonlinear amplification factor to amplify the corresponding trajectory torsion area to obtain the computing power oscillation index corresponding to each sampling moment.

[0012] This invention calculates the computing power oscillation index and closely combines the distortion of the phase space trajectory with the overall operating kinetic energy of the system. When a serious underlying fault occurs in the system and various indicators deviate sharply from the benchmark, the nonlinear amplification factor has a significant amplification effect on the trajectory distortion characteristics. Conversely, if it is only a local high-frequency fluctuation of a very small indicator, the exponential amplification effect will be suppressed because the normalized state deviation is small. Thus, at the data level, the underlying hidden fault signals are effectively separated from background random interference, avoiding the risk of misjudgment caused by the detection system capturing non-substantial noise artifacts.

[0013] Preferably, the step of recursively constructing an adaptive basis envelope based on the computing power oscillation index at each sampling time and a preset resource recovery time window length includes: constructing an attenuation adjustment factor for smoothing periodic noise floor fluctuations based on the preset resource recovery time window length; obtaining the basis smoothing center at the previous sampling time and calculating the smoothing evolution deviation between the computing power oscillation index at the current sampling time and the basis smoothing center at the previous sampling time; superimposing the product of the attenuation adjustment factor and the smoothing evolution deviation onto the basis smoothing center at the previous sampling time to recursively obtain the basis smoothing center at the current sampling time; and determining the adaptive basis envelope based on the basis smoothing center at the current sampling time and the prediction residuals between the computing power oscillation index at each sampling time and the corresponding basis smoothing center.

[0014] In this invention, the update of the base smoothing center is controlled by the decay adjustment of the time window, which can smoothly absorb the normal noise fluctuations caused by the periodic mechanisms such as the regular dirty page flushing or garbage collection of the operating system. This ensures that the base smoothing center can be robustly deduced even when short-term sudden business occurs, effectively avoiding the risk of iterative divergence caused by the judgment threshold being overly sensitive to short-term abnormal values.

[0015] Preferably, determining the adaptive base envelope includes: tracing back a historical time period defined by a preset resource recovery time window length; calculating the prediction residual between the computing power oscillation index and the corresponding base smoothing center at each sampling moment within the historical time period; obtaining the local prediction deviation standard deviation based on the statistical dispersion of all the prediction residuals, as the envelope expansion width; and superimposing the envelope expansion width onto the base smoothing center at the current sampling moment to obtain an adaptive base envelope that includes the system's periodic fluctuation tolerance.

[0016] This invention dynamically expands the envelope width based on the evolution of historical residuals, building upon the smooth center of the base. When the actual computing power fluctuation deviates more drastically from the smooth center within a historically defined time period, the adaptive base envelope shifts upward and widens the safety tolerance band, thereby establishing a dynamic defense line that fully accommodates the periodic fluctuations of the system. This allows the anomaly judgment boundary to self-adjust in accordance with the dynamic rhythm of the hardware environment, improving the adaptability in long-term complex cluster operation and maintenance scenarios.

[0017] Preferably, the step of comparing the computing power oscillation index with the adaptive base envelope to determine whether a network computing node has experienced a processing failure includes: determining that the target node has experienced a processing failure at the corresponding sampling time in response to the computing power oscillation index being greater than the adaptive base envelope; and determining that the target node is in a normal and stable processing phase in response to the computing power oscillation index not being greater than the adaptive base envelope.

[0018] Preferably, the normalization of various operational status indicators and the construction of a node state vector sequence based on the normalized operational status indicators include: obtaining the upper limit and lower limit of the hardware physical limits corresponding to each operational status indicator; and normalizing the maximum and minimum values ​​of each operational status indicator based on the upper limit and lower limit of the hardware physical limits.

[0019] Preferably, obtaining the upper and lower limits of the hardware physical limits corresponding to each operating status indicator includes: for operating status indicators without an absolute upper limit of hardware physical limits, using the maximum empirical waiting time of the corresponding hardware under rated load as the upper limit of the hardware physical limits.

[0020] Secondly, the present invention provides a network fault detection system based on data analysis, including a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the above-mentioned network fault detection method based on data analysis is implemented.

[0021] By adopting the above technical solution, a computer program is generated from the above-mentioned network fault detection method based on data analysis and stored in the memory so that it can be loaded and executed by the processor. In this way, a terminal device can be made based on the memory and the processor for convenient use.

[0022] The beneficial effects of this invention are as follows: This invention normalizes various operational status indicators and extracts state transition velocity vectors and state transition acceleration vectors, abandoning the method of simply comparing and judging based on static spatial distance, and introducing multi-dimensional phase space evolution trajectory topology into the field of network node monitoring; This invention utilizes Gram determinant features to extract the trajectory torsion area spanned by two transition vectors, and combines the modal distribution of the state transition velocity vector to obtain the computing power oscillation index. Utilizing the area collapse property of parallel vectors, it effectively suppresses the phenomenon of indicators rising in the same direction due to a sudden increase in a single normal network service, eliminating the persistent problem of massive false alarms, and effectively amplifying the disordered distortion characteristic signal when system oscillations are caused by multi-threaded deadlocks or slow memory leaks; This invention abandons the operation of manually preset fixed boundaries, and automatically deduces the adaptive base envelope for fault determination based on the preset resource reclamation time window length, enabling the anomaly judgment defense line to dynamically encompass the periodic noise fluctuations at the system's underlying level, improving the detection accuracy and fault capture robustness in the operation and maintenance of complex computing clusters. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating a network fault detection method based on data analysis according to the present invention; Figure 2 This is a schematic diagram illustrating the changes in central processing unit utilization in an embodiment of the present invention; Figure 3 This is a schematic diagram showing the change in trajectory torsion area at each sampling time in an embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the comparison between the computing power oscillation index and the adaptive base envelope, as well as the fault determination effect in an embodiment of the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0026] This invention discloses a network fault detection method based on data analysis, referring to... Figure 1 This includes steps S1-S5: S1. Collect multiple operational status indicators of the target node in real time, normalize each operational status indicator, and construct a node status vector sequence based on the normalized operational status indicators.

[0027] It should be noted that network computing nodes generate multi-dimensional state data during the processing of concurrent network tasks. Due to the huge differences in numerical scales of different dimensions of operational state indicators, the large numerical dimensions completely mask the small numerical dimensional features, which leads to a large error in the subsequent extraction of phase space geometric features. Therefore, this invention uses the physical limit boundaries of each operational state indicator to normalize each operational state indicator and transform heterogeneous data into a dimensionless sequence.

[0028] Specifically, the target node's operational status metrics are collected at a fixed sampling frequency, including CPU utilization, memory page miss rate, disk I / O wait time, and thread context switching frequency.

[0029] For example, Figure 2 This is a schematic diagram illustrating the changes in CPU utilization in an embodiment of the present invention. In the initial stage of monitoring, CPU utilization is at a low baseline level with slight random fluctuations, indicating that the system is in a normal and stable processing phase. As the sampling time progresses, CPU utilization increases, indicating that the system has entered a normal peak phase of network concurrent services. In the later stage of monitoring, CPU utilization exhibits high-frequency disordered oscillations, indicating that the system has experienced network computing node processing failures.

[0030] Obtain the upper and lower limits of the hardware physical limits corresponding to each operating status indicator. For operating status indicators such as disk input / output latency that do not have an absolute upper limit of hardware physical limits, use the empirical maximum latency of the corresponding hardware under rated load as the upper limit of the hardware physical limits.

[0031] Calculate the normalized state values ​​based on the various operating status indicators and the corresponding upper and lower limits of the hardware physical limits:

[0032] in, Indicates the first At the sampling time, the first Normalized state values ​​for each dimension; Indicates the first At the sampling time, the first Operational status indicators in several dimensions; Indicates the first The lower limit of hardware physical limits in each dimension; Indicates the first The upper limit of hardware physical limits in each dimension. When operating status indicators... When the value is larger, the normalized state value The larger the value, the more strictly data at different scales are compressed into a reasonable range of values.

[0033] The normalized state values ​​at the same sampling time are combined into a vector according to a fixed dimensional order to obtain the node state vector at that sampling time. As the sampling time progresses, the node state vectors at multiple sampling times together constitute a node state vector sequence.

[0034] S2. Obtain the difference between the node state vectors at adjacent time points in the node state vector sequence, and construct the first-order difference vector and the second-order difference vector in sequence, which are used as the state transition velocity vector and the state transition acceleration vector at each sampling time point, respectively.

[0035] Specifically, the first Node state vector at each sampling time With the Node state vector at each sampling time Perform the corresponding dimension element subtraction operation to obtain the first element. The state transition velocity vector corresponding to each sampling time. .

[0036] Furthermore, the first The state transition velocity vector corresponding to each sampling time. With the The state transition velocity vector corresponding to each sampling time. Perform the corresponding dimension element subtraction operation to obtain the first element. The state transition acceleration vector corresponding to each sampling time point .

[0037] By performing differential calculations on all vectors in the node state vector sequence, the state transition velocity vector and state transition acceleration vector corresponding to all sampling times are obtained.

[0038] S3. Construct a Gram determinant based on the state transition velocity vector and state transition acceleration vector at each sampling time, and obtain the trajectory torsion area corresponding to each sampling time.

[0039] It should be noted that, because the state indicators of network computing nodes grow synchronously and linearly when processing a single burst of high-concurrency network services, the state transition velocity vector and the state transition acceleration vector are parallel. However, when a memory leak occurs late in the process, leading to frequent paging or internal deadlock, the system resource scheduling becomes disordered and preemptive, causing the state transition velocity vector and the state transition acceleration vector to undergo a drastic orthogonal deflection. This makes it impossible for traditional distance calculation methods to distinguish the hidden anomalies caused by the vector angle. Therefore, this invention uses Gram determinant features to extract the area of ​​the parallelogram spanned by the state transition velocity vector and the state transition acceleration vector in the multidimensional phase space, which is used as the trajectory torsion area corresponding to each sampling moment, thereby separating the trajectory differences caused by normal services and system failures.

[0040] Specifically, based on the state transition velocity vector and state transition acceleration vector corresponding to each sampling moment, the trajectory torsion area at each sampling moment is calculated:

[0041] in, Indicates the first The trajectory torsion area at each sampling time is used to measure the degree of deflection and distortion of the evolving trajectory; This indicates the number of operational status indicators; Indicates the first The state transition velocity vector corresponding to each sampling time. The first in Each dimension element; Indicates the first The state transition acceleration vector corresponding to each sampling time point The first in Each dimension element.

[0042] When network computing nodes are in normal peak business periods, various resource indicators (such as CPU and IO) show a synchronous linear increase, and the state transition speed vector... With state transition acceleration vector High parallelism makes Approaching Trajectory torsion area The state transition velocity vector collapses and tends to 0, thus becoming more resilient to business fluctuations; however, when the system experiences process deadlock or resource preemption failures, the state transition velocity vector... With state transition acceleration vector The evolutionary trajectory underwent a dramatic orthogonal deflection and high-frequency distortion, resulting in Rapidly decreases the trajectory torsion area Nonlinear expansion occurs, thereby enabling accurate extraction of hidden fault characteristics.

[0043] For example, Figure 3 This is a schematic diagram illustrating the changes in trajectory torsion area at various sampling times in this embodiment of the invention. When the system is in a normal and stable processing phase and during normal peak network concurrent services, various operational status indicators change synchronously due to the unified increase in services. The extracted state transition velocity vector and state transition acceleration vector remain highly parallel, causing the trajectory torsion area to converge stably to an extremely low level, effectively suppressing the characteristic common mode amplification phenomenon caused by a sudden increase in a single normal network service. When a network computing node processing failure occurs, the evolution vector undergoes a violent orthogonal deflection due to disordered resource scheduling, causing the trajectory torsion area to jump rapidly, thereby amplifying the abnormal trajectory distortion differences caused by the hidden processing failure.

[0044] S4. Based on the trajectory torsion area at each sampling time and the modal distribution of the state transition velocity vector at each sampling time, obtain the computing power oscillation index corresponding to each sampling time.

[0045] It should be noted that since the increase in the trajectory torsion area may be due to local high-frequency fluctuations of extremely small indicators, the detection system may capture non-substantial noise artifacts. Therefore, this invention combines the modal distribution of the state transition velocity vector at each sampling moment to nonlinearly amplify the trajectory torsion area, obtain the computing power oscillation index corresponding to each sampling moment, and closely combine the distortion of the phase space trajectory with the overall operating kinetic energy invested by the system.

[0046] Specifically, based on the trajectory torsion area and state transition velocity vector at each sampling time, the computing power oscillation index at each sampling time is calculated:

[0047] in, Indicates the first The computing power fluctuation index at each sampling moment; Indicates the first The area of ​​trajectory torsion at each sampling time; This indicates the number of operational status indicators; Indicates the first The state transition velocity vector corresponding to each sampling time. The first in Perform absolute value operation on each dimension element; This represents an exponential function with the natural constant as its base.

[0048] This invention utilizes As the denominator, based on the theory of random walks in high-dimensional spaces, the sum of multidimensional absolute values... Perform statistically significant normalization scaling based on the high-dimensional random walk model. The expected deviation distance of independent random noise floor fluctuations in each dimension from the square root of the dimension. Proportional, therefore this invention adopts As a normalized scaling factor, it can offset the natural growth expectation caused by normal random noise in multidimensional space, effectively suppressing the impact of monitoring dimensions. This increases the linear accumulation of background noise, preventing the curse of dimensionality caused by exponential divergence. Compared to directly dividing by... To obtain the arithmetic mean, divide by This approach effectively approximates the sum of absolute values ​​across multiple dimensions as the true physical kinetic energy of the multidimensional phase space, ensuring that local high-frequency abnormal oscillation signals are not excessively diluted by the massive total amount of normal dimensions. When a network computing node experiences severe hardware failure or memory leaks leading to frequent paging, the system not only experiences severe trajectory distortion but also severe oscillations in various indicators deviating from the baseline. This results in a larger sum of absolute values ​​across all dimensions of the state transition speed. The larger the area of ​​the trajectory torsion. The more significant the amplification effect, the greater the computing power oscillation index. A sharp increase; conversely, if only a small amount of background noise causes the trajectory torsion area. Fluctuations are due to the sum of the absolute values ​​of each dimension of the state transition velocity. Smaller, computing power fluctuation index This will be suppressed. This invention, by calculating the computing power oscillation index, achieves the separation of underlying hidden fault signals from background random interference at the data level.

[0049] S5. Based on the computing power oscillation index at each sampling time and the preset resource recycling time window length, recursively construct the adaptive basis envelope, and compare the computing power oscillation index with the adaptive basis envelope to determine whether the network computing node has experienced a processing failure.

[0050] It should be noted that due to the chronic performance degradation of the basic operating environment of network computing nodes and the periodic resource reclamation oscillation of the operating system, the use of fixed judgment boundaries cannot adapt to long-term operation and maintenance scenarios, resulting in a large number of false alarms. Therefore, this invention uses a decay mechanism based on the reciprocal of the time window and the local prediction deviation standard deviation to recursively construct an adaptive basis envelope, and compares the computing power oscillation index with the adaptive basis envelope, so that the judgment boundary can self-adjust with the dynamic rhythm of the hardware basic environment and eliminate the risk of threshold iteration divergence.

[0051] Specifically, in the initial monitoring phase, the computing power oscillation index obtained at the first sampling time is assigned to the base smoothing center at the first sampling time to complete the initialization. Subsequently, based on multiple consecutive computing power oscillation indices prior to each sampling time, the base smoothing center at each sampling time is calculated.

[0052] in, Indicates the first The base smoothing center at each sampling time; Indicates the first The base smoothing center at each sampling time; Indicates the first The computing power fluctuation index at each sampling moment; This indicates the length of the artificially defined resource recycling time window.

[0053] Network computing nodes are affected by periodic mechanisms such as the operating system kernel's regular disk flushing and garbage collection, resulting in normal, time-dependent fluctuations in computing power. The smoothing evolution deviation of the current transient computing power oscillation index from the historical base smoothing center is extracted, representing the transient state shocks experienced by the system at the current sampling moment. To prevent the base envelope from erroneously tracking these high-frequency transient shocks, this invention introduces the resource reclamation time window length at the system's underlying layer. and take its reciprocal Constructed as an attenuation adjustment factor, due to Constrained by the physical recycling cycle of hardware and operating system, when the resource recycling time window length... The longer the time, the higher the natural decay rate. The smaller the value, the more the transformation step size from the current transient deviation to the base center is forced to be compressed, resulting in a smoother base center. The update trajectory then becomes smoother, thus absorbing the normal background noise fluctuations caused by the system's periodic resource reclamation mechanism at the underlying level. If the current computing power oscillation index... The slight drift near the smooth center indicates that the system is still within the adaptive background rhythm; the recursive mechanism ensures that even in the event of short-term sudden business, the base smooth center can be robustly deduced, effectively avoiding the risk of iterative divergence caused by the threshold being overly sensitive to abnormal values.

[0054] Furthermore, based on the basis smoothing center at each sampling time and the consecutive computing power oscillation indices prior to each sampling time, the adaptive basis envelope at each sampling time is calculated:

[0055] in, Indicates the first Adaptive basis envelope at each sampling time; Indicates the first The base smoothing center at each sampling time; Indicates the length of the artificially defined resource recycling time window; Indicates the first The first sampling time before the sampling time The computing power fluctuation index at each sampling moment; Indicates the first The first sampling time before the sampling time The basis smoothing center at each sampling time. This invention extracts the local prediction bias standard deviation of historical residual evolution and finds the basis smoothing center. Based on this, the envelope width is extended, and the resource recovery time window length is increased. The more drastic the fluctuations in actual values ​​deviating from the smoothing center within a defined historical time period, the more severe the changes. The larger the value, the better the adaptive basis envelope. The overall upward displacement and widening of the safety tolerance zone establish a dynamic defense line that fully accommodates the periodic fluctuations of the system, and effectively avoids numerical divergence during the threshold iteration process.

[0056] Resource recycling time window length The empirical value range is 3 to 10, with a preferred value of 5. This value is based on the fact that the default dirty page write-back wake-up interval for mainstream server operating systems is typically 5 seconds. To ensure the adaptive base envelope aligns with the system's underlying memory reclamation mechanism and absorbs the background noise from normal state fluctuations caused by periodic disk flushing, the resource reclamation time window length is adjusted at a sampling frequency of 1 Hz. In other embodiments, the implementer can determine the resource reclamation time window length by dividing the dirty page write-back configuration parameters of the target node's actual operating system by the data acquisition time interval. The setting is 5. Adaptive calculation and dynamic settings.

[0057] Furthermore, the computing power oscillation index at each sampling time is compared with the adaptive basis envelope at the same sampling time. If the computing power oscillation index is greater than the adaptive basis envelope, it is determined that the target node has experienced a network computing node processing failure at the corresponding sampling time; if the computing power oscillation index is not greater than the adaptive basis envelope, it is determined that the target node is in a normal and stable processing phase.

[0058] For example, Figure 4 This diagram illustrates the comparison between the computing power oscillation index and the adaptive base envelope, as well as the fault determination effect in this embodiment of the invention. When the system is in a normal, stable processing phase and during peak network concurrency periods, the adaptive base envelope smoothly follows the fluctuations in system noise floor, and the computing power oscillation index remains within the safety tolerance range of the adaptive base envelope. The system accurately determines that the target node is in a normal, stable processing phase. When a network computing node processing failure occurs, the computing power oscillation index exhibits an explosive surge and frequently exceeds the upper limit of the safety tolerance range of the adaptive base envelope. The system keenly detects the out-of-bounds characteristic and accurately determines that the target node has experienced a network computing node processing failure, effectively achieving the identification of underlying, hidden faults.

[0059] This invention also discloses a network fault detection system based on data analysis, including a processor and a memory. The memory stores computer program instructions, which, when executed by the processor, implement a network fault detection method based on data analysis according to the present invention.

[0060] The system also includes other components well known to those skilled in the art, such as communication buses and communication interfaces, the settings and functions of which are known in the art and will not be described in detail here.

Claims

1. A network fault detection method based on data analysis, characterized in that, include: Multiple operational status indicators of the target node are collected in real time, and each operational status indicator is normalized. A node status vector sequence is constructed based on the normalized operational status indicators. Obtain the difference between the node state vectors at adjacent time points in the node state vector sequence, and construct the first-order difference vector and the second-order difference vector in sequence, which are used as the state transition velocity vector and the state transition acceleration vector at each sampling time point, respectively. Construct a Gram determinant based on the state transition velocity vector and state transition acceleration vector at each sampling time, and obtain the trajectory torsion area corresponding to each sampling time. Based on the trajectory torsion area at each sampling time and the modal distribution of the state transition velocity vector at each sampling time, the computing power oscillation index corresponding to each sampling time is obtained. Based on the computing power oscillation index at each sampling time and the preset resource recycling time window length, an adaptive basis envelope is recursively constructed, and the computing power oscillation index is compared with the adaptive basis envelope to determine whether the network computing node has experienced a processing failure.

2. The network fault detection method based on data analysis according to claim 1, characterized in that, The sequential construction of first-order and second-order difference vectors, serving as the state transition velocity vector and state transition acceleration vector at each sampling time, includes: Subtract the node state vector at each sampling time from the node state vector at the previous sampling time by the corresponding dimension element to obtain the state transition velocity vector at each sampling time. The state transition velocity vector at each sampling time is subtracted from the state transition velocity vector at the previous sampling time by the corresponding dimension element to obtain the state transition acceleration vector at each sampling time.

3. The network fault detection method based on data analysis according to claim 1, characterized in that, The process of obtaining the trajectory torsion area corresponding to each sampling time includes: The sum of squares of the elements in each dimension of the state transition velocity vector is used as the velocity modulus feature; the sum of squares of the elements in each dimension of the state transition acceleration vector is used as the acceleration modulus feature. Calculate the sum of the products of the state transition velocity vector and the state transition acceleration vector in their corresponding dimensions, and then square the sum of these products to obtain the orthogonal cross features. Multiply the velocity modulus feature by the acceleration modulus feature, subtract the orthogonal cross feature, and then perform a square root operation to obtain the trajectory torsion area.

4. The network fault detection method based on data analysis according to claim 1, characterized in that, The process of obtaining the computing power oscillation index corresponding to each sampling time includes: The sum of the absolute values ​​of each dimension in the state transition velocity vector is calculated, and the sum of the absolute values ​​is divided by the square root of the number of running state indicators to obtain the normalized state offset amplitude corresponding to each sampling time. The normalized state offset amplitude is mapped to a nonlinear amplification factor using an exponential function with the natural constant as the base. The nonlinear amplification factor is then used to amplify the corresponding trajectory torsion area to obtain the computing power oscillation index corresponding to each sampling time.

5. The network fault detection method based on data analysis according to claim 1, characterized in that, The step of recursively constructing an adaptive basis envelope based on the computing power oscillation index at each sampling time and the preset resource recycling time window length includes: Based on the preset resource recycling time window length, an attenuation adjustment factor is constructed to smooth periodic noise fluctuations; the base smoothing center of the previous sampling time is obtained, and the smoothing evolution deviation between the computing power oscillation index of the current sampling time and the base smoothing center of the previous sampling time is calculated; the product of the attenuation adjustment factor and the smoothing evolution deviation is superimposed on the base smoothing center of the previous sampling time, and the base smoothing center of the current sampling time is obtained recursively. The adaptive basis envelope is determined based on the basis smoothing center at the current sampling time and the prediction residual between the computing power oscillation index and the corresponding basis smoothing center at each sampling time.

6. The network fault detection method based on data analysis according to claim 5, characterized in that, The determination of the adaptive basis envelope includes: Tracing back to a historical time period defined by a preset resource recycling time window, the prediction residual between the computing power oscillation index and the corresponding base smoothing center at each sampling moment within the historical time period is calculated; the standard deviation of the local prediction deviation is obtained based on the statistical dispersion of all the prediction residuals, which is used as the envelope expansion width; the envelope expansion width is superimposed on the base smoothing center at the current sampling moment to obtain an adaptive base envelope that includes the system's periodic fluctuation tolerance.

7. The network fault detection method based on data analysis according to claim 1, characterized in that, The step of comparing the computing power oscillation index with the adaptive basis envelope to determine whether a network computing node has experienced a processing failure includes: If the computing power oscillation index is greater than the adaptive base envelope, it is determined that the target node has experienced a processing failure at the corresponding sampling time; if the computing power oscillation index is not greater than the adaptive base envelope, it is determined that the target node is in a normal and stable processing phase.

8. The network fault detection method based on data analysis according to claim 1, characterized in that, The normalization process for each operational status indicator, and the construction of a node status vector sequence based on the normalized operational status indicators, includes: Obtain the upper and lower limits of the hardware physical limits corresponding to each operating status indicator; and normalize the maximum and minimum values ​​of each operating status indicator based on the upper and lower limits of the hardware physical limits.

9. A network fault detection method based on data analysis according to claim 8, characterized in that, The acquisition of the upper and lower limits of the hardware physical limits corresponding to each operating status indicator includes: For operational status indicators that do not have an absolute physical limit for hardware, the maximum empirical waiting time of the corresponding hardware under rated load is taken as the upper limit value of the physical limit for hardware.

10. A network fault detection system based on data analysis, characterized in that, include: A processor and a memory, wherein the memory stores computer program instructions that, when executed by the processor, implement a network fault detection method based on data analysis according to any one of claims 1-9.

Images