Computer performance degradation prediction and anomaly detection method based on multi-source data fusion

Abstract

The present application belongs to the technical field of computer performance detection, and discloses a computer performance degradation prediction and anomaly detection method based on multi-source data fusion, and the specific steps are as follows: S1. Multi-source data acquisition: collect hardware sensor data, software performance counter data, and user behavior and system log data of the computer. By fusing hardware, software and system log multi-source data, the one-sidedness of a single data source is avoided, the computer performance is accurately and comprehensively reflected, the detection accuracy is improved, and the false alarm and missed alarm rates are reduced; based on the historical data of the health state, a personalized model is trained to build a health region, which is adapted to different user habits, solves the drawbacks of fixed threshold one-size-fits-all, and enhances adaptability; the health degree trend is predicted through the LSTM model, the change from "post-detection" to "pre-prediction" is realized, and the user is helped to carry out preventive maintenance in advance; with the help of the SHAP method, the abnormal root cause is located, the fault diagnosis and repair time is shortened, and the operation and maintenance efficiency is improved.

Description

Technical Field

[0001] This invention belongs to the field of computer performance testing technology, specifically a method for predicting computer performance degradation and detecting anomalies based on multi-source data fusion. Background Technology

[0002] With the widespread use of computers and the gradual improvement of their performance, their reliability and long-term stability have become increasingly critical. Performance degradation is often the result of the combined effect of hardware aging and redundant and bloated software systems.

[0003] Existing detection methods mainly fall into the following categories: First, hardware diagnostic tools, such as monitoring real-time parameters like CPU temperature, fan speed, and voltage using software like HWMonitor, or executing hardware self-test programs at startup; these methods only reflect the instantaneous state of the hardware and are difficult to predict its long-term performance changes; second, operating system built-in tools, such as Windows Resource Monitor and Task Manager, which can view the real-time usage of system resources, but require active user intervention and cannot accurately distinguish between normal high load and abnormal states; third, rule-based threshold alarms, which set fixed thresholds for specific indicators, triggering an alarm when exceeded. While this method is simple to implement, it lacks flexibility, is difficult to adapt to different users' actual usage habits, and has a high false alarm rate. Therefore, existing methods still have limitations in long-term trend prediction, automatic state identification, and adaptability, and further optimization and improvement are necessary. Summary of the Invention

[0004] The purpose of this invention is to provide a method for predicting computer performance degradation and detecting anomalies based on multi-source data fusion, so as to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for predicting computer performance degradation and detecting anomalies based on multi-source data fusion, the specific steps of which are as follows: S1. Multi-source data acquisition: Acquire hardware sensor data, software performance counter data, and user behavior and system log data from the computer; S2. Data Preprocessing and Feature Engineering: The collected raw data is cleaned and aligned to time windows, and statistical features of the data within each time window are extracted to generate multidimensional feature vectors, which include hardware feature vectors, software feature vectors and log feature vectors. S3. Multi-source data fusion and health assessment: The hardware feature vector, software feature vector and log feature vector are concatenated to obtain a comprehensive feature vector and standardized. The historical data of the computer's health status is used to train a machine learning model, construct a healthy region, and calculate the health score of the data to be tested. S4. Performance degradation prediction and anomaly detection: Based on the time series of health scores, a dynamic threshold is set to perform real-time anomaly detection. The future health trend is predicted by a time series prediction model to achieve performance degradation early warning. S5. Results Output and Visualization: When an anomaly is detected or performance degradation is predicted, an alert is generated and an interpretable artificial intelligence method is used to locate the root cause of the anomaly, and the relevant information is displayed through a visual interface.

[0006] Preferably, the hardware sensor data in step S1 includes the utilization rate, temperature, and power consumption of each CPU core; memory occupancy rate and available memory; hard disk read / write speed, IO wait time, and SMART attributes; and the remaining capacity, fully charged capacity, and charge / discharge cycle count of the battery. The software performance counter data includes the system context switching frequency, interrupt request rate, and page fault rate. The user behavior and system log data includes the current foreground application identifier and error and warning information in the system event log.

[0007] Preferably, the data cleaning in step S2 includes processing missing values ​​and transient sensor glitches in the data. The data cleaning uses linear interpolation to supplement temporarily missing data points, as shown in the formula: In the formula, The data supplemented at time t For data at time t-1, The data is for time t+1; the data cleaning uses a moving average window to filter sensor glitches, and the formula is: In the formula, Let W be the filtered data at time t, and W be the window size. This is the data at time ti.

[0008] Preferably, the formula for generating the multidimensional feature vector in step S2 is: In the formula, For the multidimensional feature vector within the time window τ, This represents the mean of CPU-related data within the time window τ. The variance of CPU-related data within the time window τ. This represents the mean of temperature-related data within the time window τ. The mean of I / O related data within the time window τ. Let τ be the variance of the io-related data within the time window τ, and d be the total dimension of the feature.

[0009] Preferably, the machine learning model in step S3 is the SVDD model, which finds the smallest hypersphere with center a and radius R as the healthy region, and the optimization objective formula is: In the formula, R is the radius of the hypersphere, a is the center of the hypersphere, C is the penalty parameter, and M is the number of training samples in the healthy state. For the i-th training sample, As slack variables, This is a mapping function.

[0010] Preferably, the formula for calculating the health score in step S3 is: ,in, , Let be the feature vector of the data to be detected. for The distance to the center of the ball, , For Lagrange multipliers, This is the kernel function.

[0011] Preferably, the dynamic threshold in step S4 is set based on the moving average and moving standard deviation of historical health score data, using the following formula: In the formula, Let be the moving average at time t. Let be the moving standard deviation at time t, and L be the window length for calculating the moving average and moving standard deviation. This represents the health score at time ti.

[0012] Preferably, the time series prediction model in step S4 is an LSTM model, including a forgetting gate, an input gate, a cell state update, and an output gate. The final predicted value is calculated using the following formula: In the formula, Let be the predicted value at time t. This is the weight matrix. Let be the hidden state at time t. This is a bias term.

[0013] Preferably, the interpretable artificial intelligence method used in step S5 is the SHAP method, which is used to calculate the contribution of each feature to the health score, and the formula is: In the formula, Let f be the marginal contribution of the j-th feature, f be the trained model, x be the sample feature vector to be explained, S be the feature subset excluding feature j, and d be the total feature dimension. This is the output of the model when only features in subset S are applied.

[0014] The beneficial effects of this invention are as follows: By integrating multi-source data from hardware, software, and system logs, the system avoids the limitations of a single data source, comprehensively and accurately reflects computer performance, improves detection accuracy, and reduces false positives and false negatives. Personalized models are trained based on historical health status data to construct health zones, adapting to different user habits and overcoming the drawbacks of fixed thresholds, thus enhancing adaptability. LSTM models predict health trends, shifting from "post-event detection" to "pre-event prediction," helping users conduct preventative maintenance in advance. The SHAP method is used to locate the root cause of anomalies, shortening fault diagnosis and repair time, improving operational efficiency, and effectively overcoming the limitations of existing methods in trend prediction, status identification, and adaptability. Attached Figure Description

[0015] Figure 1 This is a flowchart of the present invention; Figure 2 This is a flowchart of the multi-source data fusion and health assessment process of this invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] like Figures 1 to 2 As shown in the figure, this invention provides a method for predicting computer performance degradation and detecting anomalies based on multi-source data fusion. The specific steps are as follows: S1. Multi-source data acquisition: Acquire hardware sensor data, software performance counter data, and user behavior and system log data from the computer; S2. Data Preprocessing and Feature Engineering: The collected raw data is cleaned and aligned to time windows, and statistical features of the data within each time window are extracted to generate multidimensional feature vectors, including hardware feature vectors, software feature vectors and log feature vectors. S3. Multi-source data fusion and health assessment: Hardware feature vectors, software feature vectors and log feature vectors are concatenated to obtain a comprehensive feature vector, which is then standardized. Historical data on the computer's health status is used to train a machine learning model, construct a healthy region, and calculate the health score of the data to be tested. S4. Performance degradation prediction and anomaly detection: Based on the time series of health scores, a dynamic threshold is set to perform real-time anomaly detection. The future health trend is predicted by a time series prediction model to achieve performance degradation early warning. S5. Results Output and Visualization: When an anomaly is detected or performance degradation is predicted, an alert is generated and an interpretable artificial intelligence method is used to locate the root cause of the anomaly, and the relevant information is displayed through a visual interface.

[0018] Comprehensive performance status perception is achieved through multi-source data collection, avoiding the one-sidedness of a single data source; data quality is improved through preprocessing and feature engineering, laying the foundation for subsequent analysis; multi-source fusion and personalized model construction adapt to different user habits, solving the problem of poor adaptability of fixed thresholds; combined with dynamic threshold detection and time series prediction, real-time anomaly monitoring and early warning of performance degradation are achieved, and the root cause can be located with interpretable methods, greatly improving operation and maintenance efficiency and effectively ensuring stable computer operation.

[0019] In step S1, the hardware sensor data includes the utilization rate, temperature, and power consumption of each CPU core; memory occupancy rate and available memory; hard drive read / write speed, IO wait time, and SMART attributes; and the remaining capacity, fully charged capacity, and charge / discharge cycle count of the battery. The software performance counter data includes the system context switching frequency, interrupt request rate, and page fault rate. The user behavior and system log data include the current foreground application identifier and error and warning information in the system event log.

[0020] On the hardware side, the system accurately captures the operating status of key components such as CPU, memory, and hard drive, paying particular attention to hardware aging-sensitive indicators such as hard drive I / O and SMART attributes. On the software side, the system focuses on core parameters of system resource scheduling, combining user behavior and system logs to achieve full-dimensional coverage of performance-influencing factors. At the same time, the system uses complementary and corroborating data from multiple sources to avoid the one-sidedness of single-dimensional data, providing comprehensive and accurate data support for subsequent fusion analysis, health assessment, and anomaly localization, thus ensuring the reliability of prediction and detection.

[0021] In step S2, data cleaning includes processing missing values ​​and transient sensor glitches in the data. Data cleaning uses linear interpolation to fill in temporarily missing data points, with the following formula: In the formula, The data supplemented at time t For data at time t-1, The data is from time t+1; data cleaning uses a moving average window to filter sensor glitches, the formula is: In the formula, Let W be the filtered data at time t, and W be the window size. This is the data at time ti.

[0022] Linear interpolation can accurately supplement temporarily missing data and ensure the continuity of time series; the moving average window can effectively filter abnormal fluctuations in the sensor; the two methods work together to improve the quality and reliability of the original data, avoid noise and missing values ​​from interfering with subsequent feature extraction and model training, and lay the foundation for accurately generating multi-dimensional feature vectors and ensuring the accuracy of subsequent health assessment and prediction detection.

[0023] The formula for generating the multidimensional feature vector in step S2 is as follows: In the formula, For the multidimensional feature vector within the time window τ, This represents the mean of CPU-related data within the time window τ. The variance of CPU-related data within the time window τ. This represents the mean of temperature-related data within the time window τ. The mean of I / O related data within the time window τ. Let τ be the variance of the io-related data within the time window τ, and d be the total dimension of the feature.

[0024] By extracting statistical features such as the mean and variance of hardware parameters within a time window to construct multi-dimensional feature vectors, discrete monitoring data is transformed into structured features. This not only preserves the operational patterns of key indicators such as CPU, temperature, and I / O, but also achieves the orderly integration of multi-dimensional features. The standardized feature vector format adapts to the input requirements of machine learning models, providing a high-quality and computable feature foundation for subsequent multi-source data fusion and health assessment, thereby improving the efficiency and accuracy of model analysis.

[0025] In step S3, the machine learning model is the SVDD model. This model finds the smallest hypersphere with center a and radius R as the healthy region, and the optimization objective formula is: In the formula, R is the radius of the hypersphere, a is the center of the hypersphere, C is the penalty parameter, and M is the number of training samples in the healthy state. For the i-th training sample, As slack variables, This is a mapping function.

[0026] The SVDD model is used to construct the minimum hypersphere health region. The high-dimensional feature vector is mapped to the feature space through a mapping function, which accurately defines the boundary range of the computer's healthy operation. Penalty parameters and relaxation variables are introduced to balance the model's generalization ability and anti-interference ability, and can effectively tolerate small fluctuations in healthy samples. The model trained based on historical health data is adapted to different user habits, providing accurate and personalized benchmarks for subsequent health calculation and anomaly detection.

[0027] The formula for calculating the health score in step S3 is as follows: ,in, , Let be the feature vector of the data to be detected. for The distance to the center of the ball, , For Lagrange multipliers, This is the kernel function.

[0028] This design is based on the SVDD hypersphere model to construct a health score calculation method. It avoids the problem of direct calculation in high-dimensional space by using kernel functions and simplifies the solution process by using Lagrange multipliers. It efficiently quantifies the degree of deviation between the data to be detected and the healthy region. The score value intuitively reflects the health status of computer performance. The higher the value, the better the health. It provides accurate, continuous and quantifiable judgment basis for subsequent dynamic threshold anomaly detection and performance degradation trend prediction.

[0029] In step S4, the dynamic threshold is set based on the moving average and moving standard deviation of historical health score data, using the following formula: In the formula, Let be the moving average at time t. Let be the moving standard deviation at time t, and L be the window length for calculating the moving average and moving standard deviation. This represents the health score at time ti.

[0030] This design sets dynamic thresholds based on the moving average and standard deviation of historical health scores, avoiding the drawbacks of traditional fixed thresholds and adaptively tracking changes in the health baseline. The thresholds are dynamically adjusted according to the computer's operating status and user habits, which reduces the false alarm rate under normal high-load scenarios and improves the sensitivity to slow performance degradation, providing accurate and flexible judgment criteria for real-time anomaly detection.

[0031] In step S4, the time series prediction model is an LSTM model, which includes a forgetting gate, an input gate, a cell state update, and an output gate. The final predicted value is calculated using the following formula: In the formula, Let be the predicted value at time t. This is the weight matrix. Let be the hidden state at time t. This is a bias term.

[0032] Using the LSTM model for health time series prediction, its gating structure can effectively capture the long-term temporal dependencies of health data and accurately uncover potential trends of performance degradation. At the same time, the model outputs predicted values ​​through hidden state updates and weight matrix operations, avoiding the shortcomings of traditional time series models in handling long series, and realizing the upgrade from real-time monitoring to early warning, providing users with accurate decision-making basis for preventive maintenance.

[0033] In step S5, the interpretable artificial intelligence method SHAP is used to calculate the contribution of each feature to the health score. The formula is as follows: In the formula, Let f be the marginal contribution of the j-th feature, f be the trained model, x be the sample feature vector to be explained, S be the feature subset excluding feature j, and d be the total feature dimension. This is the output of the model when only features in subset S are applied.

[0034] The SHAP method is used to quantify the marginal contribution of each feature to the health score, which can accurately locate the core factors affecting computer performance. Compared with traditional black-box detection models, this method gives the detection results interpretability, helping maintenance personnel to quickly pinpoint the root cause of anomalies and significantly shorten the troubleshooting and repair time. At the same time, the feature contribution data also provides a reliable basis for subsequent optimization of model feature selection and improvement of prediction accuracy.

[0035] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0036] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for predicting computer performance degradation and detecting anomalies based on multi-source data fusion, characterized in that, The specific steps are as follows: S1. Multi-source data acquisition: Acquire hardware sensor data, software performance counter data, and user behavior and system log data from the computer; S2. Data Preprocessing and Feature Engineering: The collected raw data is cleaned and aligned to time windows, and statistical features of the data within each time window are extracted to generate a multidimensional feature vector, which includes hardware feature vector, software feature vector and log feature vector. S3. Multi-source data fusion and health assessment: The hardware feature vector, software feature vector and log feature vector are concatenated to obtain a comprehensive feature vector and standardized. The historical data of the computer's health status is used to train a machine learning model, construct a healthy region, and calculate the health score of the data to be tested. S4. Performance degradation prediction and anomaly detection: Based on the time series of health scores, a dynamic threshold is set to perform real-time anomaly detection. The future health trend is predicted by a time series prediction model to achieve performance degradation early warning. S5. Results Output and Visualization: When an anomaly is detected or performance degradation is predicted, an alert is generated and an interpretable artificial intelligence method is used to locate the root cause of the anomaly, and the relevant information is displayed through a visual interface.

2. The method for predicting computer performance degradation and detecting anomalies based on multi-source data fusion according to claim 1, characterized in that: The hardware sensor data mentioned in step S1 includes the utilization rate, temperature, and power consumption of each CPU core, memory occupancy rate, available memory, hard disk read / write speed, IO wait time, SMART attributes, remaining battery capacity, fully charged capacity, and charge / discharge cycle count. The software performance counter data includes system context switching frequency, interruption request rate, and page error rate; the user behavior and system log data includes the current foreground application identifier and error and warning information in the system event log.

3. The method for predicting computer performance degradation and detecting anomalies based on multi-source data fusion according to claim 1, characterized in that: The data cleaning in step S2 includes processing missing values ​​and transient sensor glitches in the data. The data cleaning uses linear interpolation to fill in temporarily missing data points, as shown in the formula: In the formula, The data supplemented at time t For data at time t-1, The data is for time t+1; the data cleaning uses a moving average window to filter sensor glitches, and the formula is: In the formula, Let W be the filtered data at time t, and W be the window size. This is the data at time ti.

4. The method for predicting computer performance degradation and detecting anomalies based on multi-source data fusion according to claim 1, characterized in that: The formula for generating the multidimensional feature vector in step S2 is as follows: In the formula, For the multidimensional feature vector within the time window τ, This represents the mean of CPU-related data within the time window τ. The variance of CPU-related data within the time window τ. This represents the mean of temperature-related data within the time window τ. The mean of I / O related data within the time window τ. Let τ be the variance of the io-related data within the time window τ, and d be the total dimension of the feature.

5. The method for predicting computer performance degradation and detecting anomalies based on multi-source data fusion according to claim 1, characterized in that: The machine learning model mentioned in step S3 is the SVDD model. This model finds the smallest hypersphere with center 'a' and radius 'R' as the healthy region. The optimization objective formula is: In the formula, R is the radius of the hypersphere, a is the center of the hypersphere, C is the penalty parameter, and M is the number of training samples in the healthy state. For the i-th training sample, As slack variables, This is a mapping function.

6. The method for predicting computer performance degradation and detecting anomalies based on multi-source data fusion according to claim 1, characterized in that: The formula for calculating the health score in step S3 is as follows: ,in, , Let be the feature vector of the data to be detected. for The distance to the center of the ball, , For Lagrange multipliers, This is the kernel function.

7. The method for predicting computer performance degradation and detecting anomalies based on multi-source data fusion according to claim 1, characterized in that: The dynamic threshold mentioned in step S4 is set based on the moving average and moving standard deviation of historical health score data, and the formula is: In the formula, Let be the moving average at time t. Let be the moving standard deviation at time t, and L be the window length for calculating the moving average and moving standard deviation. This represents the health score at time ti.

8. The method for predicting computer performance degradation and detecting anomalies based on multi-source data fusion according to claim 1, characterized in that: The time series prediction model described in step S4 is an LSTM model, which includes a forgetting gate, an input gate, a cell state update, and an output gate. The final predicted value is calculated using the following formula: In the formula, Let be the predicted value at time t. This is the weight matrix. Let be the hidden state at time t. This is a bias term.

9. The method for predicting computer performance degradation and detecting anomalies based on multi-source data fusion according to claim 1, characterized in that: The interpretable artificial intelligence method used in step S5 is the SHAP method, which is used to calculate the contribution of each feature to the health score. The formula is as follows: In the formula, Let f be the marginal contribution of the j-th feature, f be the trained model, x be the sample feature vector to be explained, S be the feature subset excluding feature j, and d be the total feature dimension. This is the output of the model when only features in subset S are applied.

Images