A server fault diagnosis method and system for a server cluster

Abstract

The application discloses a kind of server fault diagnosis method and system for server group, it is related to server operation and maintenance technical field, present and propose the following scheme, including data acquisition unit, data preprocessing unit, data training unit, data diagnostic analysis unit, scheme generation unit, scheme verification trial unit, data storage unit and data early warning unit, each unit is connected by communication bus or network link and forms closed loop diagnosis system.The application constructs full-link fault diagnosis system, through multidimensional collection hardware, software, network data, improves data quality by scientific pretreatment, adopts rule matching+machine learning hybrid model, combined with self-learning mechanism, improve diagnostic accuracy and coverage, generate personalized repair scheme and be verified by simulation / real machine, avoid secondary failure, full-process data safety storage and double early warning mechanism, realize fault tracing and prevention and control in advance, greatly improve server group operation and maintenance efficiency, guarantee business continuity.

Description

Technical Field

[0001] This invention relates to the field of server operation and maintenance technology, and in particular to a server fault diagnosis method and system for server groups. Background Technology

[0002] With the rapid development of cloud computing and big data technologies, server clusters have become the core infrastructure of enterprise IT architecture. Their stable operation is directly related to business continuity. However, the expansion of server cluster size leads to an increase in failure points. Traditional fault diagnosis methods have the following shortcomings: First, data collection dimensions are limited, focusing only on hardware operating parameters and ignoring key data such as network links and software logs, resulting in a high rate of false alarms. Second, data is not preprocessed by the system, and redundant information interferes with the diagnostic results, reducing analysis efficiency. Third, diagnostic analysis lacks intelligent algorithm support and relies on human experience, making it unable to meet the real-time requirements of complex cluster faults. Fourth, fault repair solutions lack verification mechanisms, and direct deployment can easily lead to secondary faults. Fifth, there is a lack of end-to-end data storage and early warning linkage, making it difficult to achieve fault tracing and early prevention. Therefore, there is an urgent need for a full-chain fault diagnosis system that covers data acquisition, processing, diagnosis, solution generation, verification, storage, and early warning to address the shortcomings of existing technologies. Summary of the Invention

[0003] The present invention proposes a server fault diagnosis method and system for server groups, which solves the above-mentioned shortcomings of the prior art.

[0004] To achieve the above objectives, the present invention adopts the following technical solution: A server fault diagnosis system for server groups includes a data acquisition unit, a data preprocessing unit, a data training unit, a data diagnosis and analysis unit, a solution generation unit, a solution verification and trial operation unit, a data storage unit, and a data early warning unit. The units are connected through a communication bus or network link to form a closed-loop diagnosis system. The data acquisition unit is used to collect multi-dimensional operational data from each server in the server group and send the collected data information to the data preprocessing unit. The data preprocessing unit is communicatively connected to the data acquisition unit and is used to receive multi-dimensional operational data sent by the data acquisition unit and perform preprocessing. The preprocessing steps include data cleaning, data standardization, and data feature extraction. The data training unit, as the core support unit for the system's intelligent diagnosis, is used to build, train, and optimize fault diagnosis models, providing high-accuracy model support for the data diagnosis and analysis unit. The data diagnostic analysis unit is communicatively connected to the data preprocessing unit and is used to receive preprocessed feature data and data information trained by the data training unit. It uses a hybrid diagnostic model of rule matching and machine learning to diagnose faults. The scheme generation unit is communicatively connected to the data diagnosis and analysis unit, and generates a targeted fault repair scheme based on the fault diagnosis results of the data diagnosis and analysis unit.

[0005] Furthermore, the scheme verification and trial operation unit is communicatively connected to the scheme generation unit, and is used to receive the fault repair scheme sent by the scheme generation unit and conduct verification and trial operation; The data storage unit is communicatively connected to the data acquisition unit, data preprocessing unit, data diagnostic analysis unit, scheme generation unit, and scheme verification and trial operation unit, respectively, and is used to store the entire process data information, including: The data classification and storage module is divided into three sub-modules based on data type: raw data storage (stores raw data output by the data acquisition unit, retained for 180 days), preprocessing data storage (stores standardized data and feature data output by the data preprocessing unit, retained for 90 days), diagnostic result storage (stores fault diagnosis reports, permanently retained), and solution data storage (stores repair solutions and verification records, permanently retained). The data classification and storage module adopts a distributed storage architecture and supports data sharding storage. Disaster recovery and backup module: Enables off-site data backup and disaster recovery, including a scheduled backup submodule (daily incremental backup, weekly full backup), a real-time synchronization submodule (real-time synchronization of core data to the off-site backup center), and a disaster recovery submodule (supports data recovery by point in time). Hour, (minutes), the backup data of the disaster recovery backup module is stored using AES-256 encryption, and the formulas for disaster recovery backup related indicators are as follows: Recovery Time Objective (RTO): ; The longest allowed time from the occurrence of a disaster to the completion of data recovery; Recovery Point Objective (RPO): ; The longest permissible time window for data loss after a disaster; Data security management module: Ensures data storage security, including the access control submodule (based on the RBAC model, setting access permissions for different roles such as administrators, maintenance personnel, and auditors), the encrypted storage submodule (automatically encrypts critical data storage and decrypts it when reading), and the operation audit submodule (records all data access, modification, and deletion operations and retains audit logs for 1 year). Data retrieval and analysis module: Supports multi-dimensional retrieval of data throughout the entire process, including a conditional retrieval submodule (supports combined retrieval by time range, fault type, server number, data type, etc.), a full-text retrieval submodule (supports keyword retrieval, implemented based on Elasticsearch), and a statistical analysis submodule (supports statistical analysis of data such as fault type distribution, diagnostic accuracy, and solution pass rate, and generates visual reports). The data early warning unit is communicatively connected to the data acquisition unit and the data diagnostic analysis unit to realize a dual early warning mechanism. The data early warning unit supports early warning level settings (early warning levels are divided into: general early warning, important early warning and emergency early warning), and different levels correspond to different response mechanisms. The first level of early warning is real-time monitoring and early warning. The data early warning unit receives the raw data from the data acquisition unit and compares it with the preset safety threshold in real time. When the data exceeds the threshold, an early warning is immediately triggered (through audible and visual alarms, SMS notifications, pop-ups on the operation and maintenance platform, etc.). The second level of early warning is fault prediction and early warning. The data early warning unit receives the characteristic data trend analysis results from the data diagnosis and analysis unit. When it finds that the data has a fault evolution trend (such as the memory usage rate continuously increasing and the growth rate exceeding the preset threshold), it triggers an early warning and pushes potential fault types and prevention suggestions.

[0006] Furthermore, the data acquisition unit includes: Hardware data acquisition module: Collects hardware operating parameters, including CPU utilization, memory usage, hard disk read / write speed, power supply voltage, fan speed and hardware temperature, through the server's built-in sensors (CPU temperature sensor, memory pressure sensor, hard disk vibration sensor, etc.). The hardware data acquisition module supports sensor status self-test, and triggers an alarm and switches to the backup sensor when a sensor fails. Software data acquisition module: Collects software data through operating system API interfaces and application log parsers, including process running status, error codes, service start and stop records, thread blocking duration, and log exception keywords. The software data acquisition module supports multiple operating systems such as Windows, Linux, and Unix, and log parsing supports multiple formats such as JSON, XML, and text. Network data acquisition module: Deployed on the mirror ports of switches and routers, it collects network data through a packet capture tool (developed based on the Lippcap library), including inter-server communication latency, packet loss rate, bandwidth utilization, number of TCP connection establishment failures, and port open status. The network data acquisition module supports real-time traffic statistics and abnormal traffic marking. Acquisition control module: Used to coordinate the work of each acquisition module, configure the timed acquisition cycle (configurable from 1 to 30 seconds), set data threshold trigger conditions, realize the switching between timed acquisition and trigger-based acquisition, when a certain type of data exceeds the preset threshold, the acquisition control module is triggered to send an immediate acquisition command to the corresponding acquisition module, and the acquired data is marked as key data and transmitted to the data preprocessing unit with priority.

[0007] Furthermore, the data preprocessing unit includes: Data cleaning module: responsible for removing outliers, missing values ​​and duplicate data. The data cleaning module has built-in outlier detection submodule, missing value processing submodule and duplicate data removal submodule. The outlier detection submodule (based on the 3σ principle algorithm) is used to mark and delete data that exceeds the mean ± 3 times the standard deviation. The outlier detection submodule is based on the 3σ principle, and the formula is as follows: These are abnormal values ​​and will be deleted. in, For a single data sample, The mean of the dataset. The standard deviation of the dataset; Calculation steps: 1. Calculate the mean of the dataset: (n is the total number of data samples); 2. Calculate the standard deviation of the dataset: ; 3. If a data sample meets the formula conditions, it is identified as an outlier and deleted; The missing value processing submodule uses linear interpolation to complete data with a missing value rate below 5%, and discards data with a missing value rate above 5%. The formula for using linear interpolation to complete the data is as follows: ; in, For missing positions The interpolation results, , These are the known data adjacent to the missing position, respectively. , , , For time / series indexing of data; The duplicate data removal submodule deletes completely duplicate data records by comparing data hash values. The data standardization module employs the Z-score standardization method to transform data from hardware, software, network, and environmental sources with different dimensions into standardized data with a mean of 0 and a variance of 1. This module includes a built-in dimension recognition submodule for automatically identifying data types and dimensions, adapting to standardization calculations for different data types, and supporting customizable standardization parameters to meet preprocessing needs in special scenarios. The Z-score method is used in the data standardization module, as shown in the following formula: ; in, For standardized data, The original data, The mean of the original dataset. The standard deviation of the original dataset; Function: Converts hardware, software, and network data of different dimensions into standardized data with a mean of 0 and a variance of 1, eliminating interference from different dimensions; Feature extraction module: Based on principal component analysis (PCA), a feature extraction model is built to extract core feature vectors from standardized data. The feature extraction module is based on the PCA algorithm, and the first step is to calculate the covariance matrix of the standardized data. ; in, for Covariance matrix ( (for the original data dimensions) for 3D standardized data matrix ( (number of samples) for The transpose of the matrix; The feature extraction module includes: The detailed feature calculation submodule is used to calculate fault correlation, data fluctuation coefficient, and feature parameter coupling degree. The formula for calculating fault correlation (Pearson correlation coefficient) is as follows: ; in, , These are two related operational data metrics, such as CPU utilization and hard drive read / write speed; The formula for calculating the data volatility coefficient is as follows: ; in, The standard deviation of the indicator data. The mean of the indicator data; The formula for calculating the coupling degree of the characteristic parameter is as follows: ; in, , There are two feature parameters. Let the covariance of the two be , , For their respective variances; The dimensionality reduction submodule is used to remove redundant features and retain principal components with a contribution rate ≥ 85%. The contribution rate formula is as follows: ; in, For the first The contribution rate of each principal component Covariance matrix The 1 eigenvalue (sorted from largest to smallest) It is the sum of all eigenvalues; Screening rules: Principal components that meet the criteria (cumulative contribution rate or individual principal component contribution rate; in this application, the cumulative contribution rate is sufficient) are retained, while redundant features are removed. The core feature vector is calculated as follows: ; in, for 3D core feature vector matrix ( The dimensions of the principal components after screening. ), for 3D eigenvector matrix (composed of covariance matrix) The former (composed of the eigenvectors corresponding to the largest eigenvalues); The feature vector output submodule is used to format the extracted feature vectors and transmit them to the data diagnostic analysis unit.

[0008] Furthermore, the data training unit includes: Training data acquisition module: Communicates with the data preprocessing unit and the data storage unit respectively to acquire the data required for training, including real-time feature data submodule, historical sample retrieval submodule and sample screening submodule; The real-time feature data submodule is used to receive the latest feature data output by the data preprocessing unit and filter the data marked as "fault confirmed" as real-time training samples. The historical sample retrieval submodule is used to retrieve historical fault samples from the diagnostic result storage submodule and the solution data storage submodule of the data storage unit, including complete information such as feature data, fault type, diagnostic results and repair verification results; The sample screening submodule adopts a "fault type balance + data quality screening" strategy to remove invalid samples (such as samples marked as "invalid diagnosis" or "unclear reason for repair failure"), ensuring the balance and effectiveness of the training sample set, with a sample effectiveness rate of ≥90%. Sample labeling optimization module: Standardizes and optimizes the acquired training samples, including automatic labeling submodule, manual review submodule and sample augmentation submodule; The automatic labeling submodule automatically labels the samples with "fault type label", "fault level label" and "feature association label" based on the fault confirmation results of the data diagnosis and analysis unit and the repair verification results of the scheme verification and trial operation unit. The manual review submodule is used to perform sampling review on automatically labeled samples. The sampling ratio is configurable (default 10%), supports manual correction of labeling errors, and ensures labeling accuracy ≥ 99%. The sample enhancement submodule uses data augmentation algorithms (such as feature perturbation and time series interpolation) to enhance samples of scarce fault types, avoiding model overfitting. After enhancement, the difference in the number of samples of various fault types is ≤5 times. Model building and training module: Constructs a diagnostic model training framework that integrates multiple algorithms, including a basic model selection submodule, a model parameter configuration submodule, an offline training submodule, and an incremental training submodule; The basic model selection submodule supports multiple algorithms such as random forest, XGBoost, LightGBM, and deep learning (CNN-LSTM). The core algorithm is switched according to the characteristics of the fault type, and the default algorithm is the fusion algorithm of random forest + XGBoost. The model parameter configuration submodule has built-in default optimal parameter sets for different algorithms, and supports manual adjustment of parameters (such as decision tree depth, learning rate, regularization coefficient, etc.) and parameter grid search optimization; The offline training submodule performs offline batch training based on historical sample sets to generate a basic diagnostic model. During the training process, it outputs multiple evaluation indicators such as model accuracy, recall, and F1 score in real time. The incremental training submodule is used to receive real-time training samples and use incremental learning algorithms (such as online gradient descent) to iteratively update the basic model, avoiding the resource consumption caused by full retraining. The incremental training cycle is configurable (default 1 hour / time). Model Evaluation and Optimization Module: This module performs multi-dimensional evaluation and optimization of the trained model. It includes sub-modules for calculating evaluation metrics, model selection, and model pruning. The core formulas used in this module are as follows: 1. Accuracy: ; in, (True positive): The fault was correctly diagnosed as a fault; (True negative): A normal state is correctly diagnosed as normal; (False positive): A normal state is mistakenly identified as a malfunction; (False negative): The fault was mistakenly identified as normal; 2. Recall: ; Function: Measures the degree to which the model misses faults; the higher the value, the fewer faults are missed.

[0009] 3. Precision: ; Function: Measures the accuracy of the model's diagnostic results; a higher accuracy indicates fewer false positives. 4. F1 score (comprehensive evaluation index): ; Function: To balance precision and recall and comprehensively reflect the diagnostic performance of the model, this application uses an F1 score improvement of ≥5% as the model update standard; The evaluation index calculation submodule calculates the model's core indicators, including accuracy, recall, precision, F1 score, and confusion matrix, and outputs evaluation results for different fault types. When the evaluation metric (such as the overall F1 score) of the newly trained model is improved by ≥5% compared to the previous version, the model selection submodule automatically marks it as the "optimal model". If the improvement is less than 3%, parameter optimization or algorithm adjustment will be triggered. The model pruning submodule is used to prune and optimize complex models (such as deep learning models and multi-decision tree fusion models), remove redundant nodes, reduce model complexity, and ensure that the model inference latency is ≤100ms to meet the real-time diagnostic requirements. Model Deployment and Update Module: Responsible for deploying and updating the trained and optimized model, including the model encapsulation submodule, the model push submodule, and the version management submodule; The model encapsulation submodule is used to encapsulate the optimal model into a standard interface to adapt to the model calling requirements of the data diagnostic analysis unit. The model push submodule automatically pushes the newly packaged model to the machine learning diagnostic module of the data diagnostic analysis unit, supporting "hot deployment" without interrupting system operation; The version management submodule is used to record model version information (including training time, number of samples, evaluation metrics, and parameter configuration), support model version rollback (when the diagnostic accuracy drops by ≥3% after the deployment of a new model, it will automatically trigger a rollback to the previous best version), and retain the 10 most recent model versions.

[0010] Furthermore, the data diagnostic analysis unit includes: Fault rule base module: Stores feature rules for known fault types, including rule storage submodule and rule management submodule; The rule storage submodule adopts a relational database + cache architecture, which supports fast rule query; The rule management submodule supports adding, modifying, deleting, and enabling / disabling rules. The rule format is defined as "feature condition + fault type + confidence level", for example, "CPU usage ≥ 95% for 10 minutes → CPU overload fault → confidence level 99%"; Rule matching and diagnosis module: The preprocessed feature data is matched one by one with the rules in the fault rule base. A multi-threaded parallel matching mechanism is used to improve the matching efficiency. The matching results are divided into three categories: "complete match", "partial match" and "no match". Complete match directly outputs the fault diagnosis result, partial match is marked as a suspected fault, and no match enters the machine learning diagnosis process. Machine Learning Diagnostic Module: A diagnostic model is built based on the random forest algorithm, including a model training submodule, a model inference submodule, and a model optimization submodule; The model training submodule uses historical fault data for offline training and supports incremental training. The model reasoning submodule takes suspected fault data or no matching data as input, calculates the similarity with various fault types, and outputs the fault probability ranking and fault cause analysis. The model optimization submodule adjusts the model parameters based on the diagnostic results to improve diagnostic accuracy. The self-learning update module automatically generates new rules and adds them to the fault rule base based on the new fault types and feature data output by the machine learning diagnostic module. At the same time, it adds new fault data to the model training sample set, triggering incremental model training. It supports a manual review mechanism, and new rules and samples must be reviewed and approved before taking effect to avoid accidental updates.

[0011] Furthermore, the scheme generation unit includes: Repair Solution Knowledge Base Module: Used to store standard repair strategies corresponding to different fault types, including solution storage submodule and solution update submodule; The solution storage submodule adopts structured storage and supports retrieval by fault type, server type, and business scenario; The solution update submodule supports manual input of new solutions and automatic synchronization of the latest industry repair strategies. The standard repair strategy includes fields such as operation steps, required tools, precautions, and expected results. Fault Information Parsing Module: Used to parse diagnostic analysis results and extract key information, including fault type, fault level (fatal / critical / general / minor), affected servers (core / normal), affected services (core services / non-core services), and fault duration, providing data support for solution adaptation; Solution adaptation and adjustment module: Based on the fault information analysis results, the standard repair strategy is customized. For example, when the core server fails, pre-requisite steps such as "business migration" and "backup server startup" are added. When core business processes fail, shorten the execution cycle of the repair plan and increase its priority; The adjustment rules include "fault level → solution complexity", "server role → operation priority", and "business type → risk control measures". Solution Output Optimization Module: The adapted and adjusted solution is formatted and output, which clarifies the order of operation steps, execution subject (automatic execution / manual execution), estimated time, risk level (high / medium / low) and emergency rollback solution. It supports exporting the solution as PDF, Word or script format that can be directly executed by the operation and maintenance platform.

[0012] Furthermore, the scheme verification and trial operation unit includes: Simulation environment building module: Based on virtualization technologies (such as KVM, Docker), it builds a simulated running environment, including an environment replication submodule, an environment configuration submodule, and an environment destruction submodule; The environment replication submodule is used to accurately replicate the hardware configuration, software version, network topology, and data status of the faulty server; The environment configuration submodule is used to support the adjustment of environment parameters and simulate different load scenarios; The environment destruction submodule is used to automatically destroy the environment and release resources after verification, and to physically isolate the simulation environment from the production environment to avoid mutual interference. Simulation verification execution module: Automatically executes the repair plan in a simulation environment, including a step execution submodule, a process monitoring submodule, and a result judgment submodule; The step execution submodule is executed sequentially according to the steps in the scheme, and supports pause, continue, and terminate operations. The process monitoring submodule is used to monitor system resource usage, service availability, data consistency, and log output metrics in real time during the execution process. The result determination submodule is used to compare the fault indicators before and after execution to determine whether the fault has been repaired or whether a secondary fault has occurred. Real device verification adaptation module: For core servers or high-risk faults, it supports small-scale real device trial operation, including backup server screening submodule, real device environment isolation submodule and verification scope control submodule; The backup server filtering submodule is used to filter suitable backup servers based on server configuration consistency and load conditions. The real device environment isolation submodule ensures that real device verification does not affect production operations through network isolation, data isolation, and other methods. The verification scope control submodule limits the scope of verification impact, covering only fault-related functions. Solution Iteration and Optimization Module: If simulation verification or real machine verification fails, analyze the reasons for failure (such as missing steps, incorrect parameters, or environment adaptation issues), and report the problem to the solution adaptation and adjustment module of the solution generation unit to trigger solution iteration. It supports multiple rounds of iterative verification until the solution verification is successful.

[0013] Furthermore, the data early warning unit includes: Early warning threshold configuration module: used to set security thresholds for various types of data, including real-time monitoring threshold submodule (configuring normal range thresholds for hardware, software, network, and environmental data) and trend early warning threshold submodule (configuring data change trend thresholds, such as memory usage rate increasing by ≥10% within 5 minutes). The early warning threshold configuration module supports custom configuration of thresholds according to server type and business scenario, and supports batch import / export of thresholds. Real-time monitoring and early warning module: used to receive raw data from the data acquisition unit and compare it with the real-time monitoring threshold in real time. The real-time monitoring and early warning module includes a data comparison submodule (using a millisecond-level comparison frequency to ensure timely early warning), an early warning triggering submodule (to immediately trigger an early warning when the data exceeds the threshold), and an early warning notification submodule (to push early warning information through sound and light alarms, SMS, email, operation and maintenance platform pop-ups, enterprise WeChat / DingTalk robots, etc.). Fault prediction and early warning module: Used to receive the characteristic data trend analysis results of the data diagnosis and analysis unit, including trend analysis submodule (based on time series analysis algorithm to predict data change trend), prediction decision submodule (when the trend meets preset conditions, trigger prediction and early warning), and early warning suggestion generation submodule (pushing suggestions such as potential fault types, preventive measures, priorities, etc.). The early warning management module is used to manage early warning information throughout its entire lifecycle. It includes an early warning classification submodule (which divides early warnings into three levels: general early warning, important early warning, and emergency early warning, corresponding to different response time limits), an early warning processing submodule (which supports the confirmation, ignoring, processing, and closing of early warnings, and records the processing process), and an early warning statistics submodule (which collects data such as the number of early warning triggers, processing rate, and average processing time, for optimizing threshold configuration).

[0014] A server fault diagnosis method for a server group, applicable to any of the server fault diagnosis systems for server groups described above, includes the following steps: S1: Data acquisition. The hardware data acquisition module, software data acquisition module, and network data acquisition module of the data acquisition unit acquire data of the corresponding dimensions respectively. The acquisition control module coordinates the acquisition mode and generates key data markers. S2: Data preprocessing, the data cleaning module of the data preprocessing unit removes abnormal data, the data standardization module unifies the data units, and the feature extraction module extracts the core feature vectors; S3: Data diagnostic analysis. The rule matching diagnostic module of the data diagnostic analysis unit matches the fault rule library, the machine learning diagnostic module processes unmatched data, and the self-learning update module optimizes the rule library and model. S4: Solution generation. The fault information parsing module of the solution generation unit extracts key fault information, the solution adaptation and adjustment module optimizes the standard repair solution, and the solution output optimization module formats and outputs the repair solution. S5: Solution verification and trial operation. The simulation environment construction module of the solution verification and trial operation unit builds a replica environment, the simulation verification execution module executes the solution and monitors it, the real machine verification and adaptation module completes real machine verification of high-risk faults, and the solution iteration and optimization module realizes closed-loop optimization of the solution. S6: Data storage. The data storage unit's data classification storage module stores the entire process data by type, the disaster recovery backup module realizes data backup, the data security management module ensures data security, and the data retrieval and analysis module supports data query and analysis. S7: Data early warning. The real-time monitoring and early warning module of the data early warning unit triggers real-time early warnings, the fault prediction and early warning module pushes prediction information, and the early warning management module realizes full lifecycle management of early warnings.

[0015] Compared with existing technologies, the beneficial effects of this invention are: 1. This invention solves the problem of "data one-sidedness" through a multi-dimensional data acquisition system, breaking through the limitations of traditional acquisition that only focuses on hardware parameters. It constructs a three-dimensional acquisition architecture of "hardware + software + network", covering hardware indicators such as CPU utilization and memory usage, software data such as process status and error codes, and network parameters such as communication latency and data packet loss rate. This enables comprehensive capture of fault-related data, reducing the false judgment rate from the source. It also supports switching between timed acquisition and triggered acquisition, automatically marking key points for abnormal data and prioritizing transmission, balancing acquisition efficiency and data relevance, and adapting to dynamic operation scenarios of server groups. 2. This invention improves the quality of basic diagnosis through a systematic data preprocessing mechanism. Based on scientific algorithms such as the 3σ principle and linear interpolation, it achieves accurate processing of outliers, missing values, and duplicate data, ensuring data validity. Z-score standardization eliminates interference from different dimensions, solving the problem of multi-dimensional data fusion analysis. Principal component analysis (PCA) is used to extract core features, retaining principal components with a contribution rate of ≥85% and eliminating redundant information. This reduces the computational complexity of subsequent diagnosis, ensures that key fault features are not lost, and improves analysis efficiency. 3. This invention achieves the dual goals of "accuracy + efficiency" through a hybrid intelligent diagnostic model. It adopts a hybrid mode of "rule matching + machine learning". Known faults are quickly matched and output through the rule base, while unknown or suspected faults are deeply analyzed through fusion algorithms such as random forest and XGBoost. It balances diagnostic speed and coverage, and builds a self-learning update mechanism. New fault data is automatically added to the rule base and training sample set, triggering incremental training of the model. It can continuously improve diagnostic capabilities without manual intervention and adapt to the needs of dynamic changes in the fault types of server groups. In summary, this system constructs a full-link fault diagnosis system. It collects hardware, software, and network data from multiple dimensions, improves data quality through scientific preprocessing, adopts a rule-matching + machine learning hybrid model, and combines a self-learning mechanism to improve diagnostic accuracy and coverage. It generates personalized repair solutions and verifies them through simulation / real machines to avoid secondary faults. The system also features secure data storage throughout the entire process and a dual early warning mechanism to achieve fault tracing and early prevention, significantly improving the operational efficiency of server groups and ensuring business continuity. Attached Figure Description

[0016] Figure 1 This is a system block diagram of a server fault diagnosis system for a server group proposed in this invention. Figure 2 This is a block diagram of the data acquisition unit of a server fault diagnosis method and system for server groups proposed in this invention; Figure 3 This is a block diagram of the data preprocessing unit of a server fault diagnosis method and system for server groups proposed in this invention; Figure 4 This is a block diagram of the data training unit of a server fault diagnosis method and system for server groups proposed in this invention; Figure 5 This is a block diagram of the data diagnosis unit of a server fault diagnosis method and system for server groups proposed in this invention; Figure 6This is a block diagram of the scheme generation unit of a server fault diagnosis method and system for server groups proposed in this invention. Figure 7 This is a block diagram of the scheme verification and trial operation unit of a server fault diagnosis method and system for server groups proposed in this invention. Figure 8 This is a block diagram of the data storage unit of a server fault diagnosis method and system for server groups proposed in this invention. Figure 9 This is a block diagram of the data early warning unit of a server fault diagnosis method and system for server groups proposed in this invention; Figure 10 This is a flowchart of the method steps for a server fault diagnosis method and system for server groups proposed in this invention. Detailed Implementation

[0017] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0018] In the description of this invention, it should be understood that the terms "upper", "lower", "front", "rear", "left", "right", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0019] Example 1 Reference Figure 1-9 A server fault diagnosis system for server groups includes a data acquisition unit, a data preprocessing unit, a data training unit, a data diagnosis and analysis unit, a solution generation unit, a solution verification and trial operation unit, a data storage unit, and a data early warning unit. The units are connected through a communication bus or network link to form a closed-loop diagnosis system. The data acquisition unit is used to collect multi-dimensional operational data from each server in the server group and send the collected data information to the data preprocessing unit; The data preprocessing unit is connected to the data acquisition unit and is used to receive multi-dimensional operational data sent by the data acquisition unit and perform preprocessing. The preprocessing steps include data cleaning, data standardization and data feature extraction. The data training unit, as the core support unit for intelligent system diagnosis, is used to build, train, and optimize fault diagnosis models, providing high-accuracy model support for the data diagnosis and analysis unit. The data diagnostic analysis unit communicates with the data preprocessing unit to receive preprocessed feature data and data information trained by the data training unit, and uses a hybrid diagnostic model of rule matching + machine learning for fault diagnosis. The solution generation unit communicates with the data diagnosis and analysis unit and generates targeted fault repair solutions based on the fault diagnosis results from the data diagnosis and analysis unit.

[0020] In this invention, the scheme verification and trial operation unit is communicatively connected to the scheme generation unit, and is used to receive the fault repair scheme sent by the scheme generation unit and perform verification and trial operation; The data storage unit is communicatively connected to the data acquisition unit, data preprocessing unit, data diagnostic analysis unit, scheme generation unit, and scheme verification and trial operation unit, respectively, and is used to store data information for the entire process, including: The data classification and storage module is divided into three sub-modules based on data type: raw data storage (stores raw data output by the data acquisition unit, retained for 180 days), preprocessing data storage (stores standardized data and feature data output by the data preprocessing unit, retained for 90 days), diagnostic results storage (stores fault diagnosis reports, permanently retained), and solution data storage (stores repair solutions and verification records, permanently retained). The data classification and storage module adopts a distributed storage architecture and supports data sharding storage. Disaster recovery and backup module: Enables off-site data backup and disaster recovery, including a scheduled backup submodule (daily incremental backup, weekly full backup), a real-time synchronization submodule (real-time synchronization of core data to the off-site backup center), and a disaster recovery submodule (supports data recovery by point in time). Hour, (minutes), the disaster recovery backup module uses AES-256 encrypted storage for backup data, and the formulas for relevant disaster recovery backup indicators are as follows: Recovery Time Objective (RTO): ; The longest allowed time from the occurrence of a disaster to the completion of data recovery; Recovery Point Objective (RPO): ; The longest permissible time window for data loss after a disaster; Data security management module: Ensures data storage security, including the access control submodule (based on the RBAC model, setting access permissions for different roles such as administrators, maintenance personnel, and auditors), the encrypted storage submodule (automatically encrypts critical data storage and decrypts it when reading), and the operation audit submodule (records all data access, modification, and deletion operations and retains audit logs for 1 year). Data retrieval and analysis module: Supports multi-dimensional retrieval of data throughout the entire process, including a conditional retrieval submodule (supports combined retrieval by time range, fault type, server number, data type, etc.), a full-text retrieval submodule (supports keyword retrieval, implemented based on Elasticsearch), and a statistical analysis submodule (supports statistical analysis of data such as fault type distribution, diagnostic accuracy, and solution pass rate, and generates visual reports). The data early warning unit is connected to the data acquisition unit and the data diagnosis and analysis unit to realize a dual early warning mechanism. The data early warning unit supports the setting of early warning levels (early warning levels are divided into: general early warning, important early warning and emergency early warning), and different levels correspond to different response mechanisms. The first level of early warning is real-time monitoring and early warning. The data early warning unit receives the raw data from the data acquisition unit and compares it with the preset safety threshold in real time. When the data exceeds the threshold, an early warning is immediately triggered (through audible and visual alarms, SMS notifications, pop-ups on the operation and maintenance platform, etc.). The second level of early warning is fault prediction and early warning. The data early warning unit receives the characteristic data trend analysis results from the data diagnosis and analysis unit. When it finds that the data has a fault evolution trend (such as the memory usage rate continuously increasing and the growth rate exceeding the preset threshold), it triggers an early warning and pushes potential fault types and prevention suggestions.

[0021] In this invention, the data acquisition unit includes: Hardware data acquisition module: Collects hardware operating parameters, including CPU utilization, memory usage, hard disk read / write speed, power supply voltage, fan speed and hardware temperature, through the server's built-in sensors (CPU temperature sensor, memory pressure sensor, hard disk vibration sensor, etc.). The hardware data acquisition module supports sensor status self-test, triggers an alarm and switches to the backup sensor when a sensor fails. Software data acquisition module: Collects software data through operating system API interfaces and application log parsers, including process running status, error codes, service start and stop records, thread blocking duration, and log exception keywords. The software data acquisition module supports multiple operating systems such as Windows, Linux, and Unix, and log parsing supports multiple formats such as JSON, XML, and text. Network data acquisition module: Deployed on the mirror ports of switches and routers, it collects network data through a packet capture tool (developed based on the Lippcap library), including inter-server communication latency, packet loss rate, bandwidth utilization, number of TCP connection establishment failures, and port open status. The network data acquisition module supports real-time traffic statistics and abnormal traffic marking. Acquisition control module: Used to coordinate the work of each acquisition module, configure the timed acquisition cycle (configurable from 1 to 30 seconds), set data threshold trigger conditions, realize the switching between timed acquisition and trigger-based acquisition, when a certain type of data exceeds the preset threshold, the acquisition control module is triggered to send an immediate acquisition command to the corresponding acquisition module, and the acquired data is marked as key data and transmitted to the data preprocessing unit with priority.

[0022] In this invention, the data preprocessing unit includes: Data cleaning module: responsible for removing outliers, missing values ​​and duplicate data. The data cleaning module has built-in outlier detection submodule, missing value handling submodule and duplicate data removal submodule; The outlier detection submodule (based on the 3σ principle algorithm) is used to mark and delete data that exceeds the mean ± 3 times the standard deviation as outliers. The outlier detection submodule is based on the 3σ principle, and the formula is as follows: These are abnormal values ​​and will be deleted. in, For a single data sample, The mean of the dataset. The standard deviation of the dataset; Calculation steps: 1. Calculate the mean of the dataset: (n is the total number of data samples); 2. Calculate the standard deviation of the dataset: ; 3. If a data sample meets the formula conditions, it is identified as an outlier and deleted; The missing value handling submodule uses linear interpolation to complete data with a missing value rate below 5%, and discards data with a missing value rate above 5%. The formula for using linear interpolation to complete the data is as follows: ; in, For missing positions The interpolation results, , These are the known data adjacent to the missing position, respectively. , , , For time / series indexing of data; The duplicate data removal submodule deletes completely duplicate data records by comparing data hash values; The data standardization module employs the Z-score standardization method to transform data from hardware, software, network, and environmental sources with different dimensions into standardized data with a mean of 0 and a variance of 1. This module includes a built-in dimension recognition submodule to automatically identify data types and dimensions, adapting to standardization calculations for different data types. It supports custom configuration of standardization parameters to meet preprocessing needs in special scenarios. The Z-score method is used in the data standardization module, as shown in the following formula: ; in, For standardized data, The original data, The mean of the original dataset. The standard deviation of the original dataset; Function: Converts hardware, software, and network data of different dimensions into standardized data with a mean of 0 and a variance of 1, eliminating interference from different dimensions; Feature extraction module: Based on principal component analysis (PCA), a feature extraction model is built to extract core feature vectors from standardized data. The feature extraction module is based on the PCA algorithm, and the first step is to calculate the covariance matrix of the standardized data. ; in, for Covariance matrix ( (for the original data dimensions) for 3D standardized data matrix ( (number of samples) for The transpose of the matrix; The feature extraction module includes: The detailed feature calculation submodule is used to calculate fault correlation, data fluctuation coefficient, and feature parameter coupling degree. The formula for calculating fault correlation (Pearson correlation coefficient) is as follows: ; in, , These are two related operational data metrics, such as CPU utilization and hard drive read / write speed; The formula for calculating the data volatility coefficient is as follows: ; in, The standard deviation of the indicator data. The mean of the indicator data; The formula for calculating the coupling degree of the characteristic parameter is as follows: ; in, , There are two feature parameters. Let the covariance of the two be , , For their respective variances; The dimensionality reduction submodule is used to remove redundant features and retain principal components with a contribution rate ≥ 85%. The contribution rate formula is as follows: ; in, For the first The contribution rate of each principal component Covariance matrix The 1 eigenvalue (sorted from largest to smallest) It is the sum of all eigenvalues; Screening rules: Principal components that meet the criteria (cumulative contribution rate or individual principal component contribution rate; in this application, the cumulative contribution rate is sufficient) are retained, while redundant features are removed. The core feature vector is calculated as follows: ; in, for 3D core feature vector matrix ( The dimensions of the principal components after screening. ), for 3D eigenvector matrix (composed of covariance matrix) The former (composed of the eigenvectors corresponding to the largest eigenvalues); The feature vector output submodule is used to format the extracted feature vectors and transmit them to the data diagnostic analysis unit.

[0023] In this invention, the data training unit includes: Training data acquisition module: Communicates with the data preprocessing unit and the data storage unit respectively to acquire the data required for training, including real-time feature data submodule, historical sample retrieval submodule and sample screening submodule; The real-time feature data submodule is used to receive the latest feature data output by the data preprocessing unit and filter the data marked as "fault confirmed" as real-time training samples; The historical sample retrieval submodule is used to retrieve historical fault samples from the diagnostic result storage submodule and the solution data storage submodule of the data storage unit, including complete information such as characteristic data, fault type, diagnostic results and repair verification results; The sample selection submodule adopts a "fault type balance + data quality selection" strategy to remove invalid samples (such as samples marked as "invalid diagnosis" or "repair failure with unclear cause"), ensuring the balance and effectiveness of the training sample set, with a sample effectiveness rate of ≥90%. Sample labeling optimization module: Standardizes and optimizes the acquired training samples, including automatic labeling submodule, manual review submodule and sample augmentation submodule; The automatic labeling submodule automatically labels samples with "fault type label", "fault level label" and "feature association label" based on the fault confirmation results of the data diagnosis and analysis unit and the repair verification results of the scheme verification and trial operation unit. The manual review submodule is used to perform sampling review on automatically labeled samples. The sampling ratio is configurable (default 10%), and manual correction of labeling errors is supported to ensure labeling accuracy ≥ 99%. The sample augmentation submodule uses data augmentation algorithms (such as feature perturbation and time series interpolation) to augment samples of scarce fault types, avoiding model overfitting. After augmentation, the difference in the number of samples of various fault types is ≤5 times. Model building and training module: Constructs a diagnostic model training framework that integrates multiple algorithms, including a basic model selection submodule, a model parameter configuration submodule, an offline training submodule, and an incremental training submodule; The basic model selection submodule supports multiple algorithms such as random forest, XGBoost, LightGBM, and deep learning (CNN-LSTM). The core algorithm can be switched according to the characteristics of the fault type. The default algorithm is the fusion algorithm of random forest + XGBoost. The model parameter configuration submodule has built-in default optimal parameter sets for different algorithms, and supports manual adjustment of parameters (such as decision tree depth, learning rate, regularization coefficient, etc.) and parameter grid search optimization; The offline training submodule performs offline batch training based on historical sample sets to generate a basic diagnostic model. During the training process, it outputs multiple evaluation metrics in real time, including model accuracy, recall, and F1 score. The incremental training submodule is used to receive real-time training samples and use incremental learning algorithms (such as online gradient descent) to iteratively update the base model, avoiding the resource consumption caused by full retraining. The incremental training cycle is configurable (default 1 hour / cycle). Model Evaluation and Optimization Module: This module performs multi-dimensional evaluation and optimization of the trained model. It includes sub-modules for calculating evaluation metrics, model selection, and model pruning. The core formulas used in this module are as follows: 1. Accuracy: ; in, (True positive): The fault was correctly diagnosed as a fault; (True negative): A normal state is correctly diagnosed as normal; (False positive): A normal state is mistakenly identified as a malfunction; (False negative): The fault was mistakenly identified as normal; 2. Recall: ; Function: Measures the degree to which the model misses faults; the higher the value, the fewer faults are missed.

[0024] 3. Precision: ; Function: Measures the accuracy of the model's diagnostic results; a higher accuracy indicates fewer false positives. 5. F1 score (comprehensive evaluation index): ; Function: To balance precision and recall and comprehensively reflect the diagnostic performance of the model, this application uses an F1 score improvement of ≥5% as the model update standard; The evaluation index calculation submodule calculates the model's core indicators, including accuracy, recall, precision, F1 score, and confusion matrix, and outputs evaluation results for different failure types. The model selection submodule automatically marks a new model as the "optimal model" when the evaluation metric (such as the overall F1 score) of the new model generated by training is improved by ≥5% compared to the previous version. If the improvement is less than 3%, parameter optimization or algorithm adjustment will be triggered. The model pruning submodule is used to prune and optimize complex models (such as deep learning models and multi-decision tree fusion models), remove redundant nodes, reduce model complexity, and ensure that the model inference latency is ≤100ms to meet the needs of real-time diagnosis. Model Deployment and Update Module: Responsible for deploying and updating the trained and optimized model, including the model encapsulation submodule, the model push submodule, and the version management submodule; The model encapsulation submodule is used to encapsulate the optimal model into a standard interface to adapt to the model calling requirements of the data diagnostic analysis unit; The model push submodule automatically pushes the newly packaged model to the machine learning diagnostic module of the data diagnostic analysis unit, supporting "hot deployment" without interrupting system operation; The version management submodule is used to record model version information (including training time, number of samples, evaluation metrics, and parameter configuration), supports model version rollback (when the diagnostic accuracy drops by ≥3% after the deployment of a new model, it will automatically trigger a rollback to the previous best version), and retains the 10 most recent model versions.

[0025] In this invention, the data diagnostic analysis unit includes: Fault rule base module: Stores feature rules for known fault types, including rule storage submodule and rule management submodule; The rule storage submodule adopts a relational database + cache architecture, which supports fast rule query; The rule management submodule supports adding, modifying, deleting, and enabling / disabling rules. The rule format is defined as "feature condition + fault type + confidence level", for example, "CPU usage ≥ 95% for 10 minutes → CPU overload fault → confidence level 99%"; Rule matching and diagnosis module: The preprocessed feature data is matched one by one with the rules in the fault rule base. A multi-threaded parallel matching mechanism is used to improve the matching efficiency. The matching results are divided into three categories: "complete match", "partial match" and "no match". Complete match directly outputs the fault diagnosis result, partial match is marked as a suspected fault, and no match enters the machine learning diagnosis process. Machine Learning Diagnostic Module: A diagnostic model is built based on the random forest algorithm, including a model training submodule, a model inference submodule, and a model optimization submodule; The model training submodule uses historical fault data for offline training and supports incremental training; The model reasoning submodule takes suspected fault data or no matching data as input, calculates the similarity with various fault types, and outputs the fault probability ranking and fault cause analysis. The model optimization submodule adjusts model parameters based on the diagnostic results to improve diagnostic accuracy. The self-learning update module automatically generates new rules and adds them to the fault rule base based on the new fault types and feature data output by the machine learning diagnostic module. At the same time, it adds new fault data to the model training sample set, triggering incremental model training. It supports a manual review mechanism, and new rules and samples must be reviewed and approved before taking effect to avoid accidental updates.

[0026] In this invention, the scheme generation unit includes: Repair Solution Knowledge Base Module: Used to store standard repair strategies corresponding to different fault types, including solution storage submodule and solution update submodule; The solution storage submodule adopts structured storage and supports retrieval by fault type, server type, and business scenario; The solution update submodule supports manual entry of new solutions and automatic synchronization of the latest industry repair strategies. The standard repair strategy includes fields such as operation steps, required tools, precautions, and expected results. Fault Information Parsing Module: Used to parse diagnostic analysis results and extract key information, including fault type, fault level (fatal / critical / general / minor), affected servers (core / normal), affected services (core services / non-core services), and fault duration, providing data support for solution adaptation; Solution adaptation and adjustment module: Based on the fault information analysis results, the standard repair strategy is customized. For example, when the core server fails, pre-requisite steps such as "business migration" and "backup server startup" are added. When core business processes fail, shorten the execution cycle of the repair plan and increase its priority; The adjustment rules include "fault level → solution complexity", "server role → operation priority", and "business type → risk control measures". Solution Output Optimization Module: The adapted and adjusted solution is formatted and output, which clarifies the order of operation steps, execution subject (automatic execution / manual execution), estimated time, risk level (high / medium / low) and emergency rollback solution. It supports exporting the solution as PDF, Word or script format that can be directly executed by the operation and maintenance platform.

[0027] In this invention, the scheme verification and trial operation unit includes: Simulation environment building module: Based on virtualization technologies (such as KVM, Docker), it builds a simulated running environment, including an environment replication submodule, an environment configuration submodule, and an environment destruction submodule; The environment replication submodule is used to accurately replicate the hardware configuration, software version, network topology, and data status of a faulty server; The environment configuration submodule is used to support the adjustment of environment parameters and simulate different load scenarios; The environment destruction submodule is used to automatically destroy the environment and release resources after verification is completed, and to physically isolate the simulation environment from the production environment to avoid mutual interference. Simulation verification execution module: Automatically executes the repair plan in a simulation environment, including a step execution submodule, a process monitoring submodule, and a result judgment submodule; The step execution submodule executes the steps in the order of the solution steps, and supports pause, continue, and terminate operations; The process monitoring submodule is used to monitor system resource usage, service availability, data consistency, and log output metrics in real time during the execution process; The result determination submodule is used to compare the fault indicators before and after execution to determine whether the fault has been repaired or whether a secondary fault has occurred. Real device verification adaptation module: For core servers or high-risk faults, it supports small-scale real device trial operation, including backup server screening submodule, real device environment isolation submodule and verification scope control submodule; The standby server filtering submodule is used to filter suitable standby servers based on server configuration consistency and load conditions. The real device environment isolation submodule ensures that real device verification does not affect production operations through network isolation, data isolation, and other methods. The verification scope control submodule limits the scope of verification impact, covering only fault-related functions. Solution Iteration and Optimization Module: If simulation verification or real machine verification fails, analyze the reasons for failure (such as missing steps, incorrect parameters, or environment adaptation issues), and report the problem to the solution adaptation and adjustment module of the solution generation unit to trigger solution iteration. It supports multiple rounds of iterative verification until the solution verification is successful.

[0028] In this invention, the data early warning unit includes: The warning threshold configuration module is used to set security thresholds for various types of data, including the real-time monitoring threshold submodule (configuring normal range thresholds for hardware, software, network, and environmental data) and the trend warning threshold submodule (configuring data change trend thresholds, such as memory usage increasing by ≥10% within 5 minutes). The warning threshold configuration module supports custom configuration of thresholds according to server type and business scenario, and supports batch import / export of thresholds. Real-time monitoring and early warning module: Used to receive raw data from the data acquisition unit and compare it with the real-time monitoring threshold in real time. The real-time monitoring and early warning module includes a data comparison submodule (using a millisecond-level comparison frequency to ensure timely early warning), an early warning triggering submodule (which immediately triggers an early warning when the data exceeds the threshold), and an early warning notification submodule (pushing early warning information through sound and light alarms, SMS, email, operation and maintenance platform pop-ups, enterprise WeChat / DingTalk robots, etc.). Fault prediction and early warning module: Used to receive the characteristic data trend analysis results of the data diagnosis and analysis unit, including trend analysis submodule (based on time series analysis algorithm to predict data change trend), prediction decision submodule (when the trend meets preset conditions, trigger prediction and early warning), and early warning suggestion generation submodule (pushing suggestions such as potential fault types, preventive measures, priorities, etc.). The early warning management module is used to manage early warning information throughout its entire lifecycle. It includes an early warning classification submodule (which divides early warnings into three levels: general early warning, important early warning, and emergency early warning, corresponding to different response time limits), an early warning processing submodule (which supports the confirmation, ignoring, processing, and closing of early warnings, and records the processing process), and an early warning statistics submodule (which collects data such as the number of early warning triggers, processing rate, and average processing time, for optimizing threshold configuration).

[0029] Example 2 Reference Figure 10A server fault diagnosis method for a server group, applicable to any of the above-mentioned server fault diagnosis systems for server groups, includes the following steps: S1: Data acquisition. The hardware data acquisition module, software data acquisition module, and network data acquisition module of the data acquisition unit acquire data of the corresponding dimensions respectively. The acquisition control module coordinates the acquisition mode and generates key data markers. S2: Data preprocessing, the data cleaning module of the data preprocessing unit removes abnormal data, the data standardization module unifies the data units, and the feature extraction module extracts the core feature vectors; S3: Data diagnostic analysis. The rule matching diagnostic module of the data diagnostic analysis unit matches the fault rule library, the machine learning diagnostic module processes unmatched data, and the self-learning update module optimizes the rule library and model. S4: Solution generation. The fault information parsing module of the solution generation unit extracts key fault information, the solution adaptation and adjustment module optimizes the standard repair solution, and the solution output optimization module formats and outputs the repair solution. S5: Solution verification and trial operation. The simulation environment construction module of the solution verification and trial operation unit builds a replica environment, the simulation verification execution module executes the solution and monitors it, the real machine verification and adaptation module completes real machine verification of high-risk faults, and the solution iteration and optimization module realizes closed-loop optimization of the solution. S6: Data storage. The data storage unit's data classification storage module stores the entire process data by type, the disaster recovery backup module realizes data backup, the data security management module ensures data security, and the data retrieval and analysis module supports data query and analysis. S7: Data early warning. The real-time monitoring and early warning module of the data early warning unit triggers real-time early warnings, the fault prediction and early warning module pushes prediction information, and the early warning management module realizes full lifecycle management of early warnings.

[0030] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A server fault diagnosis system for server groups, characterized in that, It includes a data acquisition unit, a data preprocessing unit, a data training unit, a data diagnostic analysis unit, a scheme generation unit, a scheme verification and trial operation unit, a data storage unit, and a data early warning unit; The data acquisition unit is used to collect multi-dimensional operational data from each server in the server group and send the collected data information to the data preprocessing unit. The data preprocessing unit is used to receive multi-dimensional operational data sent by the data acquisition unit and perform preprocessing. The preprocessing steps include data cleaning, data standardization, and data feature extraction. The data training unit is used to build, train, and optimize the fault diagnosis model, providing high-accuracy model support for the data diagnosis and analysis unit. The data diagnostic analysis unit is used to receive preprocessed feature data and data information trained by the data training unit, and uses a hybrid diagnostic model of rule matching + machine learning to diagnose faults. The solution generation unit generates a targeted fault repair solution based on the fault diagnosis results from the data diagnosis and analysis unit.

2. The server fault diagnosis system for server groups according to claim 1, characterized in that, The scheme verification and trial operation unit is used to receive the fault repair scheme sent by the scheme generation unit and conduct verification and trial operation; The data storage unit is used to store full-process data information, including: Data classification and storage module: It is divided into raw data storage sub-module, preprocessing data storage sub-module, diagnostic result storage sub-module and solution data storage sub-module according to data type. The data classification and storage module adopts a distributed storage architecture and supports data sharding storage. Disaster recovery and backup module: realizes off-site backup and disaster recovery of data, including a scheduled backup submodule, a real-time synchronization submodule and a disaster recovery submodule. The backup data of the disaster recovery and backup module is stored using AES-256 encryption. Data security management module: Ensures data storage security, including access control submodule, encrypted storage submodule, and operation auditing submodule; Data retrieval and analysis module: Supports multi-dimensional retrieval of data throughout the entire process, including conditional retrieval submodule, full-text retrieval submodule, and statistical analysis submodule; The data early warning unit is communicatively connected to the data acquisition unit and the data diagnostic analysis unit to realize a dual early warning mechanism. The data early warning unit supports setting early warning levels.

3. The server fault diagnosis system for server groups according to claim 1, characterized in that, The data acquisition unit includes: Hardware data acquisition module: Collects hardware operating parameters through the server's built-in system, including CPU utilization, memory usage, hard disk read / write speed, power supply voltage, fan speed, and hardware temperature; Software data acquisition module: Collects software data through operating system API interfaces and application log parsers, including process running status, error codes, service start and stop records, thread blocking duration, and log exception keywords; Network data acquisition module: Collects network data through packet capture tools, including inter-server communication latency, packet loss rate, bandwidth utilization, number of TCP connection establishment failures, and port open status; Acquisition control module: Used to coordinate the work of each acquisition module, configure the timed acquisition cycle, set data threshold trigger conditions, and realize the switching between timed acquisition and trigger-based acquisition.

4. A server fault diagnosis system for server groups according to claim 1, characterized in that, The data preprocessing unit includes: The data cleaning module is responsible for removing outliers, missing values, and duplicate data. The data cleaning module has a built-in outlier detection submodule, a missing value handling submodule, and a duplicate data removal submodule. The outlier detection submodule is used to mark data that exceeds the mean ± 3 times the standard deviation as outliers and delete them. The missing value processing submodule uses linear interpolation to complete data with a missing value rate of less than 5%, and discards data with a missing value rate of more than 5%. The duplicate data removal submodule deletes completely duplicate data records by comparing data hash values. Data standardization module: used to convert data of different dimensions from hardware, software, network and environment. The data standardization module has a built-in dimension recognition submodule, which is used to automatically identify data type and dimension and adapt to the standardization calculation of different data. Feature extraction module: Based on principal component analysis, a feature extraction model is constructed to extract core feature vectors from standardized data. The feature extraction module includes: The feature calculation submodule is used to calculate fault correlation, data fluctuation coefficient, and feature parameter coupling degree. The dimension reduction submodule is used to remove redundant features and retain principal components with a contribution rate of ≥85%. The feature vector output submodule is used to format the extracted feature vectors and transmit them to the data diagnostic analysis unit.

5. A server fault diagnosis system for server groups according to claim 1, characterized in that, The data training unit includes: Training data acquisition module: used to acquire the data required for training, including real-time feature data submodule, historical sample retrieval submodule and sample selection submodule; Sample labeling optimization module: Standardizes and optimizes the acquired training samples, including automatic labeling submodule, manual review submodule and sample augmentation submodule; Model building and training module: Constructs a diagnostic model training framework that integrates multiple algorithms, including a basic model selection submodule, a model parameter configuration submodule, an offline training submodule, and an incremental training submodule; Model evaluation and optimization module: performs multi-dimensional evaluation and optimization of the trained model, including evaluation index calculation submodule, model selection submodule and model pruning submodule; Model Deployment and Update Module: Responsible for deploying and updating the trained and optimized model, including the model encapsulation submodule, the model push submodule, and the version management submodule.

6. A server fault diagnosis system for server groups according to claim 1, characterized in that, The data diagnostic analysis unit includes: Fault rule base module: Stores feature rules for known fault types, including rule storage submodule and rule management submodule; Rule matching and diagnosis module: The preprocessed feature data is matched one by one with the rules in the fault rule base. The matching results are divided into three categories: "complete match", "partial match" and "no match". Complete match directly outputs the fault diagnosis result, partial match is marked as suspected fault, and no match enters the machine learning diagnosis process. Machine Learning Diagnostic Module: A diagnostic model is built based on the random forest algorithm, including a model training submodule, a model inference submodule, and a model optimization submodule; The self-learning update module automatically generates new rules and adds them to the fault rule library based on the new fault types and feature data output by the machine learning diagnostic module. At the same time, it adds new fault data to the model training sample set, triggering incremental model training. It supports a manual review mechanism, and new rules and samples must be reviewed and approved before they take effect.

7. A server fault diagnosis system for server groups according to claim 1, characterized in that, The scheme generation unit includes: Repair Solution Knowledge Base Module: Used to store standard repair strategies corresponding to different fault types, including solution storage submodule and solution update submodule; Fault Information Parsing Module: Used to parse diagnostic analysis results and extract key information, including fault type, fault level, affected servers, affected services, and fault duration, to provide data support for solution adaptation; Solution adaptation and adjustment module: Based on the fault information analysis results, the standard repair strategy is adjusted in a personalized manner; Solution output optimization module: The adapted and adjusted solution is output in a formatted manner, which clarifies the order of operation steps, the executing entity, the estimated time, the risk level and the emergency rollback plan.

8. A server fault diagnosis system for server groups according to claim 2, characterized in that, The scheme verification and trial operation unit includes: Simulation environment construction module: Based on virtualization technology, it constructs a simulation runtime environment, including an environment replication submodule, an environment configuration submodule, and an environment destruction submodule; Simulation verification execution module: Automatically executes the repair plan in a simulation environment, including a step execution submodule, a process monitoring submodule, and a result judgment submodule; Real device verification adaptation module: For core servers or high-risk faults, it supports small-scale real device trial operation, including backup server screening submodule, real device environment isolation submodule and verification scope control submodule; Solution Iteration and Optimization Module: If simulation verification or real machine verification fails, analyze the reasons for the failure and report the problem to the solution adaptation and adjustment module of the solution generation unit to trigger solution iteration. It supports multiple rounds of iterative verification until the solution verification is successful.

9. A server fault diagnosis system for server groups according to claim 2, characterized in that, The data storage unit includes: Data classification and storage module: divided into raw data storage submodule, preprocessing data storage submodule, diagnostic result storage submodule and solution data storage submodule according to data type; Disaster recovery and backup module: realizes off-site backup and disaster recovery of data, including a scheduled backup submodule, a real-time synchronization submodule, and a disaster recovery submodule. The backup data of the disaster recovery and backup module is stored using AES-256 encryption. Data security management module: Ensures data storage security, including access control submodule, encrypted storage submodule, and operation auditing submodule; Data retrieval and analysis module: Supports multi-dimensional retrieval of data throughout the entire process, including conditional retrieval submodule, full-text retrieval submodule, and statistical analysis submodule; The data early warning unit includes: Early warning threshold configuration module: used to set security thresholds for various types of data, including real-time monitoring threshold submodule and trend early warning threshold submodule; Real-time monitoring and early warning module: used to receive raw data from the data acquisition unit and compare it with the real-time monitoring threshold in real time. The real-time monitoring and early warning module includes a data comparison submodule, an early warning triggering submodule, and an early warning notification submodule. Fault prediction and early warning module: used to receive the characteristic data trend analysis results from the data diagnosis and analysis unit, including trend analysis submodule, prediction decision submodule, and early warning suggestion generation submodule; Early warning management module: used for full lifecycle management of early warning information, including early warning classification submodule, early warning processing submodule, and early warning statistics submodule.

10. A server fault diagnosis method for a server group, applicable to the server fault diagnosis system for a server group as described in any one of claims 1-9, characterized in that, Includes the following steps: S1: Data acquisition. The hardware data acquisition module, software data acquisition module, and network data acquisition module of the data acquisition unit acquire data of the corresponding dimensions respectively. The acquisition control module coordinates the acquisition mode and generates key data markers. S2: Data preprocessing, the data cleaning module of the data preprocessing unit removes abnormal data, the data standardization module unifies the data units, and the feature extraction module extracts the core feature vectors; S3: Data diagnostic analysis. The rule matching diagnostic module of the data diagnostic analysis unit matches the fault rule library, the machine learning diagnostic module processes unmatched data, and the self-learning update module optimizes the rule library and model. S4: Solution generation. The fault information parsing module of the solution generation unit extracts key fault information, the solution adaptation and adjustment module optimizes the standard repair solution, and the solution output optimization module formats and outputs the repair solution. S5: Solution verification and trial operation. The simulation environment construction module of the solution verification and trial operation unit builds a replica environment, the simulation verification execution module executes the solution and monitors it, the real machine verification and adaptation module completes real machine verification of high-risk faults, and the solution iteration and optimization module realizes closed-loop optimization of the solution. S6: Data storage. The data storage unit's data classification storage module stores the entire process data by type, the disaster recovery backup module realizes data backup, the data security management module ensures data security, and the data retrieval and analysis module supports data query and analysis. S7: Data early warning. The real-time monitoring and early warning module of the data early warning unit triggers real-time early warnings, the fault prediction and early warning module pushes prediction information, and the early warning management module realizes full lifecycle management of early warnings.

Images