Chip fault prediction method, device and equipment and computer readable medium

Abstract

The invention relates to a chip fault prediction method, device and equipment and a computer readable medium. The method comprises the following steps: constructing a standard feature sequence based on a time sequence corresponding to dynamic data of a chip; clustering the standard feature sequence based on a clustering algorithm, and determining a typical working mode and a corresponding reference time sequence; calculating a deviation degree of a test sample through an LSTM auto-encoder based on the reference time sequence, and calculating a health degree score based on the deviation degree; and determining a final prediction result through a multi-model fusion method in combination with the health degree score of the sliding time window and hardware statistical characteristics of the chip. The problems that chip fault prediction depends on a large number of known label samples, the accuracy of risk prediction is low, and the load uncertainty in the actual operation process of the chip is difficult to deal with are solved.

Description

Technical Field

[0001] The present application relates to the field of chip detection technology, and in particular to a chip fault prediction method, device, equipment and computer-readable medium. Background Art

[0002] High-performance computing chips are prone to aging, degradation, or system-level failures during long-term operation due to factors such as heavy workloads and complex environments. Furthermore, chip health monitoring primarily relies on analyzing error reports when they occur, providing post-event warnings and lacking the ability to predict potential risks in advance.

[0003] Currently, related technologies primarily address this issue by building fault classification models based on labeled supervised learning to predict potential chip risks in advance. However, model construction relies on a large number of known positive fault samples. In actual production, chip failures are sporadic and labels are scarce, making it difficult to build a stable and reliable fault classification model. The accuracy of chip risk prediction is low, making it difficult to cope with the actual load uncertainty during operation. Furthermore, the data that can be labeled is primarily concentrated in cases of ECC memory errors. In computationally intensive deep learning network computing scenarios, the various chip modules are highly sensitive to uncorrectable faults, making it impossible to guarantee the safety of chip operation.

[0004] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention

[0005] This application provides a chip fault prediction method, apparatus, device, and computer-readable medium to address the technical problem that chip fault prediction relies on a large number of known label samples, resulting in low risk prediction accuracy and difficulty in coping with actual load uncertainty during operation.

[0006] According to one aspect of an embodiment of the present application, the present application provides a chip fault prediction method, including: constructing a standard feature sequence based on the time series corresponding to the dynamic data of the chip; clustering the standard feature sequence based on a clustering algorithm to determine a typical working mode and a corresponding benchmark time series; calculating the degree of deviation of the test sample through an LSTM autoencoder based on the benchmark time series, and calculating a health score based on the degree of deviation; combining the health score of the sliding time window with the hardware statistical characteristics of the chip, and determining the final prediction result through a multi-model fusion method.

[0007] Optionally, a standard feature sequence is constructed based on the time series corresponding to the dynamic data of the chip, including: obtaining the time series corresponding to the dynamic data of the chip; smoothing the time series according to a preset time window size and sliding step size to obtain a smoothed time series; standardizing the smoothed time series to obtain a standard time series; constructing a matrix corresponding to the standard time series and using it as the standard feature sequence.

[0008] Optionally, the standard feature sequence is clustered based on a clustering algorithm to determine a typical working mode and a corresponding benchmark time series, including: clustering the standard feature sequence through a kmeans++ clustering algorithm; determining the load type of different nodes of the chip based on the clustering results; determining the typical working mode of different nodes of the chip based on the load type; and obtaining a benchmark time series corresponding to the typical working mode.

[0009] Optionally, the degree of deviation of the test sample is calculated based on the reference time series through the LSTM autoencoder, including: obtaining the time domain statistical features and frequency domain statistical features corresponding to the reference time series; constructing a multimodal input vector in combination with the time series, the time domain statistical features and the frequency domain statistical features; inputting the multimodal input vector into the LSTM autoencoder, and generating a reconstructed sequence corresponding to the multimodal input vector through the decoding end of the LSTM autoencoder; calculating the mean square error between the reconstructed sequence and the input sequence corresponding to the test sample, and using the mean square error as the degree of deviation of the test sample.

[0010] Optionally, the calculation formula of the mean square error includes:

[0011] Where: R represents the mean square error, represents the length of the time series, represents the feature dimension of the input sequence, represents the value of the input sequence at time t, Represents the value of the reconstructed sequence at time t.

[0012] Optionally, calculating the health score based on the degree of deviation includes: setting a preset value for the degree of deviation in combination with the data state distribution of historical behavior and baseline behavior; calculating the relationship between the degree of deviation and the preset value for the degree of deviation; marking an input sequence whose degree of deviation is greater than the preset value for the degree of deviation as an abnormal sequence; and calculating the health score according to the preset value for the degree of deviation and the degree of deviation corresponding to the abnormal sequence.

[0013] Optionally, the calculation formula for the health score includes:

[0014] Where: represents the health score, Indicates the degree of deviation of the test sample, Indicates the degree of deviation from the preset value. Indicates the width of the preset value of the deviation from the normal distribution.

[0015] Optionally, the health score of the sliding time window is combined with the hardware statistical characteristics of the chip, and the final prediction result is determined by a multi-model fusion method, including: taking the corresponding moment of the health score as the end point sliding time window, obtaining the anomaly scores of a preset number of time windows; constructing a prediction sequence based on the anomaly score combined with the hardware statistical characteristics corresponding to the static data of the chip cluster; calculating the first prediction score, second prediction score and third prediction score corresponding to the prediction sequence through a gradient boosting decision tree model, a multi-layer perceptron model and a long short-term memory model respectively; calculating the failure probabilities corresponding to different models based on the first prediction score, the second prediction score and the third prediction score; sorting the failure probabilities of different models from large to small, and determining the final prediction result according to the intersection of the failure probability sorting of different models.

[0016] Optionally, after determining the final prediction result through the multi-model fusion method, the method further includes: detecting the prediction score corresponding to the final prediction result in real time; when the deviation between the prediction score and the actual detection score exceeds a deviation threshold or a fault omission event occurs, triggering a retraining mechanism.

[0017] According to another aspect of the embodiments of the present application, the present application provides a chip fault prediction device, which is used to perform the steps of the above-mentioned chip fault prediction method, including: A feature construction module is used to construct a standard feature sequence based on the time series corresponding to the dynamic data of the chip; A benchmark learning module, configured to cluster the standard feature sequences based on a clustering algorithm to determine a typical operating mode and a corresponding benchmark time sequence; An anomaly detection module, configured to calculate the degree of deviation of a test sample based on the benchmark time series through an LSTM autoencoder, and calculate a health score based on the degree of deviation; The prediction module is used to combine the health score of the sliding time window with the hardware statistical characteristics of the chip and determine the final prediction result through a multi-model fusion method.

[0018] According to another aspect of an embodiment of the present application, the present application provides an electronic device, including a memory, a processor, a communication interface and a communication bus, wherein the memory stores a computer program that can be run on the processor, the memory and the processor communicate through the communication bus and the communication interface, and the processor implements the steps of the above-mentioned chip fault prediction method when executing the computer program.

[0019] According to another aspect of an embodiment of the present application, the present application also provides a computer-readable medium having a non-volatile program code executable by a processor, where the program code enables the processor to execute the steps of the above-mentioned chip fault prediction method.

[0020] The above technical solution provided by the embodiment of the present application has the following advantages compared with the related art: This application uses a clustering algorithm to analyze standard feature sequences, eliminating the need for pre-labeling fault samples. Instead, it directly mines typical operating patterns from the chip's dynamic data during normal operation. When calculating the health score, an LSTM autoencoder calculates the deviation of test samples. The LSTM autoencoder requires only normal data for training. By learning the characteristic distribution of normal patterns, it identifies test samples as potential risks when they deviate from the baseline time series, avoiding reliance on fault labels and making it more suitable for scenarios where faults occur sporadically. When constructing the standard feature sequence, the chip's dynamic data is converted into a time series to capture the fluctuation patterns of features under load. By combining a sliding time window with the health score, the impact of short-term load fluctuations on chip status can be tracked in real time. Hardware statistical features are combined with the health score to generate a final prediction using weighted fusion or ensemble learning methods. When the LSTM autoencoder detects feature deviation and high-frequency cache errors in the hardware statistics, multiple models can more accurately determine the risk level, reducing the misjudgment rate of a single model. The standard feature sequence is constructed based on real-time data from different chip modules, rather than a single memory metric, to obtain multi-dimensional feature vectors and cover more potential fault sources. By calculating the health score based on the degree of deviation, the chip status can be converted into a quantifiable indicator, and health trends can be observed through historical data. The real-time computing characteristics of the sliding time window and LSTM autoencoder continuously update the health score during load changes, promptly responding to sudden risks and improving chip operation safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0023] Figure 1 A schematic diagram of a chip fault prediction method according to an embodiment of the present application; Figure 2 This is a structural block diagram of a chip fault prediction device provided according to an embodiment of the present application; Figure 3 A schematic structural diagram of an optional electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0025] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of this application and have no specific meaning. Therefore, "module" and "component" can be used interchangeably.

[0026] In related technologies, when predicting chip faults, a label-supervised learning-based method is used to build a fault classification model to predict the potential risks of the chip. This relies on a large number of known fault positive samples. In addition, chip faults occur sporadically during chip operation and labels are scarce, resulting in unstable prediction models and low prediction accuracy.

[0027] In order to solve the problem mentioned in the background technology, according to one aspect of the embodiments of the present application, an embodiment of a chip fault prediction method is provided.

[0028] refer to Figure 1 As shown, an embodiment of the present application provides a chip fault prediction method, comprising: Step S102: constructing a standard feature sequence based on the time series corresponding to the dynamic data of the chip.

[0029] Specifically, a standard feature sequence is constructed based on the time series corresponding to the dynamic data of the chip, including: Obtain the time series corresponding to the chip's dynamic data; Smoothing the time series according to the preset time window size and sliding step size to obtain a smoothed time series; Normalize the smoothed time series to obtain a standard time series; Construct the matrix corresponding to the standard time series and use it as the standard feature sequence.

[0030] As can be understood, this embodiment is primarily applicable to online fault prediction for cluster chips. Cluster chips refer to multiple chips working together in a computer system. Cluster chips include high-performance computing chips such as GPUs (graphics processing units), TPUs (tensor processing units), and NPUs (neural network processing units), or a combination thereof. This embodiment uses a GPU as an example for online fault prediction. Before executing step S102, this embodiment requires collecting relevant chip data. Dynamic and static data are collected non-invasively using a hardware feature collection framework. Dynamic data refers to data that changes over time during chip operation, while static data refers to data that does not change over time during chip operation. Non-invasive data collection can be implemented using the chip's built-in sensors or standard interfaces. This allows dynamic data collection without disrupting the chip's normal operating procedures or modifying the chip's hardware architecture or software logic. This ensures the stability and integrity of chip operation and prevents inaccurate fault detection due to misidentification of faults.

[0031] Chip dynamic data includes temperature, voltage, graphics frequency, core utilization, memory utilization, and video memory usage. Chip static data includes the chip model, ID, IP address, driver version, and the chip's location ID within the machine. In this embodiment, in addition to collecting dynamic and static data, log data (fault log data) of chip failures is also collected. This log data includes environmental parameters, error type, and contextual information at the time of the failure. This data can be used to assist in the training and validation of subsequent prediction models, thereby improving the accuracy of chip failure prediction.

[0032] Furthermore, the corresponding time series is extracted from the dynamic data of the chip, and the time window size is set as the sliding step size. For example, when the time window size is 100 and the sliding step size is 5, it means that 100 seconds of data are intercepted and a new window is generated every 5 seconds. The extracted time series is low-pass filtered and smoothed to obtain a smoothed time series. Smoothing can eliminate high-frequency noise in the time series. Low-pass filtering removes the part above a specific cutoff frequency through frequency domain filtering. The high-frequency noise in the dynamic data of the chip during operation usually comes from electromagnetic interference or sampling errors. After the smoothing process obtains the smoothed time series, the smoothed time series is standardized to eliminate the dimensional differences between different smoothed time series and obtain a standard time series. In this embodiment, the Z-Score standardization method is used to standardize the smoothed time series, and the matrix corresponding to the standard time series is constructed as N×M (where N is the number of GPU nodes and M is the number of windows). Each time window includes a standard feature sequence, and the standard feature sequence of each GPU node is expressed as Si={xi,1,xi,2,...,xi,k} (Si represents the feature sequence of the i-th GPU node, and xi,k represents the feature vector of the k-th time window of the i-th GPU node).

[0033] In this embodiment, random fluctuations in hardware sampling are eliminated by low-pass filtering, making the time series closer to the real physical trend and effectively suppressing the high-frequency noise of the time series. Standardization processing can eliminate the dimensional differences of features such as temperature and voltage, achieve multi-feature dimensional unification, avoid the judgment of the model dominated by high-value features, and achieve feature weight balance. By presetting the time window and sliding step size, the time series can be converted into a window sequence of fixed dimensions, explicitly expressing short-term trends and medium-term dependencies. Constructing a standard feature sequence in the form of a matrix can make the standard feature sequence adapt to subsequent autoencoder detection, improve the data processing effect and the autoencoder training speed, and improve the accuracy and efficiency of chip fault prediction.

[0034] Step S104: clustering the standard feature sequence based on a clustering algorithm to determine a typical working mode and a corresponding reference time sequence.

[0035] Specifically, the standard feature sequences are clustered based on the clustering algorithm to determine the typical working mode and the corresponding benchmark time series, including: Clustering of standard feature sequences using kmeans++ clustering algorithm; Determine the load types of different nodes of the chip based on the clustering results; Determine the typical operating modes of different nodes of the chip based on the load type; Obtain the benchmark time series corresponding to the typical working mode.

[0036] It is understandable that chip time series usually have a certain time offset (for example, the starting point of temperature rise at different nodes is different), and the use of Euclidean distance cannot guarantee accurate measurement of similarity. Therefore, in this embodiment, when using the kmeans++ clustering algorithm to cluster the standard feature sequence, the time series similarity of the standard feature sequence is calculated based on the DTW (Dynamic Time Warping) distance, and the cluster center is calculated using the DBA (Dynamic Time Warping Average) method. The two sequences are aligned by bending the time axis and the minimum cumulative distance is calculated, that is:

[0037] Where, represents the query sequence, represents a template sequence, represents a time-warped path, represents the DWY distance between Q and C, represents the weight of time point i, Indicates that time point i in the query sequence is mapped to the time point in the template sequence. Optionally, the preset number of clusters is set to 3 to identify three typical working modes. For example, the three typical working modes include computing-intensive state, memory-intensive state and mixed computing state. The computing-intensive state is mainly used for high-intensity computing tasks, such as AI reasoning and scientific computing. The memory-intensive state is mainly used for tasks with frequent data reading and writing, such as database operations and cache access. The task type of the mixed computing state includes both computing and memory operations. Among them, the computing characteristics in the computing-intensive state exceed the high point of the distribution, the computing-storage characteristics in the memory-intensive state exceed the high point of the distribution, and the computing / storage characteristics in the mixed computing state fluctuate by >30%. For example, the Core utilization of the computing-intensive state is set to >70% and the Memory utilization is <30%, the Memory utilization of the memory-intensive state is set to >70% and the Core utilization is <30%, and the computing / storage characteristics of the mixed wave state fluctuate by >30%.

[0038] Alternatively, in this embodiment, assume that standard feature sequences for three GPUs are collected. Each standard feature sequence includes 21 time windows, and each time window contains three features: temperature, voltage, and core utilization. Taking the core utilization sequences of any two GPUs as an example, the sequence O = [0.2, 0.3, 0.8, 0.9, 0.85, 0.7, 0.6] and the sequence C = [0.1, 0.15, 0.2, 0.7, 0.8, 0.85, 0.9] have a time offset. When calculating cluster centers, if a cluster contains the sequence A = [0.2, 0.3, 0.8, 0.9, 0.8], the sequence B = [0.1, 0.2, 0.7, 0.8, 0.9], and the sequence C = [0.3, 0.2, 0.6, 0.7, 0.8], the DBA method is used to perform a weighted average based on the DTW paths of each sequence and the initial center. The center sequence is iteratively updated until convergence, resulting in the cluster center. The default number of clusters is set to 3, and three typical operating modes are ultimately identified: compute-intensive, memory-intensive, and mixed-computation. After clustering, DBA is applied to the sequences within each cluster to generate a baseline time series for the corresponding mode, which serves as a reference for subsequent chip fault prediction. For example, if the memory utilization of a compute-intensive GPU consistently exceeds 50%, a warning of a possible memory leak is issued.

[0039] In the above embodiment, the chip nodes can be divided into compute-intensive, memory-intensive, IO-intensive and mixed-load types through kmeans++ clustering, and the classification accuracy is high. Based on sliding window clustering, the temporal changes of the working mode can be captured, and the delay in mode conversion recognition is low. Each cluster center corresponds to a reference time series, so that the abnormal threshold is upgraded from a global unified threshold to a dynamic threshold for each mode, reducing the false alarm rate of fault prediction. The deviation between the reference time series and the real-time data can quantify the stage of fault evolution. For example, the drift path of the memory leak fault in the cluster space is: normal mode → mixed load mode → abnormal outlier, which can significantly extend the advance warning time.

[0040] Step S106: Calculate the deviation degree of the test sample through the LSTM autoencoder based on the benchmark time series, and calculate the health score based on the deviation degree.

[0041] Specifically, the deviation degree of the test sample is calculated based on the benchmark time series through the LSTM autoencoder, including: Obtaining time domain statistical characteristics and frequency domain statistical characteristics corresponding to the benchmark time series; Combining time series, time domain statistical features and frequency domain statistical features to construct a multimodal input vector; Input the multimodal input vector into the LSTM autoencoder, and generate the reconstructed sequence corresponding to the multimodal input vector through the decoding end of the LSTM autoencoder; Calculate the mean square error between the reconstructed sequence and the input sequence corresponding to the test sample, and use the mean square error as the degree of deviation of the test sample.

[0042] Specifically, the calculation formula of the mean square error includes:

[0043] Where: R represents the mean square error, represents the length of the time series, represents the feature dimension of the input sequence, represents the value of the input sequence at time t, Represents the value of the reconstructed sequence at time t.

[0044] It can be understood that the LSTM autoencoder in this embodiment is an unsupervised learning model based on deep learning, consisting of an encoder and a decoder. The core function of the decoder is to reconstruct an output sequence similar to the original input based on the hidden features extracted by the encoder. In the chip health assessment scenario of this embodiment, the decoder reconstructs the test sample by learning the normal pattern of the benchmark time series, and measures the degree of data deviation using the reconstruction error. The encoder is composed of multiple layers of LSTM units. After receiving a multimodal input vector, it dynamically controls the forget gate, input gate, and output gate to gradually compress the key features in the time series data and generate a low-dimensional hidden state vector. During the decoder's time series reconstruction process, the hidden state vector output by the encoder is used as the decoder's initial hidden state and is combined with an initial input vector. The decoder updates the hidden state time step by time step through the LSTM unit's gating mechanism (forget gate, input gate, output gate), including using the forget gate to determine whether to discard historical information in the original sequence. The decoder predicts the normal timing data of the test sample based on the hidden features of the reference pattern. For example, when an abnormal voltage increase occurs in the test sample, the decoder corrects the abnormal voltage increase according to the reference pattern and generates a reconstructed sequence close to the normal state.

[0045] In this embodiment, the length is Time-domain statistical features are extracted from the benchmark time series within the time window. These features include mean, standard deviation, skewness, and kurtosis. Frequency-domain statistical features are obtained by performing an FFT transform on the benchmark time series. These features primarily include the dominant frequency component, spectral entropy, and energy distribution. The original time series and the statistical features are then concatenated to form a multimodal input vector. The original time series has a time series length of T = 100 and a feature dimension of d = 8 (e.g., temperature and voltage). Five time-domain statistics and four frequency-domain features are extracted from each feature, resulting in a final input dimension of 100*8+8*(5+4)=872. The model structure of the bidirectional LSTM autoencoder is set as: input layer (872) → bidirectional LSTM encoder (hidden layer 128) → potential representation layer (64) → bidirectional LSTM self-decoder (hidden layer 128) → output layer (800). The bidirectional LSTM encoder simultaneously learns the dependency between historical data and predicted data. The potential representation layer of the LSTM self-decoder compresses features to a low-dimensional space to capture the essential pattern of the data. The LSTM self-decoder reconstructs the original time series through a bidirectional LSTM with reversed time steps.

[0046] In this embodiment, multi-dimensional analysis of chip operating status is achieved by fusing time-domain and frequency-domain features. Time-domain features reflect signal trends and fluctuations (e.g., a linear temperature increase), while frequency-domain features reveal periodic anomalies (e.g., frequency fluctuations caused by a cooling fan failure). The combination of time-domain and frequency-domain features can identify complex fault modes that are difficult to capture with a single dimension, such as a sudden temperature rise accompanied by high-frequency voltage oscillation. A bidirectional LSTM autodecoder learns long-term dependencies in time series and constructs a latent representation of baseline patterns. The LSTM autodecoder's encoder compresses multimodal inputs into a low-dimensional space, preserving the core characteristics of normal operation. When the LSTM autodecoder reconstructs the sequence, the reconstruction error for abnormal samples (e.g., memory leaks) increases significantly, completing the closed loop of "normal mode learning - abnormal deviation detection." The reconstruction error is standardized using a mathematical formula to eliminate the influence of feature dimension and time length. The mean squared error (MSE) uniformly measures the deviation between different features (temperature / voltage), preventing high-amplitude features from dominating judgments. For example, the equivalent quantization of a 1°C temperature deviation and a 0.1V voltage deviation. This metric can directly map the severity of the fault and provide a linear benchmark for the health score, that is, the larger the error, the lower the corresponding health score.

[0047] Furthermore, a health score is calculated based on the degree of deviation, including: Set a preset value for the degree of deviation based on the data status distribution of historical behavior and benchmark behavior; Calculate the relationship between the degree of deviation and the preset value of the degree of deviation; Mark the input sequence whose deviation is greater than the preset deviation value as an abnormal sequence; The health score is calculated based on the deviation degree of the preset deviation degree and the deviation degree corresponding to the abnormal sequence.

[0048] Specifically, the calculation formula for the health score includes:

[0049] Where: represents the health score, Indicates the degree of deviation of the test sample, Indicates the degree of deviation from the preset value. Indicates the width of the preset value of the deviation from the normal distribution.

[0050] In this embodiment, a preset value of the deviation degree is first set, and the reconstruction error distribution of the historical behavior data and the benchmark behavior data is collected to collect the reconstruction errors of a preset number of normal samples and calculate the mean. and standard deviation ,set up (k is the standard deviation coefficient). Set the deviation preset value width to (m is the distribution coefficient), This reflects the normal fluctuation range of the preset deviation level. An anomaly threshold is set based on the 95th percentile of historical behavior data. When the reconstruction error exceeds the error threshold, it is marked as an anomaly. For example, if the 95th percentile of a node's historical error distribution is 0.05, then MSE > 0.05 triggers a flag. The health score of the anomaly sequence is then calculated using the health score calculation formula.

[0051] In this embodiment, a preset value of the degree of deviation is set in combination with historical behavior and baseline behavior. The dynamic adjustment driven by historical data can adapt to chip aging and avoid false alarms caused by natural degradation of hardware. The baseline behavior constraint ensures that the threshold does not deviate from the design safety boundary and prevents the omission of critical faults due to over-adaptation to abnormal conditions. By comparing the value of the degree of deviation with the threshold, anomaly detection is upgraded from "binary judgment" to "degree quantification", which can accurately define the level of anomalies and provide a differentiated response basis for chip operation and maintenance. The subjectivity of manual judgment is eliminated, and the anomaly judgment standard is reproducible. The health score uses an exponential function to map the degree of deviation, which enhances the sensitivity to severe anomalies and slight deviations (RE close to ), the score decreases slowly to avoid false alarms caused by environmental fluctuations, and to deal with serious deviations (RE is much greater than ), the score drops exponentially, highlighting the warning effect of high-risk failures. For example, when the health score drops from 80 points to 40 points, the increase in the probability of failure is significantly greater than that of the linear model. Set the preset value width based on the data distribution , enhance the anti-interference ability of anomaly detection and suppress noise and outlier interference. Reflects the normal fluctuation range, can filter hardware sampling noise, and make the score insensitive to random disturbances. When the chip is in a new load mode, The statistical characteristics of the chip health score can avoid misjudgment caused by deviation from the baseline pattern and maintain the reliability of the chip health score.

[0052] Step S108 , combining the health score of the sliding time window with the hardware statistical characteristics of the chip, and determining the final prediction result through a multi-model fusion method.

[0053] Specifically, the health score of the sliding time window is combined with the hardware statistical characteristics of the chip, and the final prediction result is determined through a multi-model fusion method, including: Use the corresponding moment of the health score as the end point of the sliding time window to obtain the abnormality scores of a preset number of time windows; Construct a prediction sequence based on the anomaly score combined with the hardware statistical features corresponding to the static data of the chip cluster; The first prediction score, second prediction score, and third prediction score corresponding to the prediction sequence are calculated respectively by using the gradient boosting decision tree model, the multi-layer perceptron model, and the long short-term memory model; Calculating failure probabilities corresponding to different models based on the first prediction score, the second prediction score, and the third prediction score; The failure probabilities of different models are sorted from large to small, and the final prediction result is determined based on the intersection of the failure probability sorting of different models.

[0054] In this embodiment, a sliding window of fixed length W (e.g., W = 30 seconds) is used to collect historical anomaly scores, with the current moment as the end point and each sliding step length S = 5 seconds to generate a continuous overlapping input sequence. For example, the score sequence of the last 30 seconds is used. This data serves as model input to capture short-term trend changes. Hardware feature statistical extraction calculates statistics for hardware parameters (such as temperature and voltage) within each window, including mean, standard deviation, maximum, and minimum values. For example, the mean of a temperature series within a window reflects the average thermal load, and the standard deviation reflects the fluctuation amplitude. These features are combined with the anomaly score to form a multidimensional input vector (of dimension W × F, where F is the number of features). After obtaining the predicted sequence, the corresponding prediction scores are calculated using a gradient boosted decision tree (GBDT), a multi-layer perceptron (MLP), and a long short-term memory network (LSTM). The gradient boosted decision tree iteratively constructs multiple decision trees, focusing on samples predicted incorrectly by the previous tree, capturing the nonlinear correlation between hardware features and faults. The gradient boosted decision tree analyzes the combined patterns of temperature spikes and voltage drops. For example, when the mean temperature exceeds 85°C and the voltage standard deviation is greater than 0.1V, it predicts the probability of future failure. The Multilayer Perceptron (MLP) uses a fully connected neural network to learn high-dimensional feature interactions, making it suitable for processing complex coupling relationships between features. It can identify a combination of features such as memory utilization consistently exceeding 90% and video memory usage fluctuations exceeding 20%. This pattern often indicates a memory controller failure. The Long Short-Term Memory (LSTM) uses temporal memory units to capture long-term dependencies between anomaly scores. For example, if the scores of three consecutive windows show a trend of increasing, and the health score decreases at a rate of 0.5 points per second, the LSTM can predict that this trend will continue for the next 10 seconds, triggering an alert.

[0055] In this embodiment, in the process of calculating the prediction score based on the gradient boosting decision tree, multiple decision trees are iteratively trained, and each tree focuses on correcting the prediction error of the previous model, forming a "residual iteration" mode. The training steps include: Initialization: Use the first decision tree to predict the failure probability and obtain the initial result ; Calculate the residual: Calculate the true fault label y and the initial result The difference between the predicted failure probability of the first decision tree ; Training new trees: using residuals As the goal, train the second decision tree , predict the residual of the second decision tree ; Updated forecast: ( The updated predicted failure probability, is the learning rate, controlling the contribution of each tree); Repeat the steps of calculating residuals and updating predictions until the residuals converge or the maximum number of trees (for example, 100) is reached.

[0056] After training is complete, feature importance is ranked: mean temperature > health score > voltage standard deviation > core utilization. For example, if the mean temperature is > 85°C and the voltage standard deviation is > 0.1V, then the probability of failure (P1 = 0.8); if the health score is < 40 and the core utilization is > 80%, then the probability of failure (P1 = 0.7); if the mean temperature is < 70°C and all features are normal, then the probability of failure (P1 = 0.2). The failure probability P1 is directly output in the range of 0-1, with higher values ​​indicating a higher risk of failure.

[0057] In this embodiment, in the process of calculating the prediction score based on the multi-layer perceptron, the network structure of the multi-layer perceptron is set as follows: the input layer dimension is "sliding window length + hardware statistical feature number"; the hidden layer is set to 2 layers (64 neurons per layer), the activation function is ReLU (to solve the gradient disappearance problem); the output layer is set to 1 neuron, the activation function is Sigmoid (to compress the output to the range of 0-1). Through the multi-layer perceptron forward propagation and training optimization, the input vector X is calculated by the first hidden layer to obtain the output vector of the first hidden layer , is the weight matrix from the input layer to the first hidden layer, is the bias vector of the first hidden layer; the second hidden layer calculates the output vector of the second hidden layer , is the weight matrix from the first hidden layer to the second hidden layer, is the bias vector of the second hidden layer; the output layer is calculated , obtained That is the failure probability P2, where is the weight matrix from the second hidden layer to the output layer, is the output layer bias vector.

[0058] In this embodiment, when calculating the prediction score based on the LSTM network, the health score sequence and hardware features are reshaped into a three-dimensional tensor (1, W, F) (batch = 1, window length W = 30, number of features F = 4), preserving the time dimension. The LSTM network architecture is as follows: the input layer directly receives time series data; there are two LSTM layers (32 memory cells per layer) to capture long-range dependencies; and the output layer is a fully connected layer with Sigmoid function, outputting the failure probability. The prediction score P3 is determined by the LSTM network's temporal feature capture mechanism. The LSTM's "forget gate" discards irrelevant information from the historical state (e.g., temperature fluctuations from 10 seconds ago); the "input gate" selectively adds current-time features (e.g., sudden voltage drops); and the "output gate" generates a hidden state based on the updated cell state. For trend identification, the input sequence is set to a three-window decline in health scores (80 → 75 → 70); the LSTM calculates a decline slope of -0.5 minutes / second, predicting that the score will drop to 65 in the next 10 seconds, and outputting P3 = 0.75, the prediction score based on the LSTM network.

[0059] After calculating the prediction scores for different models, each model is fed a W-dimensional historical sequence and the anomaly score is predicted for the next H steps (e.g., H = 10 seconds). If the score for a step exceeds the 95th percentile threshold (e.g., 0.05) of the normal reconstruction error, it is identified as a potential fault point. The failure probability = the number of steps exceeding the quantile threshold in the next H steps / H. For example, if 7 steps out of H = 10 have a score > 0.05, the failure probability is 70%. After obtaining the failure probabilities output by the three models, all failure probabilities for each model are sorted from high to low, and the intersection of the top K failure probabilities is used as the final decision. For example, if the GBDT predicts a probability of 80%, the MLP predicts 75%, and the LSTM predicts 70%, the intersection of the top two models is taken to avoid misjudgments by a single model. Model weights are assigned based on historical performance. If the GBDT has higher accuracy in the memory failure scenario, it is given a 40% weight, while the MLP and LSTM are each given a 30% weight. The weights are updated every 7 days based on new fault logs. For example, if there is an increase in temperature-related faults in a certain quarter, the LSTM weights are increased (because they are more sensitive to time series trends).

[0060] In the above embodiment, health scores and hardware statistical features are integrated through a sliding window, achieving information complementarity in both temporal and spatial dimensions. The time window captures performance degradation trends, while hardware features reveal abnormal fluctuation patterns. The combination of these two can identify complex faults that are difficult to detect using a single dimension. The fusion of static data and dynamic features enables the model to understand the impact of inherent hardware properties on faults. Three heterogeneous models, GBDT, MLP, and LSTM, are used for parallel prediction. GBDT excels at capturing nonlinear relationships between features, such as a surge in fault probability when temperature exceeds 85°C and voltage is less than 0.8V. MLP learns high-dimensional feature interactions, such as the joint distribution of temperature, voltage, and utilization, through multi-layer nonlinear transformations, making it suitable for complex pattern recognition. LSTM utilizes temporal memory units to capture long-term dependencies, such as the fault risk of a health degradation for three consecutive hours, and is sensitive to trending anomalies. The integrated model significantly improves prediction accuracy compared to single models, particularly for intermittent faults (such as memory outages). A decision-making mechanism based on probability sorting and intersection allows for conservative anomaly determination. Warnings are triggered only when multiple models simultaneously categorize a sample as high-risk, avoiding false positives caused by overfitting a single model. Sorting the prediction probabilities of different models focuses on anomaly data with high consensus, reducing the missed detection rate of chip fault predictions.

[0061] Preferably, after determining the final prediction result by the multi-model fusion method, the method further includes: Real-time detection of the prediction score corresponding to the final prediction result; When the deviation between the predicted score and the actual detection score exceeds the deviation threshold or a fault omission event occurs, the retraining mechanism is triggered.

[0062] In this embodiment, the prediction score and actual detection score corresponding to the final prediction result are monitored in real time. When the deviation between the prediction score and the actual detection score exceeds a deviation threshold or a fault miss occurs, three models—GBDT, MLP, and LSTM—are trained for each window length, forming three independent model groups (short-term model group, medium-term model group, and long-term model group), each containing three sub-models. The weight of sudden abnormal samples is increased, and the learning rate is optimized to 0.1 to quickly capture transient changes. A regularization parameter λ = 0.01 is used to suppress noise interference from aging trends. If the last training time is less than one month, the last model parameters are used as initialization, and only new data is used for fine-tuning to reduce training time. Seven days of actual data (including at least five known faults) are selected, covering normal, warning, and fault states. Model evaluation metrics are obtained, including accuracy, miss rate, and lead time. Accuracy is the number of correct predictions divided by the total number of samples and must be greater than or equal to 85%. The miss rate is the number of unpredicted faults divided by the total number of faults and must be less than or equal to 10%. The lead time is the average lead time before a failure reaches a prediction probability greater than 70%, which must be greater than or equal to 12 hours. Each model group undergoes 5-fold cross-validation. For example, the mid-term model group divides 7 days of data into five time periods, each with 4 days of training and 1 day of testing. The average evaluation result is taken. The comprehensive score of each model group is compared (accuracy × 0.5 + lead time × 0.3 - false negative rate × 0.2). The model group with the highest score is selected (for example, the mid-term model group with the lowest false negative rate for recent failures). The newly obtained model is first tested on 10% of the nodes for 24 hours to compare the prediction consistency between the old and new models (full deployment is required if the consistency is ≥90%) to avoid false alarms caused by misjudgments of the new model. The models trained from the previous three training runs are saved as backups. If the new model score falls below the historical average, the model is automatically rolled back to the most recently reliable model (such as the second trained model). When training is triggered by a false negative, the missing features of the missed fault are analyzed. For example, if a memory fault was not predicted, the model may lack the memory bandwidth fluctuation feature, and this feature will be automatically added to the feature pool for the next training run. If retraining improves accuracy but increases the false alarm rate, the anomaly threshold is automatically relaxed (for example, adjusting the 95th percentile to 98th percentile). A one-week trial run is conducted to balance accuracy and false alarm rates. Fault types from the past three months are analyzed. If frequent faults are short-term, sudden (such as power outages), the model weight for the short-term window (W1) is automatically increased to 50% during the next training session.

[0063] In the above-mentioned embodiment, a closed-loop self-optimization mechanism is established by real-time monitoring of the deviation between predicted scores and actual detection scores. When the prediction deviation exceeds a threshold or a false negative occurs, model retraining is automatically triggered to prevent the degradation of prediction accuracy due to chip aging, environmental changes, or new failure modes. This dynamic update strategy continuously adapts to evolving hardware states (such as declining heat dissipation efficiency) and unknown failure scenarios, maintaining high long-term prediction accuracy. Furthermore, by tracing the source of false negatives, the fault mode library is continuously expanded, achieving an intelligent evolutionary "detection-feedback-optimization" process, significantly improving the long-term reliability and adaptability of the system.

[0064] According to another aspect of the embodiment of the present application, Figure 2 As shown, a chip fault prediction device is provided, which is used to execute the chip fault prediction method in the above embodiment, and the device includes: A feature construction module 201 is used to construct a standard feature sequence based on the time series corresponding to the dynamic data of the chip; A benchmark learning module 203 is used to cluster the standard feature sequence based on a clustering algorithm to determine a typical working mode and a corresponding benchmark time series; Anomaly detection module 205, configured to calculate the degree of deviation of the test sample based on the benchmark time series through the LSTM autoencoder, and calculate the health score based on the degree of deviation; The prediction module 207 is used to combine the health score of the sliding time window with the hardware statistical characteristics of the chip and determine the final prediction result through a multi-model fusion method.

[0065] It should be noted that the feature construction module 201 in this embodiment can be used to execute step S102 in the embodiment of the present application, the benchmark learning module 203 in this embodiment can be used to execute step S104 in the embodiment of the present application, the anomaly detection module 205 in this embodiment can be used to execute step S106 in the embodiment of the present application, and the prediction module 207 in this embodiment can be used to execute step S108 in the embodiment of the present application.

[0066] According to another aspect of the embodiment of the present application, the present application provides an electronic device, such as Figure 3 As shown, it includes a memory 301, a processor 303, a communication interface 305 and a communication bus 307. The memory 301 stores a computer program that can be run on the processor 303. The memory 301 and the processor 303 communicate through the communication interface 305 and the communication bus 307. When the processor 303 executes the computer program, the steps of the above method are implemented.

[0067] The memory and processor in the electronic device communicate via a communication bus and a communication interface. The communication bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus can be divided into an address bus, a data bus, a control bus, and the like.

[0068] The memory may include random access memory (RAM) or non-volatile memory, such as at least one disk storage. Alternatively, the memory may be at least one storage device located away from the processor.

[0069] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.

[0070] According to another aspect of the embodiments of the present application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of any of the above embodiments.

[0071] Optionally, in an embodiment of the present application, the computer-readable medium is configured to store program codes for the processor to execute the following steps: Step S102: constructing a standard feature sequence based on the time series corresponding to the dynamic data of the chip.

[0072] Step S104: clustering the standard feature sequence based on a clustering algorithm to determine a typical working mode and a corresponding reference time sequence.

[0073] Step S106: Calculate the deviation degree of the test sample through the LSTM autoencoder based on the benchmark time series, and calculate the health score based on the deviation degree.

[0074] Step S108 , combining the health score of the sliding time window with the hardware statistical characteristics of the chip, and determining the final prediction result through a multi-model fusion method.

[0075] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0076] When implementing the embodiments of the present application, reference may be made to the above embodiments, which have corresponding technical effects.

[0077] It is understood that the embodiments described herein may be implemented using hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit may be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or a combination thereof.

[0078] For software implementation, the technology described herein can be implemented by a unit that performs the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0079] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0080] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0081] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the modules is merely a logical function division. In actual implementation, there may be other division methods, such as multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0082] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0083] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0084] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application are essentially or partly contributed to the prior art or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard drive, a ROM, a RAM, a magnetic disk, or an optical disk. It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such a process, method, article or device. Without further constraints, an element defined by the phrase "comprises a..." does not preclude the existence of additional identical elements in the process, method, article or apparatus that includes the element.

[0085] The foregoing is merely a list of specific embodiments of the present application, intended to enable those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the broadest scope consistent with the principles and novel features of the present application.

Claims

1. A chip failure prediction method, characterized in that: include: Construct a standard feature sequence based on the time series corresponding to the dynamic data of the chip; Clustering the standard feature sequence based on a clustering algorithm to determine a typical working mode and a corresponding benchmark time series; Calculating the degree of deviation of the test sample based on the benchmark time series through the LSTM autoencoder, and calculating the health score based on the degree of deviation; The health score of the sliding time window is combined with the hardware statistical characteristics of the chip, and the final prediction result is determined through a multi-model fusion method.

2. The chip failure prediction method according to claim 1, characterized in that: A standard feature sequence is constructed based on the time series corresponding to the dynamic data of the chip, including: Obtain the time series corresponding to the chip's dynamic data; Smoothing the time series according to a preset time window size and sliding step size to obtain a smoothed time series; performing standardization processing on the smoothed time series to obtain a standard time series; A matrix corresponding to the standard time series is constructed and used as a standard feature sequence.

3. The chip failure prediction method according to claim 1, characterized in that: Clustering the standard feature sequence based on a clustering algorithm to determine a typical working mode and a corresponding benchmark time series includes: Clustering the standard feature sequences using the kmeans++ clustering algorithm; Determine the load types of different nodes of the chip based on the clustering results; Determining typical operating modes of different nodes of the chip based on the load type; Obtain a reference time series corresponding to the typical operating mode.

4. The chip failure prediction method according to claim 1, characterized in that: The deviation degree of the test sample is calculated based on the benchmark time series through the LSTM autoencoder, including: Obtaining time domain statistical features and frequency domain statistical features corresponding to the reference time series; Constructing a multimodal input vector by combining the time series, the time domain statistical features, and the frequency domain statistical features; Inputting the multimodal input vector into an LSTM autoencoder, and generating a reconstructed sequence corresponding to the multimodal input vector through a decoding end of the LSTM autoencoder; A mean square error between the reconstructed sequence and an input sequence corresponding to the test sample is calculated, and the mean square error is used as the deviation degree of the test sample.

5. The chip failure prediction method according to claim 4, characterized in that: The calculation formula of the mean square error includes: Where: R represents the mean square error, represents the length of the time series, represents the feature dimension of the input sequence, represents the value of the input sequence at time t, Represents the value of the reconstructed sequence at time t.

6. The chip failure prediction method according to claim 1 or 4, characterized in that: Calculating a health score based on the degree of deviation includes: Set a preset value for the degree of deviation based on the data status distribution of historical behavior and benchmark behavior; Calculating the magnitude relationship between the deviation degree and the preset deviation degree value; Marking the input sequence whose deviation degree is greater than the preset deviation degree value as an abnormal sequence; A health score is calculated according to the preset deviation value and the deviation degree corresponding to the abnormal sequence.

7. The chip failure prediction method according to claim 6, characterized in that: The calculation formula of the health score includes: Where: represents the health score, Indicates the degree of deviation of the test sample, Indicates the degree of deviation from the preset value. Indicates the width of the preset value of the deviation from the normal distribution.

8. The chip failure prediction method according to claim 1, characterized in that: Combining the health score of the sliding time window with the hardware statistical characteristics of the chip, the final prediction result is determined through a multi-model fusion method, including: Using the corresponding moment of the health score as the end point of the sliding time window, obtain the abnormality scores of a preset number of time windows; Constructing a prediction sequence based on the anomaly score combined with hardware statistical features corresponding to static data of the chip cluster; Calculate the first prediction score, the second prediction score, and the third prediction score corresponding to the prediction sequence respectively by using a gradient boosting decision tree model, a multilayer perceptron model, and a long short-term memory model; Calculating failure probabilities corresponding to different models based on the first prediction score, the second prediction score, and the third prediction score; The failure probabilities of different models are sorted from large to small, and a final prediction result is determined according to the intersection of the sorted results of the failure probabilities of different models.

9. The chip failure prediction method according to claim 1 or 8, characterized in that: After determining the final prediction result by the multi-model fusion method, the method further includes: Detecting the prediction score corresponding to the final prediction result in real time; When the deviation between the predicted score and the actual detection score exceeds a deviation threshold or a fault omission event occurs, a retraining mechanism is triggered.

10. A chip fault prediction device, configured to execute the steps of the chip fault prediction method according to any one of claims 1 to 9, characterized in that: include: A feature construction module is used to construct a standard feature sequence based on the time series corresponding to the dynamic data of the chip; A benchmark learning module, configured to cluster the standard feature sequences based on a clustering algorithm to determine a typical operating mode and a corresponding benchmark time sequence; An anomaly detection module, configured to calculate the degree of deviation of a test sample based on the benchmark time series through an LSTM autoencoder, and calculate a health score based on the degree of deviation; The prediction module is used to combine the health score of the sliding time window with the hardware statistical characteristics of the chip and determine the final prediction result through a multi-model fusion method.

11. An electronic device comprising a memory, a processor, a communication interface, and a communication bus, wherein the memory stores a computer program that can be run on the processor, and the memory and the processor communicate via the communication bus and the communication interface, characterized in that: When the processor executes the computer program, the steps of the chip failure prediction method according to any one of claims 1 to 9 are implemented.

12. A computer-readable medium having a non-volatile program code executable by a processor, characterized in that The program code enables the processor to execute the steps of the chip failure prediction method according to any one of claims 1 to 9.

Images