A multi-modal data fusion and causal inference hard disk failure prediction method and related equipment

Abstract

The application discloses a kind of multimodal data fusion and causal inference hard disk failure prediction method and related equipment, the method includes: four different types of data in the multimodal data are respectively carried out single mode feature extraction, obtain four different types of data respectively corresponding single mode feature;Four different types of data respectively corresponding single mode feature is carried out feature fusion processing using multi-head self-attention mechanism, and fusion feature is obtained;The fusion feature is input into pre-trained bayesian causal model to carry out failure prediction, and the failure prediction result of the hard disk to be measured is obtained;The failure prediction result includes failure probability, fault type, residual life and failure root cause;The failure prediction result of the hard disk to be measured is visualized output, and when the failure probability exceeds corresponding preset threshold, corresponding grade of hierarchical alarm is executed.The application can effectively improve the accuracy and reliability of hard disk failure prediction.

Description

Technical Field

[0001] This invention relates to the field of fault prediction technology, and in particular to a multimodal data fusion and causal inference method and related equipment for hard disk fault prediction. Background Technology

[0002] With the rapid development of cloud computing and big data industries, data centers, as the core carriers of data storage and computing, directly determine the business continuity and data security of enterprises through their stable operation. Hard drives, as the most critical underlying storage unit in data centers, are a major cause of data loss, unplanned downtime, and soaring maintenance costs due to hardware failures. Therefore, accurate prediction, root cause identification, and early warning of hard drive failures are essential requirements in the field of data center operations and maintenance.

[0003] Currently, hard drive failure prediction technologies in the industry are mainly divided into two categories. One category is a fixed threshold alarm scheme based on the hard drive's SMART attributes. This scheme is simple to deploy and low in cost, and is a common basic solution in the industry, but it has obvious limitations in accuracy. The other category is an intelligent prediction scheme based on machine learning and deep learning. It trains a prediction model through historical operating data to predict the probability of failure. Compared with the threshold scheme, it has a certain improvement in accuracy and is currently a research hotspot.

[0004] However, both of these approaches still face numerous technical challenges in large-scale engineering implementation. Currently, most solutions rely solely on the hard drive's SMART attributes as the only input data source, neglecting multi-dimensional heterogeneous data strongly correlated with failure occurrence, such as hard drive operation logs and error codes, data center environmental conditions, and the hard drive's physical operating status. Hard drive failures are essentially the result of multiple factors, including mechanical wear, electronic component failure, and environmental triggers. A single data source cannot fully depict the hard drive's comprehensive health status, making it difficult to capture early, weak anomaly signals before a failure, leading to frequent false alarms and missed faults. Even those few solutions that attempt to incorporate multimodal data mostly employ simple feature concatenation and fixed-weight fusion methods. They fail to adaptively adjust the contribution weights of different modalities to fault prediction based on the hard drive's real-time operating conditions. This easily results in key weak features strongly correlated with the fault being overwhelmed by noise and redundant features, failing to generate low-redundancy, high-discrimination fusion features, further limiting the improvement of prediction accuracy. At the same time, existing deep learning-based prediction solutions are essentially data-driven correlation fitting black box models that can only output a single probability of failure occurrence. They cannot distinguish between the real causal dependence and spurious correlation between features and failures. They cannot locate the root cause of failures, nor can they accurately predict the type of failure and remaining service life. It is difficult to provide maintenance personnel with a practical basis for pre-emptive handling.

[0005] Therefore, developing a hard disk fault prediction scheme with multi-dimensional data coverage, adaptive feature fusion, causal root cause localization capability, and high prediction accuracy has become a core technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] The purpose of this invention is to provide a multimodal data fusion and causal inference method and related equipment for hard disk fault prediction, which can effectively improve the accuracy and reliability of hard disk fault prediction.

[0007] To achieve the above objectives, the present invention employs the following technical solution: In a first aspect, the present invention provides a method for multimodal data fusion and causal inference in hard disk fault prediction, comprising: Acquire multimodal data of the hard drive under test; the multimodal data includes four different types of data, namely SMART attribute data, operation log and error code data, data center environment data and physical status data; Single-modal features are extracted from the four different types of data in the multimodal data to obtain the single-modal features corresponding to each of the four different types of data; A multi-head self-attention mechanism is used to perform feature fusion processing on the single-modal features corresponding to four different types of data to obtain fused features; The fused features are input into a pre-trained Bayesian causal model for fault prediction, resulting in a fault prediction result for the hard drive under test. The fault prediction result includes the fault probability, fault type, remaining lifespan, and root cause of the fault. The system visualizes the fault prediction results of the hard drive under test and executes a graded alarm of the corresponding level when the fault probability exceeds the corresponding preset threshold.

[0008] Optionally, before performing single-modal feature extraction on the four different types of data in the multimodal data, a data preprocessing step is also included: performing data cleaning, outlier removal, missing value imputation, and Z-score standardization on the four different types of multimodal data respectively; wherein, missing value imputation adopts linear interpolation or mean imputation, and outlier removal adopts the 3σ criterion.

[0009] Optionally, single-modal feature extraction is performed on the four different types of data in the multimodal data, including: For SMART attribute data, extract the mean, variance, standard deviation, trend slope, number of mutations, and peak time-domain statistical features as the unimodal features corresponding to the SMART attribute data; For the operation logs and error code data, textual semantic features are extracted through word embedding model, and abnormal statistical features such as the frequency of error occurrence, error type distribution and duration of continuous errors are statistically analyzed per unit time, which are used as the single-modal features corresponding to the operation logs and error code data. For the data center environment data and physical status data, the mean, variance, and peak time-domain statistical features are extracted respectively. The energy distribution frequency domain features of different frequency bands are extracted by fast Fourier transform, which serve as the single-mode features corresponding to the data center environment data and the single-mode features corresponding to the physical status data.

[0010] Optionally, a multi-head self-attention mechanism can be used to perform feature fusion processing on the single-modal features corresponding to the four different types of data, including: The single-modal features corresponding to the four different types of data are mapped to the feature space of the same preset dimension through a linear fully connected layer to obtain four single-modal features with consistent dimensions to be fused. The four-dimensional consistent single-modal features to be fused are input into a multi-head self-attention module with a preset number of heads, and a corresponding query vector, key vector and value vector are generated for each attention head. For each attention head, the dot product similarity between the query vector and the key vector under that head is calculated, and the dot product similarity is normalized by scaling the dimensions and then using the Softmax activation function to obtain the fault association attention weights corresponding to each single modality feature under that head. For each attention head, the attention weights corresponding to each single modality feature under that head are weighted and summed with the corresponding value vectors to obtain the modality association weighted features output by that attention head; The modal correlation weighted features output by all attention heads are concatenated by channels, and then the dimensionality is reduced to the preset target dimension through a linear mapping layer to obtain the final fused features.

[0011] Optionally, the pre-training process of the Bayesian causal model includes: Obtain a pre-training sample set, which includes fault labels corresponding to historical multimodal data of hard disk samples; The PC algorithm is used to learn the causal structure of the pre-trained sample set, and the causal dependency between features and faults is screened by the conditional independence test to construct a directed acyclic hard disk fault causal topology network. Under the premise of keeping the hard disk failure causal topology network structure unchanged, based on the pre-trained sample set, the probability parameters of the Bayesian causal model are learned by the maximum likelihood estimation method, and the prior probability distribution of each node and the conditional probability distribution table between nodes are fitted to obtain the pre-trained Bayesian causal model.

[0012] Optionally, when the failure probability exceeds a corresponding preset threshold, a graded alarm of the corresponding level is executed, including: Three sets of continuously increasing fault warning thresholds are preset: low-risk threshold, medium-risk threshold, and high-risk threshold. When the failure probability of the hard drive under test exceeds the low-risk threshold but does not exceed the medium-risk threshold, a level 1 alarm is executed, and the hard drive under test is highlighted in red on the visual interface. When the failure probability of the hard drive under test exceeds the medium risk threshold but does not exceed the high risk threshold, a level 2 alarm is executed, a warning pop-up window appears in the visualization interface, and an email alarm is sent to the operation and maintenance personnel. When the failure probability of the hard drive under test exceeds the high-risk threshold, a level 3 alarm is triggered. The hard drive under test is highlighted in red on the visual interface, and an alarm is sent to the maintenance personnel via SMS and voice call.

[0013] Secondly, the present invention provides a multimodal data fusion and causal inference hard disk fault prediction system, comprising: The data acquisition module is used to acquire multimodal data of the hard drive under test; the multimodal data includes four different types of data, namely SMART attribute data, operation log and error code data, data center environment data and physical status data; The single feature extraction module is used to extract single-modal features from the four different types of data in the multimodal data, and obtain the single-modal features corresponding to each of the four different types of data. The fusion module is used to perform feature fusion processing on the single-modal features corresponding to four different types of data using a multi-head self-attention mechanism to obtain fused features. The fault prediction module is used to input fused features into a pre-trained Bayesian causal model to predict faults and obtain the fault prediction results of the hard drive under test. The fault prediction results include fault probability, fault type, remaining lifespan, and root cause of the fault. The interactive module is used to visualize the fault prediction results of the hard drive under test and to execute the corresponding level of graded alarm when the fault probability exceeds the corresponding preset threshold.

[0014] Thirdly, the present invention provides an electronic device, comprising: a memory and a processor; The memory is used to store programs; The processor is used to execute the program to implement the multimodal data fusion and causal inference hard disk fault prediction method as described in the first aspect.

[0015] Fourthly, the present invention provides a readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, it implements the multimodal data fusion and causal inference hard disk fault prediction method as described in the first aspect.

[0016] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: This invention provides a multimodal data fusion and causal inference method and related equipment for hard disk fault prediction. The method simultaneously acquires four types of multimodal data: hard disk SMART attributes, operation logs and error codes, data center environmental conditions, and physical operating status. This comprehensively covers all dimensions of causes of hard disk failure, including mechanical wear, electronic component failure, and environmental interference. It can comprehensively characterize the real-time health status of the hard disk, effectively capture early weak abnormal signals before a failure occurs, and significantly reduce the probability of false negatives and missed faults from the data source level. Furthermore, this invention employs a multi-head self-attention mechanism to achieve adaptive weighted fusion of multimodal features. It can automatically adjust the contribution weight of different modal features to fault prediction based on the real-time operating conditions of the hard disk, specifically strengthening key features strongly correlated with faults, and simultaneously suppressing the interference of noise and redundant features to generate low-redundancy, high-discrimination data. The fusion of forces significantly improves the accuracy of fault prediction and the generalization ability under different operating conditions from the feature processing level. This invention also builds a Bayesian causal prediction model based on causal structure learning, which breaks through the inherent limitation of existing deep learning black box models that can only perform correlation fitting. It can not only stably output the probability of hard disk failure, but also simultaneously and accurately predict the fault type, remaining service life, and locate the root cause of the failure. It provides maintenance personnel with complete and directly implementable pre-handling basis, while effectively avoiding false alarms caused by false correlations in the data. It can fully meet the core needs of high-reliability operation and maintenance of data centers. The overall solution forms a closed loop from data collection, feature processing, fault prediction to result output. It can be adapted to the operation and maintenance environment of data centers with different brands, different usage scenarios, and large-scale cluster deployments, and has a strong engineering implementation capability. Attached Figure Description

[0017] Figure 1 The diagram shown is a flowchart of a multimodal data fusion and causal inference hard disk fault prediction method in one embodiment of the present invention. Figure 2 The diagram shown is a schematic representation of the structure of a hard disk fault prediction system for multimodal data fusion and causal inference in one embodiment of the present invention. Detailed Implementation

[0018] The present invention will be further described below with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and should not be used to limit the scope of protection of the present invention.

[0019] Example 1

[0020] like Figure 1 As shown in the figure, this invention provides a method for predicting hard disk failures using multimodal data fusion and causal inference, comprising the following steps: In a first aspect, the present invention provides a method for multimodal data fusion and causal inference in hard disk fault prediction, comprising: S1: Acquire multimodal data of the hard drive under test; the multimodal data includes four different types of data, namely SMART attribute data, operation log and error code data, data center environment data and physical status data; S2: Extract single-modal features from the four different types of data in the multimodal data to obtain the single-modal features corresponding to each of the four different types of data; S3: Use a multi-head self-attention mechanism to perform feature fusion processing on the single-modal features corresponding to the four different types of data to obtain fused features; S4: Input the fused features into the pre-trained Bayesian causal model to perform fault prediction, and obtain the fault prediction result of the hard disk under test; the fault prediction result includes fault probability, fault type, remaining lifespan and root cause of fault; S5: Visualize the fault prediction results of the hard drive under test, and execute the corresponding level of graded alarm when the fault probability exceeds the corresponding preset threshold.

[0021] In this embodiment, the multimodal data in step S1 includes SMART attribute data, operation log and error code data, data center environment data, and physical status data; Among them, SMART attribute data includes core hardware health indicators output by the hard drive's built-in self-monitoring and analysis system, including the number of reallocated sectors, seek error rate, and cumulative power-on time of mechanical hard drives, as well as key monitoring items such as wear leveling times, cumulative bad block count, and available reserved space ratio of solid-state drives, which are used to directly reflect the basic health status of the hard drive's core hardware and firmware. The operation logs and error code data include the block device operation logs of the server operating system kernel, the operation records reported by the hard disk firmware, and hardware self-test error codes, including IO read / write exception records, media error codes, communication failure records, etc., used to capture early warning signals of instantaneous hardware anomalies and electronic / interface failures of the hard disk. The data center environment data includes the hardware under test. Data center environmental data, including rack real-time temperature and humidity, vibration amplitude, power supply voltage fluctuations and other operating conditions, are used to reflect the impact of the external operating environment on hard drive lifespan and failure occurrence; Physical status data includes physical operating characteristics collected by the micro sensors attached to the hard drive, including the hard drive surface operating temperature, operating noise, and vibration data of the read / write head and spindle motor. This data is used to directly reflect the real-time operating status of the hard drive's mechanical components and to detect early, subtle anomalies in mechanical failures.

[0022] In this embodiment, before performing single-modal feature extraction on the four different types of data in the multimodal data in step S2, a data preprocessing step is also included: performing data cleaning, outlier removal, missing value imputation, and Z-score standardization on the four different types of multimodal data respectively; wherein, missing value imputation adopts linear interpolation or mean imputation, and outlier removal adopts the 3σ criterion.

[0023] Specifically, the data cleaning process filters out invalid, redundant, and interfering data unrelated to hard drive health, and performs differentiated processing on four types of data. For SMART attribute data, firstly, null value collection records during hard drive offline or power outage periods are filtered out, and invalid values ​​reported by firmware and redundant data collected repeatedly are removed, retaining only complete hardware monitoring data within the normal hard drive operation cycle; for operation logs and error code data, log content unrelated to hard drive hardware operation, such as system services and network scheduling, is filtered out, retaining only valid records related to block device IO anomalies, hard drive firmware errors, and hardware communication failures; for data center environment data and physical status data, timestamp alignment is performed first, and invalid data frames that do not match the SMART attribute data collection cycle are removed to ensure that the time series windows of the four types of data are completely consistent.

[0024] The outlier removal process primarily employs the 3σ criterion, supplemented by verification based on the physical laws governing hard drive hardware operation. Specifically, for each monitoring indicator, its mean μ and standard deviation σ within a 24-hour sliding window are calculated. Values ​​falling outside the [μ-3σ, μ+3σ] interval are identified as outliers and removed. Furthermore, optimizations are made to suit the characteristics of different data: for irreversibly increasing indicators in SMART attribute data, such as the number of reallocated sectors and the cumulative bad block count, an additional trend rationality check is added to remove reverse jump values ​​that do not conform to hardware wear patterns; for log error code data, ultra-high frequency error counts that do not conform to actual operating logic are filtered out; for data center environment and physical status data, invalid extreme values ​​exceeding the sensor's reasonable range are removed to prevent outliers from interfering with the accuracy of subsequent feature extraction.

[0025] In the missing value imputation stage, linear interpolation or mean imputation methods are flexibly matched according to the data type and duration of the missing value to ensure the continuity of time-series data while avoiding the introduction of false fault characteristics. For continuous time-series data such as SMART attributes, data center environment, and physical status, when there is a short-term missing value in a single frame or for no more than 3 consecutive acquisition cycles, linear interpolation is used to fit and generate imputation values ​​that conform to the time-series change trend based on the effective acquisition values ​​before and after the missing time point. For discrete statistical data such as logs and error codes, or for time-series data with long-term continuous missing values ​​exceeding 3 acquisition cycles, mean imputation is used, which fills in the missing value with the mean of the effective data within the 7-day sliding window of the corresponding indicator on the hard drive, to restore the true operating state of the hard drive to the greatest extent.

[0026] Z-score standardization aims to eliminate dimensional differences and numerical magnitude gaps between different indicators, preventing high-magnitude indicators from overshadowing weak fault features during subsequent feature fusion. After standardization, all indicator data are mapped to a unified feature space with a mean of 0 and a standard deviation of 1, providing a unified data foundation for subsequent single-modal feature extraction and multi-head self-attention fusion.

[0027] In this embodiment, step S2 extracts single-modal features from the four different types of data in the multimodal data, including: For SMART attribute data, extract the mean, variance, standard deviation, trend slope, number of mutations, and peak time-domain statistical features as the unimodal features corresponding to the SMART attribute data; For the operation logs and error code data, textual semantic features are extracted through word embedding model, and abnormal statistical features such as the frequency of error occurrence, error type distribution and duration of continuous errors are statistically analyzed per unit time, which are used as the single-modal features corresponding to the operation logs and error code data. For the data center environment data and physical status data, the mean, variance, and peak time-domain statistical features are extracted respectively. The energy distribution frequency domain features of different frequency bands are extracted by fast Fourier transform, which serve as the single-mode features corresponding to the data center environment data and the single-mode features corresponding to the physical status data.

[0028] In this embodiment, step S3 utilizes a multi-head self-attention mechanism to perform feature fusion processing on the single-modal features corresponding to the four different types of data, including: S31: The single-modal features corresponding to the four different types of data are mapped to the feature space of the same preset dimension through a linear fully connected layer to obtain four single-modal features to be fused with consistent dimensions. Specifically, in this embodiment, the original dimensions of the single-modal features corresponding to the four different types of data are significantly different. In order to eliminate the dimension mismatch problem between modalities and ensure the fairness and effectiveness of subsequent attention weight calculation, the preset unified mapping dimension is 64 dimensions. After mapping through a linear fully connected layer, the single-modal features corresponding to the four different types of data are all transformed into a unified feature space of 64 dimensions. This not only fully preserves the original fault discrimination information of each modality, but also achieves complete dimension alignment, thus avoiding the natural suppression of low-dimensional features by high-dimensional features from the root.

[0029] S32: Input the four-dimensional consistent single-modal features to be fused into a multi-head self-attention module with a preset number of heads, and generate corresponding query vector, key vector and value vector for each attention head; Specifically, in this embodiment, based on the core causes of hard drive failures, the multi-head self-attention module is preset to have four heads. Each attention head covers the complete features of all four modalities, and each head is assigned an independent failure mode learning objective to avoid interference between features of different failure types. The learning objectives of the four attention heads are as follows: the first attention head focuses on learning cross-modal correlations related to mechanical failures, the second attention head focuses on learning cross-modal correlations related to electronic and interface failures, the third attention head focuses on learning cross-modal correlations related to environmental causes, and the fourth attention head focuses on learning global failure correlations across all modalities. Each attention head completes vector mapping through an independent linear transformation matrix, and the mapping processes are independent and do not interfere with each other. This allows for the capture of potential failure correlation features between different modal combinations.

[0030] S33: For each attention head, calculate the dot product similarity between the query vector and the key vector under that head, and normalize the dot product similarity by scaling its dimensions and then using the Softmax activation function to obtain the fault association attention weights corresponding to each single modality feature under that head. The dimension scaling operation is implemented by dividing by the square root of the feature dimension. This is primarily to avoid the problem of the softmax activation function's gradient disappearing when the feature dimension is high, as the dot product result is too large. In this embodiment, a minimum weight constraint is added to address the core requirement of early warning of hard disk failures. This prevents the modal weights corresponding to weak early signals strongly correlated with failures from being excessively suppressed, ensuring that the failure information of each modality can be effectively captured. After softmax normalization, the sum of the weights corresponding to the four single-modal features under each head is 1. The weight value directly reflects the contribution of the corresponding modal feature to the failure association learning objective of that head.

[0031] S34: For each attention head, the attention weights corresponding to each single modality feature under that head are weighted and summed with the corresponding value vectors to obtain the modality association weighted features output by that attention head; The core function of this weighting process is to achieve adaptive screening of fault features, enhance the signals of modal features with high correlation to corresponding fault modes, and actively suppress noise and redundant features. For example, in the case of a computer room operating under continuous high temperature conditions, the third attention point, which focuses on the correlation between environmental factors, will automatically increase the weight of the corresponding data between the computer room environment and physical state data, and strengthen the fault features caused by environmental factors. When a large number of sector reallocation anomalies occur on the hard drive, the first attention point, which focuses on the correlation between media and mechanical faults, will automatically increase the weight of the corresponding SMART attribute data, and strengthen the core features of media faults.

[0032] S35: Concatenate the modal correlation weighted features output by all attention heads, and then reduce the dimensionality to the preset target dimension through a linear mapping layer to obtain the final fused features.

[0033] In this embodiment, the 64-dimensional weighted features output by the four attention heads are concatenated through channels to obtain 256-dimensional global correlation features. These features are then reduced to a preset target dimension of 28 dimensions through a linear mapping layer. This dimension perfectly matches the input dimension of the subsequent pre-trained Bayesian causal model, ensuring seamless integration between the feature processing flow and the model inference flow. The final output fused features retain the core fault discrimination information of the four single-modal features while fully exploring the potential fault correlations between modalities. They are characterized by low redundancy and high discriminative power, which can effectively improve the fault prediction accuracy and root cause localization accuracy of the subsequent Bayesian causal model.

[0034] In this embodiment, the pre-training process of the Bayesian causal model includes: Obtain a pre-training sample set, which includes fault labels corresponding to historical multimodal data of hard disk samples; The PC algorithm (Peter-Clark Algorithm) is used to learn the causal structure of the pre-trained sample set, and the causal dependency between features and faults is screened through conditional independence test to construct a directed acyclic hard disk fault causal topology network. Specifically, in this embodiment, causal structure learning uses historical fusion features obtained from a pre-trained sample set as the sole input. First, it initializes a fully connected undirected graph with each dimension of the fusion features as feature nodes and the fault label as the target node. Then, through progressive conditional independence tests from order 0 to higher orders, it gradually removes nodes with no causal dependencies, thus completing the undirected graph pruning. Finally, it determines the causal direction based on V-structure rules, generating a directed acyclic hard disk fault causal topology network that retains only the true causal associations of faults, serving as an unmodifiable fixed skeleton for the Bayesian causal model.

[0035] Under the premise of keeping the hard disk failure causal topology network structure unchanged, based on the pre-trained sample set, the probability parameters of the Bayesian causal model are learned by the maximum likelihood estimation method, and the prior probability distribution of each node and the conditional probability distribution table between nodes are fitted to obtain the pre-trained Bayesian causal model.

[0036] In this embodiment, step S5 executes a graded alarm of the corresponding level when the failure probability exceeds the corresponding preset threshold, including: Three sets of continuously increasing fault warning thresholds are preset: low-risk threshold, medium-risk threshold, and high-risk threshold. When the failure probability of the hard drive under test exceeds the low-risk threshold but does not exceed the medium-risk threshold, a level 1 alarm is executed, and the hard drive under test is highlighted in red on the visual interface. When the failure probability of the hard drive under test exceeds the medium risk threshold but does not exceed the high risk threshold, a level 2 alarm is executed, a warning pop-up window appears in the visualization interface, and an email alarm is sent to the operation and maintenance personnel. When the failure probability of the hard drive under test exceeds the high-risk threshold, a level 3 alarm is triggered. The hard drive under test is highlighted in red on the visual interface, and an alarm is sent to the maintenance personnel via SMS and voice call.

[0037] Example 2

[0038] like Figure 2 As shown in the illustration, this invention describes a system for implementing the multimodal data fusion and causal inference hard disk fault prediction method provided in Embodiment 1, comprising: The data acquisition module is used to acquire multimodal data of the hard drive under test; the multimodal data includes four different types of data, namely SMART attribute data, operation log and error code data, data center environment data and physical status data; The single feature extraction module is used to extract single-modal features from the four different types of data in the multimodal data, and obtain the single-modal features corresponding to each of the four different types of data. The fusion module is used to perform feature fusion processing on the single-modal features corresponding to four different types of data using a multi-head self-attention mechanism to obtain fused features. The fault prediction module is used to input fused features into a pre-trained Bayesian causal model to predict faults and obtain the fault prediction results of the hard drive under test. The fault prediction results include fault probability, fault type, remaining lifespan, and root cause of the fault. The interactive module is used to visualize the fault prediction results of the hard drive under test and to execute the corresponding level of graded alarm when the fault probability exceeds the corresponding preset threshold.

[0039] The specific functions of each module described above are explained in the relevant content of the method in Embodiment 1, and will not be repeated here.

[0040] Example 3

[0041] This embodiment describes an electronic device, including: a memory and a processor; The memory is used to store programs; The processor is used to execute the program to implement the multimodal data fusion and causal inference hard disk fault prediction method in Embodiment 1.

[0042] Example 4

[0043] This embodiment describes a readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements the multimodal data fusion and causal inference hard disk fault prediction method in Embodiment 1.

[0044] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0045] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0046] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0047] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A multimodal data fusion and causal inference method for hard disk failure prediction, characterized in that, include: Acquire multimodal data of the hard drive under test; the multimodal data includes four different types of data, namely SMART attribute data, operation log and error code data, data center environment data and physical status data; Single-modal features are extracted from the four different types of data in the multimodal data to obtain the single-modal features corresponding to each of the four different types of data; A multi-head self-attention mechanism is used to perform feature fusion processing on the single-modal features corresponding to four different types of data to obtain fused features; The fused features are input into a pre-trained Bayesian causal model for fault prediction, resulting in a fault prediction result for the hard drive under test. The fault prediction result includes the fault probability, fault type, remaining lifespan, and root cause of the fault. The system visualizes the fault prediction results of the hard drive under test and executes a graded alarm of the corresponding level when the fault probability exceeds the corresponding preset threshold.

2. The multimodal data fusion and causal inference hard disk failure prediction method according to claim 1, characterized in that, Before performing single-modal feature extraction on the four different types of data in the multimodal data, a data preprocessing step is also included: data cleaning, outlier removal, missing value imputation, and Z-score standardization are performed on the four different types of multimodal data respectively; among which, missing value imputation adopts linear interpolation or mean imputation, and outlier removal adopts the 3σ criterion.

3. The multimodal data fusion and causal inference hard disk failure prediction method according to claim 2, characterized in that, Single-modal feature extraction is performed on the four different types of data in the multimodal data, including: For SMART attribute data, extract the mean, variance, standard deviation, trend slope, number of mutations, and peak time-domain statistical features as the unimodal features corresponding to the SMART attribute data; For the operation logs and error code data, textual semantic features are extracted through word embedding model, and abnormal statistical features such as the frequency of error occurrence, error type distribution and duration of continuous errors are statistically analyzed per unit time, which are used as the single-modal features corresponding to the operation logs and error code data. For the data center environment data and physical status data, the mean, variance, and peak time-domain statistical features are extracted respectively. The energy distribution frequency domain features of different frequency bands are extracted by fast Fourier transform, which serve as the single-mode features corresponding to the data center environment data and the single-mode features corresponding to the physical status data.

4. The multimodal data fusion and causal inference hard disk failure prediction method according to claim 3, characterized in that, A multi-head self-attention mechanism is used to perform feature fusion processing on the single-modal features corresponding to four different types of data, including: The single-modal features corresponding to the four different types of data are mapped to the feature space of the same preset dimension through a linear fully connected layer to obtain four single-modal features with consistent dimensions to be fused. The four-dimensional consistent single-modal features to be fused are input into a multi-head self-attention module with a preset number of heads, and a corresponding query vector, key vector and value vector are generated for each attention head. For each attention head, the dot product similarity between the query vector and the key vector under that head is calculated, and the dot product similarity is normalized by scaling the dimensions and then using the Softmax activation function to obtain the fault association attention weights corresponding to each single modality feature under that head. For each attention head, the attention weights corresponding to each single modality feature under that head are weighted and summed with the corresponding value vectors to obtain the modality association weighted features output by that attention head; The modal correlation weighted features output by all attention heads are concatenated by channels, and then the dimensionality is reduced to the preset target dimension through a linear mapping layer to obtain the final fused features.

5. The multimodal data fusion and causal inference hard disk failure prediction method according to claim 4, characterized in that, The pre-training process of the Bayesian causal model includes: Obtain a pre-training sample set, which includes fault labels corresponding to historical multimodal data of hard disk samples; The PC algorithm is used to learn the causal structure of the pre-trained sample set, and the causal dependency between features and faults is screened by the conditional independence test to construct a directed acyclic hard disk fault causal topology network. Under the premise of keeping the hard disk failure causal topology network structure unchanged, based on the pre-trained sample set, the probability parameters of the Bayesian causal model are learned by the maximum likelihood estimation method, and the prior probability distribution of each node and the conditional probability distribution table between nodes are fitted to obtain the pre-trained Bayesian causal model.

6. The multimodal data fusion and causal inference hard disk failure prediction method according to claim 5, characterized in that, When the probability of a failure exceeds a preset threshold, a tiered alarm of the corresponding level is executed, including: Three sets of continuously increasing fault warning thresholds are preset: low-risk threshold, medium-risk threshold, and high-risk threshold. When the failure probability of the hard drive under test exceeds the low-risk threshold but does not exceed the medium-risk threshold, a level 1 alarm is executed, and the hard drive under test is highlighted in red on the visual interface. When the failure probability of the hard drive under test exceeds the medium risk threshold but does not exceed the high risk threshold, a level 2 alarm is executed, a warning pop-up window appears in the visualization interface, and an email alarm is sent to the operation and maintenance personnel. When the failure probability of the hard drive under test exceeds the high-risk threshold, a level 3 alarm is triggered. The hard drive under test is highlighted in red on the visual interface, and an alarm is sent to the maintenance personnel via SMS and voice call.

7. A multimodal data fusion and causal inference hard disk failure prediction system, characterized in that, include: The data acquisition module is used to acquire multimodal data of the hard drive under test; the multimodal data includes four different types of data, namely SMART attribute data, operation log and error code data, data center environment data and physical status data; The single feature extraction module is used to extract single-modal features from the four different types of data in the multimodal data, and obtain the single-modal features corresponding to each of the four different types of data. The fusion module is used to perform feature fusion processing on the single-modal features corresponding to four different types of data using a multi-head self-attention mechanism to obtain fused features. The fault prediction module is used to input the fused features into the pre-trained Bayesian causal model to predict faults and obtain the fault prediction results of the hard drive under test. The fault prediction results include fault probability, fault type, remaining lifespan, and root cause of the fault. The interactive module is used to visualize the fault prediction results of the hard drive under test and to execute the corresponding level of alarm when the fault probability exceeds the corresponding preset threshold.

8. An electronic device, characterized in that, include: Memory and processor; The memory is used to store programs; The processor is used to execute the program to implement the multimodal data fusion and causal inference hard disk fault prediction method as described in any one of claims 1-6.

9. A readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the multimodal data fusion and causal inference hard disk failure prediction method as described in any one of claims 1-6.

Images