Data reconstruction methods and electronic devices

Abstract

This application discloses a data reconstruction method and electronic device, relating to the field of computer technology. The method includes determining the current disk's operating status information and the access popularity information of multiple data blocks within the current disk. The operating status information includes a status variable indicating at least one current physical state of the current disk. Based on the operating status information and access popularity information, a risk assessment value for the current disk is determined. If the risk assessment value of the current disk exceeds a fault warning threshold, a target dataset from at least one target data block in the current disk is written to persistent memory. If a fault is detected in the current disk, the faulty data in the current disk is reconstructed based on the target dataset obtained from persistent memory. This method solves the technical problem of low data reconstruction efficiency in existing technologies.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more particularly to a data reconstruction method and an electronic device. Background Technology

[0002] In the modern data storage field, Redundant Array of Independent Disks (RAID) technology is widely used to improve data reliability and storage performance. RAID achieves data fault tolerance and accelerated access by distributing and redundant data across multiple disks. However, with the continuous expansion of storage system scale and the surge in data volume, the potential risk of disk failure also increases, making RAID reconstruction a critical step in maintaining data integrity and system availability. During storage system use, minimizing the impact of RAID reconstruction on storage system performance, reducing RAID reconstruction recovery time, and avoiding disruption to core business continuity have become key pain points in the storage system reconstruction process.

[0003] However, existing technologies can only take passive measures after a failure occurs. The reconstruction process is highly dependent on the processor's computing power, resulting in a large amount of CPU resources being consumed during large-scale data recovery, which seriously affects the system's normal I / O processing capabilities. In other words, existing technologies suffer from low data reconstruction efficiency. Summary of the Invention

[0004] This application provides a data reconstruction method and an electronic device to at least solve the technical problem of low data reconstruction efficiency in the prior art.

[0005] This application provides a data reconstruction method, comprising: determining the current disk's operating status information and the access frequency information of multiple data blocks in the current disk, wherein the operating status information includes a status quantity indicating at least one current physical state of the current disk; determining a risk assessment value for the current disk based on the operating status information and the access frequency information; if the risk assessment value of the current disk is greater than a fault warning threshold, writing a target dataset from at least one target data block in the current disk into persistent memory, wherein the fault warning threshold is determined based on the operating status of the current disk at a reference time, the reference time being the time when the current disk fails, the access frequency information of the target data block meets the filtering conditions, and the access speed to data in persistent memory is greater than the access speed to data on the disk; and if a current disk failure is detected, reconstructing the faulty data in the current disk based on the target dataset obtained from persistent memory.

[0006] This application also provides a data reconstruction apparatus, comprising: a first determining unit for determining the current disk's operating status information and the access popularity information of multiple data blocks in the current disk, wherein the operating status information includes a status quantity for indicating at least one current physical state of the current disk; a second determining unit for determining a risk assessment value of the current disk based on the operating status information and the access popularity information; a data read / write unit for writing a target dataset from at least one target data block in the current disk into persistent memory when the risk assessment value of the current disk is greater than a fault warning threshold, wherein the fault warning threshold is determined based on the operating status of the current disk at a reference time, the reference time being the time when the current disk fails, the access popularity information of the target data block meets the filtering conditions, and the access speed to data in persistent memory is greater than the access speed to data in the disk; and a reconstruction unit for reconstructing the faulty data in the current disk based on the target dataset obtained from persistent memory when a current disk failure is detected.

[0007] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for implementing the steps of any of the above-described data reconstruction methods when executing the computer program.

[0008] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-described data reconstruction methods.

[0009] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described data reconstruction methods.

[0010] This application determines the current disk's operating status information and the access frequency information of multiple data blocks within the disk. The operating status information includes a status variable indicating at least one current physical state of the disk. Based on the operating status information and access frequency information, a risk assessment value for the current disk is determined. By combining the operating status and access frequency information, the disk's failure risk is quantified, thereby enabling the prediction of disk failures. If the current disk's risk assessment value exceeds a failure warning threshold, the target dataset from at least one target data block on the current disk is written to persistent memory. The failure warning threshold is determined based on the current disk's operating status at a reference time, where the reference time is the moment the disk fails. The access frequency information of the target data block meets the filtering criteria, meaning the access speed to data in persistent memory is greater than the access speed to data on the disk. The failure warning threshold can be dynamically adjusted to improve the accuracy and timeliness of warnings, allowing preventative measures to be taken before a disk failure occurs, reducing business interruption time. Thus, upon detecting a disk failure, the faulty data on the current disk is reconstructed based on the target dataset obtained from persistent memory, and recovery is performed using pre-stored hot data, reducing data reconstruction time and solving the technical problem of low data reconstruction efficiency in existing technologies. Attached Figure Description

[0011] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A hardware structure block diagram of a server device for a data reconstruction method provided in this application embodiment;

[0013] Figure 2 A flowchart illustrating a data reconstruction method provided in this application embodiment;

[0014] Figure 3 A flowchart illustrating another data reconstruction method provided in this application embodiment;

[0015] Figure 4 This is a schematic diagram of an abnormal feature type provided in an embodiment of this application;

[0016] Figure 5 A flowchart illustrating yet another data reconstruction method provided in this application embodiment;

[0017] Figure 6 This is a schematic diagram of a data reconstruction apparatus according to an embodiment of this application;

[0018] Figure 7 This is a schematic diagram of the structure of an electronic device for data reconstruction according to an embodiment of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0020] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0021] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0022] The methods and embodiments provided in this application can be executed on a server device or a similar computing device. Taking running on a server device as an example, Figure 1 This is a hardware structure block diagram of a server device for a data reconstruction method according to an embodiment of this application. For example... Figure 1 As shown, the server device may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 (which may include, but is not limited to, a central processing unit (CPU), a microprocessor (MCU), or a programmable logic device (FPGA)) and a memory 104 for storing data are also shown. The server device may further include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the server equipment described above. For example, the server equipment may also include components that are more... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0023] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the data reconstruction method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thus implementing the above-described method. The memory 104 may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to server devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0024] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider for the server device. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0025] The embodiments of this application provide a data reconstruction method, and the method is described in detail below in conjunction with the execution flow of the data reconstruction method.

[0026] To better understand the technical solutions provided in the embodiments of this application, the key terms involved in the embodiments of this application will be introduced first:

[0027] Redundant Array of Independent Disks (RAID).

[0028] Input and Output (IO).

[0029] Self-Monitoring, Analysis, and Reporting Technology (SMART).

[0030] Logical Block Addressing (LBA).

[0031] Long Short-Term Memory Network (LSTM).

[0032] Convolutional Neural Network (CNN).

[0033] Persistent memory (PMem).

[0034] Fourier Transform (FFT).

[0035] Parity check (Exclusive OR, XOR).

[0036] As an optional implementation method, such as Figure 2 As shown, the above data reconstruction method includes:

[0037] S202, determine the current disk's operating status information and the access popularity information of each of the multiple data blocks in the current disk, wherein the operating status information includes a status quantity used to indicate at least one current physical state of the current disk;

[0038] S204. Determine the current risk assessment value of the disk based on the running status information and access popularity information;

[0039] S206, if the current disk risk assessment value is greater than the fault warning threshold, write the target dataset in at least one target data block in the current disk into persistent memory. The fault warning threshold is determined based on the current disk's operating status at a reference time, where the reference time is the moment when the current disk fails. The access popularity information of the target data block meets the filtering conditions, and the access speed to the data in persistent memory is greater than the access speed to the data on the disk.

[0040] S208, if a disk failure is detected, the faulty data in the current disk is reconstructed based on the target dataset obtained from persistent memory.

[0041] As an optional implementation, in step S202, the current disk's operating status information and the access popularity information of each of the multiple data blocks in the current disk are determined, wherein the operating status information includes a status quantity used to indicate at least one current physical state of the current disk.

[0042] Optionally, the above-mentioned operating status information includes the current physical status of the disk, such as temperature, vibration level, SMART indicators (such as remapped sector count, bad sector count, etc.), and disk performance indicators such as I / O latency, I / O operation frequency, etc.; the above-mentioned access heat information is used to reflect the frequency of data block access.

[0043] In step S204 above, the risk assessment value of the current disk is determined based on the running status information and access popularity information.

[0044] Optionally, the risk score of disk D_k is Risk_Score = W1 × LSTM_output + W2 × CNN_heatmap_score + W3 × vibration_FFT + W4 × temp_gradient, where W1, W2, W3, and W4 are the weights of each predicted value; LSTM_output is the predicted failure probability value based on time-series SMART data, including but not limited to multiple parameters collected by the disk's built-in SMART sensors, such as remapped sector count, seek error rate, and uncorrectable error count; CNN_heatmap_score represents the abnormal distribution score based on the I / O access heatmap, which is constructed by using the LBA access logs recorded by the storage controller, read / write operation frequency, and access latency distribution to construct the load heatmap, and obtained by converting the heatmap into a three-channel image (RGB = read frequency / write frequency / latency) and inputting it into a convolutional neural network (CNN) for feature extraction; vibration_FFT represents the intensity of fault features in the mechanical vibration spectrum, vibration_FFT = current frequency domain energy / total energy of the entire frequency band; and temp_gradient represents the severity of disk temperature changes.

[0045] In step S206 above, if the risk assessment value of the current disk is greater than the fault warning threshold, the target dataset in at least one target data block in the current disk is written into persistent memory. The fault warning threshold is determined based on the operating status of the current disk at a reference time, where the reference time is the time when the current disk fails. The access popularity information of the target data block meets the filtering conditions, and the access speed to the data in persistent memory is greater than the access speed to the data on the disk.

[0046] Optionally, when the risk score of disk D_k is greater than N, the system will initiate a data migration process to identify the target dataset in the target data block, that is, those data blocks that show high read / write frequency and low latency in the heatmap. The datasets in these hot data blocks will be asynchronously written to persistent memory, and the verification data in persistent memory will be updated to ensure data consistency and integrity.

[0047] It is understood that the aforementioned persistent memory (such as PMem) is a non-volatile storage technology that combines the speed of memory with the persistence of disk, making data access faster than traditional disk storage; the target dataset in the aforementioned target data block is critical business data or frequently accessed metadata; the aforementioned filtering conditions can be conditions used to determine hot data, such as read / write frequency greater than a preset threshold, and no specific restrictions are imposed here.

[0048] In step S208 above, if a current disk failure is detected, the faulty data in the current disk is reconstructed based on the target dataset obtained from persistent memory.

[0049] Optionally, the aforementioned data reconstruction involves recovering lost data based on parity data and hot data stored in persistent memory after a disk failure is detected. Specifically, when a disk D_k is detected to be faulty, the storage system immediately reads the hot data and parity data related to the faulty disk from persistent memory, processes them using the XOR parity module, reconstructs the lost data blocks, and writes the reconstructed data to a hot spare disk or replaces the corresponding location on the disk. It should be noted that when the system detects data updates, it will synchronously update the parity data in persistent memory to reflect the status of the new disk, maintaining RAID redundancy and consistency.

[0050] The above-described embodiments of this application determine the current disk's operating status information and the access frequency information of multiple data blocks on the current disk. The operating status information includes a status variable indicating at least one current physical state of the current disk. Based on the operating status information and access frequency information, a risk assessment value for the current disk is determined. By combining the operating status and access frequency information, the disk's failure risk is quantified, thereby enabling the prediction of disk failures. If the current disk's risk assessment value is greater than a failure warning threshold, the target dataset from at least one target data block on the current disk is written into persistent memory. The failure warning threshold is determined based on the current disk's operating status at a reference time, where the reference time is the moment the current disk fails. The access frequency information of the target data block meets the filtering conditions, and the access speed to data in persistent memory is greater than the access speed to data on the disk. The failure warning threshold can be dynamically adjusted to improve the accuracy and timeliness of the warning. Preventive measures are taken before a disk failure occurs, reducing business interruption time. Thus, when a disk failure is detected, the faulty data on the current disk is reconstructed based on the target dataset obtained from persistent memory, and recovery is performed using pre-stored hot data, reducing the data reconstruction time and solving the technical problem of low data reconstruction efficiency in the prior art.

[0051] In one optional implementation, determining the current disk's operating status information includes:

[0052] S1-1, Based on the temperature information collected by the temperature sensor on the disk during the target detection cycle, determine the temperature change information indicated by the current disk's operating status information;

[0053] S1-2, Based on the disk vibration information collected by the accelerometer on the disk during the target detection cycle, determine the energy distribution information indicated by the current disk operating status information;

[0054] S1-3 determines multiple disk self-test index values ​​indicating the current disk's operating status information within the target detection cycle.

[0055] In step S1-1 above, the temperature change information indicated by the current disk operating status information is determined based on the temperature information collected by the temperature sensor on the disk during the target detection cycle. Optionally, the system periodically receives temperature data from the temperature sensor on the disk, such as once every 10 seconds, and obtains the above-mentioned temperature change information, such as the temperature change rate or temperature fluctuation range, by analyzing the time-series temperature data.

[0056] In steps S1-2 above, the energy distribution information indicated by the current disk operating status information is determined based on the disk vibration information collected by the accelerometer on the disk during the target detection period; optionally, vibration data is acquired from the accelerometer on the disk during the detection period, the time domain signal of the vibration is collected, and further converted into a frequency domain signal by Fast Fourier Transform (FFT) to analyze the energy distribution.

[0057] In steps S1-3 above, multiple disk self-test indicator values ​​are determined based on the current disk operating status information within the target detection period; the disk self-test indicator values ​​are various indicator values ​​generated by the disk self-test mechanism (such as SMART); optionally, the disk's SMART data is read within the target detection period, and the SMART data includes a series of indicators reflecting the disk's health status, such as remapped sector count, bad sector count, seek error rate, etc.

[0058] By regularly monitoring temperature changes, potential thermal stress problems on the disk can be identified. Disk vibration information reflects the stability of the disk during operation. By analyzing energy distribution information, it is possible to determine whether there are problems such as mechanical damage, loose components, or bearing wear on the disk. Monitoring SMART data allows the system to not only react to the immediate physical state, but also to provide early warnings of changes in the long-term health status of the disk. Multimodal physical state monitoring can reveal the health status of the disk from different perspectives, improve the accuracy of fault early warning, and proactively migrate thermal data to persistent memory. This ensures that data reconstruction can be performed quickly after a disk failure, shortening the reconstruction time and improving the efficiency of data reconstruction.

[0059] In one optional implementation, after determining the current disk's operating status information, the process includes:

[0060] S2-1, Determine the current temperature change rate of the disk based on temperature change information;

[0061] S2-2, Determine the current vibration intensity value of the disk based on energy distribution information;

[0062] S2-3, Determine the current disk health prediction value based on multiple disk self-test indicator values;

[0063] S2-4, based on the temperature change rate, vibration intensity value, and health prediction value, determine the current disk operation risk prediction value.

[0064] In step S2-1 above, the current temperature change rate of the disk is determined based on the temperature change information. The system acquires continuous temperature readings from the disk temperature sensor and then calculates the temperature change rate between two readings. Taking data acquisition every 10 seconds as an example, the temperature change rate can be calculated as follows:

[0065]

[0066] Furthermore, the scoring formula is as follows, which allows for the determination of the corresponding evaluation score based on the range to which the temperature change rate falls:

[0067]

[0068] The segmented scoring can be dynamically adjusted based on the actual situation.

[0069] In step S2-2 above, the vibration intensity value of the current disk is determined based on the energy distribution information; the vibration time-domain signal collected by the accelerometer is converted into a frequency-domain signal by Fast Fourier Transform (FFT) to obtain the frequency and energy distribution map. The calculation formula is: vibration_FFT = current frequency domain energy / total energy of the entire frequency band.

[0070] In steps S2-3 above, the current disk health prediction value is determined based on multiple disk self-test index values; optionally, the disk's SMART data is read, including but not limited to remapped sector count, bad sector count, seek error rate, uncorrectable error count, etc., and machine learning models (such as random forest, support vector machine) are used to predict the disk's health status.

[0071] In steps S2-4 above, the predicted operational risk value of the disk is determined based on the temperature change rate, vibration intensity value, and health prediction value. For example, the predicted operational risk value of the disk can be calculated by weighting and summing the assessment score corresponding to the temperature change rate, the vibration intensity value, and the disk's health prediction value.

[0072] By monitoring the rate of temperature change, disk thermal stability issues can be detected in a timely manner, providing a basis for early warning of potential failures. By analyzing the spectrum of disk vibration signals, the vibration patterns and intensities of the disk can be revealed. Abnormal vibration intensity values ​​may indicate problems with the disk's mechanical structure, such as loose parts or worn bearings. By analyzing self-test indicators such as SMART data, the long-term health trend of the disk can be assessed, and potential disk failure risks can be predicted. Furthermore, by integrating temperature changes, vibration intensity, and disk health indicators, the risk prediction value can provide a comprehensive disk status assessment result. This enables intelligent judgment of the disk's health status and failure probability, thereby proactively migrating hot data to the persistent memory resource pool when the risk is high, ensuring that data can be quickly retrieved from persistent memory for data reconstruction in the event of a failure.

[0073] In one optional implementation, determining the access frequency information of each of the multiple data blocks in the current disk further includes:

[0074] S3-1, Based on the current disk's log data during the target detection cycle, determine the read frequency of performing data read operations on multiple data blocks as indicated by the current disk's access heat information;

[0075] S3-2, Based on the current disk's log data during the target detection period, determine the write frequency of performing data write operations on multiple data blocks as indicated by the current disk's access heat information;

[0076] S3-3, determine the data read latency and data write latency of each of the multiple data blocks indicated by the current disk access heat information within the target detection period.

[0077] Optionally, in step S3-1 above, log data of all data reading operations within the target detection period can be collected, and for each data block, the reading frequency can be obtained based on the number of reading operations.

[0078] Optionally, in step S3-2 above, log data of all data write operations within the target detection period can be collected, and for each data block, the write frequency can be obtained based on the number of write operations.

[0079] Optionally, in step S3-3 above, by obtaining log data of all read and write operations within the detection period, including the start and end times of the operations, for each data block, the average read and write latency within the target detection period is calculated. For example, the read latency of a data block can be determined by the average difference of the read timestamps. For each data block, the time span of all read and write operations within 1 hour can also be obtained to calculate the average latency, without any specific restrictions.

[0080] By analyzing read and write frequencies, we can identify which data blocks are hot data, i.e., those data blocks that are frequently accessed. Hot data is pre-stored in persistent memory for quick recovery in the event of disk failure, thereby significantly shortening the time required for data reconstruction. Analyzing latency can identify which data blocks have inefficient read and write operations, which may be due to disk aging or physical damage. These inefficient data blocks are marked as key monitoring targets.

[0081] In one optional implementation, after determining the access popularity information of each of the multiple data blocks in the current disk, the method further includes:

[0082] S4-1, If ​​the read frequency or write frequency of the target data block is greater than the first threshold, determine the hot spot anomaly information of the target data block;

[0083] S4-2, If the data read delay or data write delay of the target data block exceeds the second threshold, determine the delay anomaly information of the target data block;

[0084] S4-3, when the read frequency or write frequency of the target data block increases from a low frequency state to a high frequency state during the target detection cycle, determine the cold data mutation information of the target data block;

[0085] S4-4, under the condition that the target difference condition is met at multiple time intervals within the target detection period, the stripe imbalance information of the target data block is determined based on the read frequency or write frequency between the target data block and the reference data block. Here, the target data block and the reference data block are two adjacent data blocks in the current disk.

[0086] S4-5, determine the predicted value of the current disk access anomaly based on the hot spot anomaly information, latency anomaly information, cold data mutation information and stripe imbalance information of the target data block.

[0087] Optionally, in step S4-1 above, if the read frequency or write frequency of the target data block is greater than the first threshold, hot spot anomaly information of the target data block is determined; hot spot anomaly information is the abnormal situation identified by the system when the read or write frequency of the data block is abnormally high, which is a hot data phenomenon caused by frequent data access.

[0088] Optionally, in step S4-2 above, if the data read latency or data write latency of the target data block exceeds the second threshold, the latency anomaly information of the target data block is determined; the latency anomaly information reflects an abnormally high data block access latency, indicating that there may be a data read / write bottleneck or disk performance problem.

[0089] Optionally, in step S4-3 above, when the read frequency or write frequency of the target data block increases from a low frequency state to a high frequency state within the target detection period, the cold data mutation information of the target data block is determined; the cold data mutation information is detailed information of the cold data mutation when the data block suddenly changes from a less accessed state to a highly active state, including the LBA of the data block, the frequency values ​​before and after the mutation, and the mutation time.

[0090] Optionally, in step S4-4 above, if the read frequency or write frequency between the target data block and the reference data block meets the target difference condition at multiple time intervals within the target detection period, the stripe imbalance information of the target data block is determined. Here, the target data block and the reference data block are two adjacent data blocks in the current disk. The stripe imbalance information reflects the uneven distribution of RAID stripe data. By comparing the read and write frequencies of the target data block and the adjacent reference data block, if the frequency difference exceeds the target difference condition at multiple time intervals, the stripe imbalance information is identified and recorded. This stripe imbalance information may include the LBA range, frequency difference, and imbalance duration of the target data block and the reference data block.

[0091] Optionally, in steps S4-5 above, the predicted value of the current disk access anomaly is determined based on the hot spot anomaly information, latency anomaly information, cold data mutation information and stripe imbalance information of the target data block; for example, the predicted value of the current disk access anomaly is determined by comprehensively evaluating the above anomaly information (such as by weighted summation calculation).

[0092] The above analysis reveals that hotspot anomalies indicate high-frequency access patterns of data blocks, suggesting that the disk areas containing these blocks are likely under high load, increasing the risk of physical wear and potential failures. Latency anomalies indicate that read and write operations on data blocks are slower than normal, potentially indicating internal disk problems (such as head positioning errors or disk surface defects). Cold data mutations refer to a sudden surge in access to data blocks that were previously accessed infrequently. By monitoring cold data mutations, the system can predict and mitigate the impact of such sudden loads. Stripe imbalance anomalies mean that data is unevenly distributed across disk stripes, with some disks or disk areas experiencing significantly higher loads than others, leading to performance bottlenecks and a higher risk of failure. Analyzing these anomalies can guide the system to take targeted preventative measures, such as data migration, load balancing, disk cooling, or adding backup resources. This not only helps predict and prevent failures and implement preventative measures but also optimizes data access patterns during normal operation, avoiding performance bottlenecks.

[0093] In one optional implementation, the risk assessment value of the current disk is determined based on the operating status information and access frequency information, including:

[0094] S5-1, determine the predicted value of operating risk based on the current disk operating status information;

[0095] S5-2. Determine the current disk risk assessment value based on the weighted sum of the current disk operation risk prediction value and access anomaly prediction value.

[0096] In step S5-1 above, the risk of disk failure can be predicted by a machine learning model. The input of the model may include, but is not limited to, multiple parameters such as temperature change rate, vibration intensity value and SMART index value. The output is the disk operation risk prediction value. The higher the operation risk prediction value, the higher the risk of potential failure indicated by the physical state of the disk.

[0097] In step S5-2 above, the weights of the operational risk prediction value and the access anomaly prediction value are set according to the contribution of the disk's physical state and data access pattern to the disk risk. For example, the weight of the operational risk prediction value is (w1), and the weight of the access anomaly prediction value is (w2). The operational risk prediction value and the access anomaly prediction value are multiplied by their respective weights and summed to obtain the disk risk assessment value.

[0098] By comprehensively analyzing operational status information and access anomaly information, the system can more accurately predict disk failures, avoid the limitations of single-indicator early warning, and improve the comprehensiveness and accuracy of early warnings. Based on intelligent prediction, preventive measures such as data backup and migration can be taken in advance, reducing the data reconstruction time after disk failure and ensuring business continuity and rapid data recovery.

[0099] In one alternative implementation, before writing the target dataset from at least one target data block in the current disk to persistent memory, the following steps are included:

[0100] S6-1, Determine the target operating environment based on the current disk data access mode information, wherein the data access mode is used to indicate the method of performing data read or data write operations on the current disk;

[0101] S6-2, Determine the reference operating status information of the current disk in the target operating environment at the reference time, and the reference access hotness information of the data blocks in the current disk;

[0102] S6-3, determine the fault warning threshold based on the reference operating status information and reference access popularity information.

[0103] Optionally, in step S6-1 above, read and write operations on the disk are monitored, and data access pattern characteristics are analyzed to determine the current target operating environment of the disk. Data access patterns include, but are not limited to, random access, continuous access, mixed access, hot access, or cold access. Furthermore, system resource allocation and optimization strategies can be determined based on the access patterns.

[0104] For example, approximately 70% of read operations and 30% of write operations are concentrated in the LBA 0-20GB range, while the access frequency of other LBAs is extremely low. In read operations, sequential reads account for 60% of the total read operations, while random reads account for 40%. In write operations, random writes account for 80% of the total write operations. Analysis determines that the current disk is in the ideal operating environment of the "hotspot mixed access" mode (i.e., the optimal operating environment for executing the "hotspot mixed access" mode). The "hotspot mixed access" mode is characterized by the existence of hotspot access areas, which simultaneously mix sequential and random access modes within these areas.

[0105] Optionally, in steps S6-2 to S6-3 above, the reference time can be selected as the same time point on the same day a week ago, when the disk's operation is similar to the current "hotspot mixed access" mode. At the reference time, the disk's operating status in the target operating environment and the access heat information of the data blocks in the current disk can be obtained. The operating status includes operating status information such as temperature, vibration, and SMART value, as well as access heat information such as read / write frequency and latency. The access heat information of representative data blocks can be selected, such as those with specific attributes or uses. No specific restrictions are imposed here. The fault warning threshold can be further determined based on the above information.

[0106] For example, historical data shows that in an ideal operating environment under "continuous access" mode, the average temperature change rate is 1°C / min. Therefore, the fault warning threshold can be set to a temperature change rate of 2°C / min. The fault warning threshold can be adjusted periodically or automatically according to changes in disk access mode and operating environment to ensure that the threshold setting matches the actual situation.

[0107] By identifying the target operating environment, the system can intelligently match current load demands and resource allocation. For example, it can pre-identify and optimize the distribution of hot data, reduce IO bottlenecks during data reconstruction, and thus improve data reconstruction efficiency. Based on historical data reference information, a model of normal operating conditions can be established, thereby identifying abnormal access patterns that deviate from the normal state. Furthermore, the fault warning threshold is a dynamically adjusted indicator that can adapt to different operating environments and data loads. This enables the system to intelligently monitor the health status of the disk and data access patterns. When the actual operating status or access intensity of the disk exceeds the warning threshold, measures can be taken in advance, such as preventive data migration, adding backups, or initiating early data reconstruction. This avoids the need for a complete rebuild process from scratch when the disk fails, thereby significantly reducing data reconstruction time.

[0108] In one alternative implementation, determining the target dataset in at least one target data block in the current disk includes at least one of the following:

[0109] Method 1: Determine the read frequency of each of the multiple data blocks, construct a data read heatmap of the current disk based on the read frequency, determine the first high saturation area in the data read heatmap, and determine the data block corresponding to the first high saturation area as the target data block. The color depth of the first high saturation area is greater than the first color threshold.

[0110] Method 2: Determine the write frequency of each of the multiple data blocks, construct a data write heatmap of the current disk based on the write frequency, determine the second high saturation area in the data write heatmap, and determine the data block corresponding to the second high saturation area as the target data block. The color depth of the second high saturation area is greater than the second color threshold.

[0111] Method 3: Determine the data read latency and data write latency of each of the multiple data blocks. Construct a data latency heatmap of the current disk based on the data read latency and data write latency. Determine the third high saturation region in the data latency heatmap. Select the data block corresponding to the third high saturation region as the target data block. The color depth of the third high saturation region is greater than the third color threshold.

[0112] As an optional implementation, in Method 1 above, read operation information is extracted from the storage controller's IO logs. The LBA (Logical Block Address) of each read operation is recorded. Data blocks with the same LBA are counted, and the read frequency of each data block within the target detection period (e.g., the most recent hour) is calculated. This frequency is converted into color depth to construct a heatmap, indicating which areas are frequently accessed and which are relatively less accessed. The heatmap is analyzed to identify areas with high read frequency, i.e., those with a color depth exceeding a first color threshold. These areas are considered high-saturation areas and may contain hotspot access data.

[0113] All data blocks within the first high-saturation region are marked as "target data blocks," indicating the location of frequently accessed hot data. Detailed information for each target data block is recorded, including LBA, read frequency, and average read latency. The target data block information will be used in the implementation of key steps such as setting fault warning thresholds and making data migration decisions to ensure that hot data is given priority processing and protection.

[0114] By constructing a data reading heatmap and identifying high-saturation areas, the most frequently read data blocks can be quickly identified and located, providing precise targets for fault prediction and data reconstruction. The identification of target data blocks can optimize data layout and resource allocation in advance, shorten business recovery time, and improve the efficiency of data reconstruction.

[0115] As an optional implementation method, in the above-mentioned Method 2, all write operations in the past hour were monitored. It was found that the average write frequency of data blocks in the LBA10GB-20GB range was 500 times / h, which was higher than other areas. Based on this, a data write heatmap was constructed, in which the LBA10GB-20GB area was represented by dark blue, and the remaining areas were represented by different shades of blue according to the write frequency. By analyzing the heatmap, the dark blue area was determined to be the second high saturation area, and its color depth exceeded the preset second color threshold. The data blocks in the LBA10GB-20GB range were marked as "target data blocks".

[0116] By constructing a data write heatmap and identifying high-saturation areas, the data blocks with the highest write frequency can be quickly identified and located, providing precise targets for fault prediction and data reconstruction. The identification of target data blocks can optimize data layout and resource allocation in advance, shorten business recovery time, and improve the efficiency of data reconstruction.

[0117] As an optional implementation method, in the above method three, all read and write operations of the disk over a period of time are monitored, the average read latency and write latency of each data block are calculated, and the above latency information is converted into a latency heatmap. The LBA20GB-30GB area with higher latency is represented by a dark color, and other areas are represented by different shades of the same color according to the latency duration. By analyzing the heatmap, it is confirmed that the dark area is the third highest saturation area, and its color depth exceeds the preset third color threshold. The data block corresponding to the third highest saturation area is determined as the target data block.

[0118] By constructing a data latency heatmap, disk performance can be monitored and analyzed visually, especially high-latency areas. By setting a color depth threshold, data block areas with particularly poor latency performance, namely the third highest saturation area, can be filtered out. This allows for targeted optimization of the read and write performance of these data blocks, such as by data migration or using persistent memory to accelerate access, thereby reducing latency during data reconstruction and improving overall reconstruction efficiency.

[0119] In one alternative implementation, upon detecting a current disk failure, data reconstruction is performed on the failed data in the current disk based on a target dataset obtained from persistent memory, including:

[0120] S7-1, perform an XOR check operation based on the check data and the target dataset to determine the lost dataset, wherein the check data is calculated based on the datasets in multiple data blocks in the current disk;

[0121] S7-2, Reconstruct the faulty data in the current disk based on the lost dataset to obtain fault recovery data;

[0122] S7-3 stores fault recovery data in candidate disks, where candidate disks are determined based on the operating status of multiple disks in the disk array.

[0123] In steps S7-1 to S7-2 above, when a disk failure is detected, the system retrieves the disk's verification data and target dataset from persistent memory, and then uses an XOR verification operation to determine which data is lost, i.e., constructs the lost dataset. The XOR verification operation is a data verification and recovery method based on mathematical logic. By performing an XOR operation on the verification data and the target dataset (a pre-stored set of undamaged data blocks), missing or damaged data blocks can be identified.

[0124] Specifically, select the parity algorithm corresponding to the RAID level (e.g., XOR parity for RAID5), perform an XOR operation on the parity data of normal data stored in persistent memory in advance, and determine the lost dataset corresponding to the data block on the faulty disk based on the result of the XOR operation (if the result does not match the data block on the normal disk, then this part of the data is the lost dataset). Combine the lost data with other data in the target dataset to obtain complete fault recovery data. Perform integrity verification on the fault recovery data, and store the fault recovery data after successful verification.

[0125] As an optional implementation, assuming that disk D1 fails in a RAID5 configuration, the system has already migrated the hot data on D1 to persistent memory based on the data read frequency and latency. When the D1 failure triggers reconstruction, the hot data can be read directly from persistent memory without accessing the failed disk or waiting for slow disk I / O. Furthermore, based on the hot data and parity data in persistent memory (and if necessary, the required data can also be obtained from other normal disks D2, D3, and D4), an XOR operation is performed to calculate the lost data blocks on D1. The calculated fault recovery data is then quickly stored in the candidate disk or the temporary space of persistent memory. Finally, the data is written to a new disk to complete the reconstruction.

[0126] Further in step S7-3 above, by analyzing the operating status and health of all disks in the disk array, including information such as temperature, vibration, SMART index and current load, and based on the disk evaluation results, a disk with the best status (such as low temperature, light load and good health) is selected as a candidate disk for storing fault recovery data. The fault recovery data is then migrated from persistent memory or other disks in the current disk array to the candidate disk to complete the data reconstruction.

[0127] By pre-stored verification data in persistent memory, data recovery can be performed quickly, reducing data reconstruction time. The high-speed read capability of persistent memory shortens data retrieval time, thereby reducing computational latency during data reconstruction and improving data recovery speed. Intelligent selection of candidate disks avoids additional writes to high-load disks during data reconstruction, thus balancing the load on the disk array and improving the overall performance of data reconstruction. XOR verification operations quickly identify which data blocks are lost or corrupted, allowing for targeted reconstruction of these data. Furthermore, a dedicated XOR verification module calculates verification data in persistent memory without consuming processor resources, ensuring that the processor can focus on handling I / O requests during data reconstruction, avoiding performance bottlenecks and further optimizing data reconstruction efficiency.

[0128] The following complete embodiment describes this application:

[0129] Multimodal data acquisition by the multimodal perception layer; the multimodal data acquired by the multimodal perception layer mainly includes: SMART attributes (bad sector count, remapped sectors, seek error rate, etc.) collected by sensors on each disk; real-time I / O load heatmap collected and statistically analyzed by the storage system (recording read / write frequency / latency by LBA range), and summarizing and generating an IO heatmap matrix: Heatmap_Matrix[i][j]=(read access count, write access count, average latency), where i represents the logical disk number, j represents the LBA block, and the collection period can be adjusted according to the prediction accuracy and IO load feedback to ensure increasingly accurate collection; physical sensor data (vibration spectrum > 5kHz sampling, temperature gradient, current fluctuation, etc.).

[0130] Fault AI Prediction Engine. It employs a three-level hybrid model: the first level processes time-series SMART data, the second level analyzes heatmaps and sensor spectrograms, and the third level performs multimodal feature fusion. Combining this data, it performs dynamic fault risk assessment and ultimately outputs a disk risk assessment value.

[0131] For example, the risk score of disk D_k is Risk_Score = W1×LSTM_output+W2×CNN_heatmap_score+W3×vibration_FFT+W4×temp_gradient. An alert is triggered when Risk_Score > N, where N is a threshold that can be adjusted based on feedback. W1, W2, W3, and W4 are the weights of each predicted value.

[0132] Specifically, LSTM_output is a fault probability prediction value based on time-series SMART data. It primarily originates from multiple parameters collected by the disk's built-in SMART sensors, such as remapped sector count, seek error rate, and uncorrectable error count. The score determination process is as follows: Figure 3 As shown: S302, acquire raw data on self-monitoring, analysis, and reporting technical attributes, specifically including but not limited to indicators such as disk bad sector count, remapped sectors, and seek error rate; S304, data standardization, eliminating dimensional differences and numerical ranges between different SMART attributes, ensuring the data is on the same scale before being input into the neural network, avoiding bias in model predictions due to some attributes being too large or too small; S306, sliding window processing, capturing trends and periodic changes in time series data, improving the model's understanding of time series data, thereby improving the sensitivity and reliability of fault prediction; S308, Long Short-Term Memory Neural Network prediction, LSTM can capture long-term trends and short-term fluctuations in data, enabling the model to make predictions not only based on recent SMART data, but also to consider disk state changes over a longer time span, thereby improving the accuracy of fault prediction; finally, execute S310 to output the fault probability.

[0133] `CNN_heatmap_score` represents an anomaly distribution score based on the I / O access heatmap, which constructs the load heatmap using LBA access logs recorded by the storage controller, read / write operation frequency, and access latency distribution. For example, a data matrix can be constructed to represent the working state of the storage device, starting from LBA blocks 0-1MB and continuing to 1023-1024MB, with each block representing a continuous 1MB range of data on the storage device. Optionally, for LBA blocks of 0-1MB, the recorded read operation frequency is 12.5 times, the write operation frequency is 3.8 times, and the average latency for each read / write request is 125 milliseconds. As the block moves towards a higher LBA value, such as reaching 1-2MB, the read frequency drops to 8.2 times, the write frequency significantly increases to 15.6 times, and the average latency slightly decreases to 98 milliseconds. For the last block, 1023-1024MB, the read frequency is 0.3 times, the write frequency is 22.1 times, and the latency is 420 milliseconds.

[0134] The heatmap is converted into a three-channel image (RGB = read frequency / write frequency / latency), which is then input into a convolutional neural network (CNN) for feature extraction to obtain the final score CNN_heatmap_score. It should be noted that the three-channel image obtained from the heatmap needs to be normalized, that is, the original read frequency is mapped to the range [0,255] (the standard value range of RGB channels).

[0135] Assuming the current read frequency of 1250 IOPS is greater than the maximum read frequency (e.g., 1000 IOPS), the truncation ratio is set to 1.0 (to prevent overflow). R = (current read frequency / maximum read frequency × 255), and the R channel value is 255 (red channel saturation indicates the read frequency has reached or exceeded the system-defined maximum value). The purpose of truncation is to directly mark it as "severely abnormal" (corresponding to the maximum RGB value of 255) when the read frequency exceeds the maximum threshold. The maximum read frequency can be set to Max_Read_IOPS = max(the maximum value of all block read frequencies within a certain time period), adapting to different load scenarios (e.g., the threshold will naturally increase during peak business periods). The maximum read frequency can be determined based on the maximum read frequency at a historical moment.

[0136] Image normalization ensures all values ​​are within the valid RGB range (0-255). The same principle applies to write frequency, latency, and read frequency calculations; the B channel value is set to 255 (blue channel saturation indicates latency has reached the maximum warning threshold). Transforming physical metrics into intuitive image features facilitates convolutional neural networks in identifying spatial anomalies, real-time statistics of actual maximum values ​​in historical data, continuous recording of read frequencies for all LBA blocks (e.g., sampling every 5 minutes), maintenance of a sliding time window (e.g., 24 hours), saving the peak read frequency within the window, and using the current maximum value as the normalization benchmark.

[0137] CNN_heatmap_score=σ(∑w_i×f_i+b), where w_i is the weight of the i-th feature (e.g., hotspot weight=0.3, delay weight=0.4). The weights reflect the contribution of different features. For example, if the delay weight is greater than the maximum, it indicates that more attention is paid to delay anomalies. Delay is directly related to the health of the medium. f_i is the value of the i-th feature, and b is the bias term. The above formula means that the fully connected layer maps the features to the scoring space and compresses the linear output to (0,1) through the Sigmoid function.

[0138] like Figure 4 As shown, the abnormal feature types include: hot spots. When the read and write operations of a certain LBA area are abnormally frequent and exceed the normal level, the corresponding position in the image will show continuous and obvious red or green highlights, indicating that the area may be worn or have its lifespan shortened due to frequent access and overuse. Frequent hot spot access will not only lead to uneven disk load, but may also accelerate the aging process of the disk and increase the risk of failure.

[0139] An abnormal latency is represented by a localized blue highlighted area in the image. If the read / write latency of a certain LBA block suddenly increases, the corresponding block will be displayed as a blue highlight in the image. This indicates that there is physical damage (media failure), difficulty in reading, or other problems within the disk media, such as bad sectors, controller failure, or increased difficulty in data retrieval, causing the storage device to respond slowly when processing I / O requests and reducing the overall performance of the system.

[0140] Cold data mutations occur when areas of cold data, which normally appear as dark areas in an image (where there are few read / write operations), suddenly experience read / write spikes, appearing as bright spots within dark areas. This could be due to malware activity, data scanning, or unusual access patterns.

[0141] Striping imbalance, in RAID-based storage configurations, manifests as periodic bright and dark stripes in an image. This indicates that data is not evenly distributed among the disks (i.e., the stripe distribution of the independent disk redundant array is uneven), causing some disks to bear excessive I / O load, while other disks are relatively idle.

[0142] When the calculated score is between 0.0 and 0.3, the system is in a normal state and the storage device is in a uniform access mode. When the score rises to the range of 0.3 to 0.6, the system enters the warning stage, which indicates that local hotspots are beginning to appear, meaning that some LBA blocks are experiencing more frequent read and write access. When the score reaches 0.6 to 0.8, the system is in a moderately abnormal state, with hotspot areas appearing and the latency in these areas also increasing. When the score jumps to the range of 0.8 to 1.0, the system is in a severely abnormal state, with hotspot areas spreading and latency increasing sharply.

[0143] The calculation process of the CNN_heatmap_score is as follows: Figure 5 As shown: S502, construct a three-channel heatmap image, such as converting the disk read frequency, write frequency and access latency distribution into a three-channel (RGB) image format; S504, convolutional layer processing; S506, activation function processing; S508, pooling layer processing; S510, fully connected layer processing, converting the local features of the heatmap into a comprehensive score of the overall disk status; S512, anomaly score output.

[0144] Vibration_FFT represents the intensity of fault characteristics in the mechanical vibration spectrum. The data source is a triaxial accelerometer built into the disk drive. A Fast Fourier Transform (FFT) is used to convert the time-domain vibration signal into a frequency-domain energy distribution. The calculation formula is: vibration_FFT = Current frequency-domain energy / Total energy across the entire frequency band.

[0145] `temp_gradient` reflects the drastic change in disk temperature and is used to warn of the following faults: short circuit in the motor coil, abnormal current causing rapid heating (>2℃ / min); cooling system failure, reduced heat dissipation capacity causing a temperature spike; the data comes from the temperature sensor on the disk. The temperature change per unit time can be obtained using the following formula:

[0146]

[0147] Furthermore, the scoring formula is as follows:

[0148]

[0149] The segmented scoring can be dynamically adjusted based on the actual situation.

[0150] Finally, the AI ​​decision-making expert model will execute corresponding operations based on the Risk_Score assessment result. For example, if the risk is determined to be moderately abnormal (e.g., the disk risk assessment value falls within the range of 0.5-0.7), a preventative migration will be triggered. In this mode, disk data will be divided into hot and cold data. Cold data will be migrated to a portion of the reserved backup area first, while hot data will be processed according to a different strategy. When writing, the data will first be written to the persistent memory resource pool allocated to the disk, and then written to the disk. Thus, in the event of a real disk failure, data can be quickly reconstructed from persistent memory to the backup area, completing a rapid RAID reconstruction.

[0151] Specifically, using persistent memory as a write cache or read cache (especially a metadata cache) for the RAID controller can significantly improve performance. Persistent memory, as a storage layer, is functionally divided into a part for storing a parity data pool and a part for storing a hot data pool. The parity data is allocated to each disk according to the number of disks to store the corresponding disk parity data. When the parity data is modified, it is first retrieved from the corresponding persistent memory area and then asynchronously written to the actual disk. The hot data storage is used to store the hot data that triggers the migration of abnormal rating data during rapid reconstruction.

[0152] Based on PMem's atomicity guarantee and fast recovery, and leveraging PMem's persistence and byte-addressing characteristics, a faster and more reliable fault recovery and consistency guarantee mechanism can be achieved. Meanwhile, the calculation of the verification data in the persistent memory resource pool is carried out through a dedicated XOR acceleration module. All RAID data verification is calculated through the XOR acceleration module and saved to persistent memory, and then asynchronously written to disk without going through the device processor. This ensures that the processor can focus on IO processing during the reconstruction process, avoiding impact on business continuity and performance.

[0153] According to another aspect of the embodiments of this application, a data reconstruction apparatus for implementing the above-described data reconstruction method is also provided. For example... Figure 6 As shown, the device includes:

[0154] The first determining unit 602 determines the current disk's operating status information and the access popularity information of each of the multiple data blocks in the current disk, wherein the operating status information includes a status quantity used to indicate at least one current physical state of the current disk.

[0155] The second determining unit 604 determines the current risk assessment value of the disk based on the running status information and access popularity information;

[0156] The data read / write unit 606 writes the target dataset from at least one target data block in the current disk into persistent memory when the risk assessment value of the current disk is greater than the fault warning threshold. The fault warning threshold is determined based on the operating status of the current disk at a reference time, where the reference time is the time when the current disk fails. The access popularity information of the target data block meets the filtering conditions, and the access speed to the data in persistent memory is greater than the access speed to the data on the disk.

[0157] Reconstruction unit 608, when a disk failure is detected, reconstructs the faulty data in the current disk based on the target dataset obtained from persistent memory.

[0158] Optionally, the first determining unit 602 mentioned above includes: a third determining module, used to determine the temperature change information indicated by the current disk's operating status information based on the temperature information collected by the temperature sensor on the current disk during the target detection period; to determine the energy distribution information indicated by the current disk's operating status information based on the disk vibration information collected by the accelerometer on the current disk during the target detection period; and to determine multiple disk self-test index values ​​indicated by the current disk's operating status information during the target detection period.

[0159] Optionally, the third determining module is further configured to determine the current disk temperature change rate based on temperature change information; determine the current disk vibration intensity value based on energy distribution information; determine the current disk health prediction value based on multiple disk self-test index values; and determine the current disk operation risk prediction value based on temperature change rate, vibration intensity value, and health prediction value.

[0160] Optionally, the first determining unit 602 mentioned above includes: a fourth determining module, configured to determine, based on the log data of the current disk in the target detection period, the read frequency of performing data read operations on multiple data blocks as indicated by the access heat information of the current disk; determine, based on the log data of the current disk in the target detection period, the write frequency of performing data write operations on multiple data blocks as indicated by the access heat information of the current disk; and determine the data read latency and data write latency of each of the multiple data blocks indicated by the access heat information of the current disk in the target detection period.

[0161] Optionally, the fourth determining module is further configured to: determine hotspot anomaly information of the target data block when the read frequency or write frequency of the target data block is greater than a first threshold; determine latency anomaly information of the target data block when the data read latency or data write latency of the target data block is greater than a second threshold; determine cold data mutation information of the target data block when the read frequency or write frequency of the target data block increases from a low-frequency state to a high-frequency state within the target detection period; determine stripe imbalance information of the target data block when the read frequency or write frequency between the target data block and the reference data block meets the target difference condition at multiple time intervals within the target detection period, wherein the target data block and the reference data block are two adjacent data blocks in the current disk; and determine the access anomaly prediction value of the current disk based on the hotspot anomaly information, latency anomaly information, cold data mutation information, and stripe imbalance information of the target data block.

[0162] Optionally, the second determining unit 604 is further configured to determine the predicted operating risk value based on the current disk's operating status information; and to determine the risk assessment value of the current disk based on the weighted sum of the predicted operating risk value and the predicted access anomaly value.

[0163] Optionally, the aforementioned data read / write unit 606 is further configured to determine the target operating environment based on the current disk's data access mode information, wherein the data access mode is used to indicate the method of performing data read or data write operations on the current disk; determine the reference operating status information of the current disk in the target operating environment at the reference time, and the reference access heat information of the data blocks in the current disk; and determine the fault warning threshold based on the reference operating status information and the reference access heat information.

[0164] Optionally, the aforementioned data read / write unit 606 is further configured to: determine the read frequency of each of the multiple data blocks; construct a data read heatmap of the current disk based on the read frequency; determine a first high-saturation region in the data read heatmap; and determine the data block corresponding to the first high-saturation region as the target data block, wherein the color depth of the first high-saturation region is greater than a first color threshold; determine the write frequency of each of the multiple data blocks; construct a data write heatmap of the current disk based on the write frequency; determine a second high-saturation region in the data write heatmap; and determine the data block corresponding to the second high-saturation region as the target data block, wherein the color depth of the second high-saturation region is greater than a second color threshold; determine the data read latency and data write latency of each of the multiple data blocks; construct a data latency heatmap of the current disk based on the data read latency and data write latency; determine a third high-saturation region in the data latency heatmap; and determine the data block corresponding to the third high-saturation region as the target data block, wherein the color depth of the third high-saturation region is greater than a third color threshold.

[0165] Optionally, the aforementioned reconstruction unit 608 includes a calculation module, used to perform an XOR check operation based on the check data and the target dataset to determine the lost dataset, wherein the check data is calculated based on the datasets in multiple data blocks in the current disk; to reconstruct the fault data in the current disk based on the lost dataset to obtain fault recovery data; and to store the fault recovery data in a candidate disk, wherein the candidate disk is determined based on the operating status of multiple disks in the disk array.

[0166] Optionally, the aforementioned calculation module is also used to perform an XOR check operation based on the verification data and the target dataset to determine the lost dataset; and to reconstruct the faulty data in the current disk based on the lost dataset to obtain fault recovery data.

[0167] For a description of the features in the embodiment corresponding to the data reconstruction device, please refer to the relevant description of the embodiment corresponding to the data reconstruction method, which will not be repeated here.

[0168] Embodiments of this application also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above-described data reconstruction method embodiments.

[0169] The electronic device used in this application can be a terminal device or a server. This embodiment uses a mobile phone or computer as an example for illustration. Figure 7 As shown, the electronic device includes a memory 702 and a processor 704. The memory 702 stores a computer program, and the processor 704 is configured to execute the steps in any of the above method embodiments via the computer program.

[0170] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0171] Alternatively, as those skilled in the art will understand, Figure 7 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones (such as Android phones, iOS phones, etc.), tablets, PDAs, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 7 This does not limit the structure of the aforementioned electronic devices. For example, the electronic device may also include components that are more... Figure 7 The more or fewer components shown (such as network interfaces, etc.), or having the same Figure 7 The different configurations shown.

[0172] The memory 702 can be used to store software programs and modules, such as the program instructions / modules corresponding to the data reconstruction method and apparatus in this embodiment. The processor 704 executes various functional applications by running the software programs and modules stored in the memory 702, thereby realizing the aforementioned data reconstruction method. The memory 702 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 702 may further include memory remotely located relative to the processor 704, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 702 may be used, but is not limited to, for storing information such as signals and data. As an example, such as... Figure 7 As shown, the memory 702 may include, but is not limited to, the first determining unit 602, the second determining unit 604, the data read / write unit 606, and the reconstruction unit 608 in the data reconstruction device. Furthermore, it may include, but is not limited to, other module units in the data reconstruction device, which will not be elaborated upon in this example.

[0173] Optionally, the transmission device 706 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 706 includes a Network Interface Controller (NIC), which can be connected to other network devices and a router via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 706 is a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0174] In addition, the above-mentioned electronic device also includes: a display 708; and a connection bus 710 for connecting the various module components in the above-mentioned electronic device.

[0175] In other embodiments, the aforementioned terminal device or server can be a node in a distributed system, wherein the distributed system can be a blockchain system, which is a distributed system formed by connecting multiple nodes through network communication. The nodes can form a point-to-point network, and any form of computing device, such as a server, terminal, or other electronic device, can become a node in the blockchain system by joining this point-to-point network.

[0176] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described data reconstruction method embodiments at runtime.

[0177] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0178] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described data reconstruction method embodiments.

[0179] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above-described data reconstruction method embodiments.

[0180] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0181] The data reconstruction method and electronic device provided in this application have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only intended to help understand the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A data reconstruction method, characterized by, Comprise: Determine the running state information of the current disk, and the access heat information of each data block in the current disk, wherein the running state information comprises at least one state quantity indicating at least one current physical state of the current disk; Determine the target data set in at least one target data block in the current disk, including at least one of the following: determine the read frequency of each data block, construct a data read heat map of the current disk according to the read frequency, determine a first high saturation area in the data read heat map, and determine the data block corresponding to the first high saturation area as the target data block, wherein the color depth of the first high saturation area is greater than a first color threshold; determine the write frequency of each data block, construct a data write heat map of the current disk according to the write frequency, determine a second high saturation area in the data write heat map, and determine the data block corresponding to the second high saturation area as the target data block, wherein the color depth of the second high saturation area is greater than a second color threshold; determine the data read delay time and the data write delay time of each data block, construct a data delay heat map of the current disk according to the data read delay time and the data write delay time, determine a third high saturation area in the data delay heat map, and determine the data block corresponding to the third high saturation area as the target data block, wherein the color depth of the third high saturation area is greater than a third color threshold; According to the hot spot abnormal information, delay abnormal information, cold data mutation information and strip imbalance information of the target data block determined based on the access heat information, determine the access abnormal prediction value of the current disk; According to the running risk prediction value of the current disk, determine the running risk prediction value according to the running state information of the current disk; According to the weighted sum result of the running risk prediction value and the access abnormal prediction value of the current disk, determine the risk evaluation value of the current disk; In the case where the risk evaluation value of the current disk is greater than the failure warning threshold, write the target data set in at least one target data block in the current disk to the persistent memory, wherein the failure warning threshold is determined according to the running state of the current disk at the reference time, the reference time is the time when the current disk fails, the access heat information of the target data block meets the screening condition, and the access speed to the data in the persistent memory is greater than the access speed to the data in the disk; In the case where the current disk is detected to fail, reconstruct the fault data in the current disk according to the target data set obtained from the persistent memory.

2. The method of claim 1, wherein the running state information of the current disk is determined by: According to the temperature information collected by the temperature sensor on the current disk within the target detection period, determine the temperature change information indicated by the running state information of the current disk; ​ According to the disk vibration information collected by the acceleration sensor on the current disk in the target detection period, the energy distribution information indicated by the running state information of the current disk is determined. A plurality of disk self-check index values of the current disk in the target detection period are determined.

3. The method of claim 2, wherein after determining the running state information of the current disk, the method further comprises: According to the temperature change information, the temperature change rate of the current disk is determined. According to the energy distribution information, the vibration intensity value of the current disk is determined. According to a plurality of disk self-check index values, the health prediction value of the current disk is determined. According to the temperature change rate, the vibration intensity value and the health prediction value, the running risk prediction value of the current disk is determined.

4. The method of claim 1, wherein the access heat information of each of a plurality of data blocks in the current disk is determined, and the method further comprises: According to the log data of the current disk in the target detection period, the read frequency of performing data read operation on each of a plurality of data blocks indicated by the access heat information of the current disk is determined. According to the log data of the current disk in the target detection period, the write frequency of performing data write operation on each of a plurality of data blocks indicated by the access heat information of the current disk is determined. A plurality of data read delay time lengths and data write delay time lengths of each of a plurality of data blocks indicated by the access heat information of the current disk in the target detection period are determined.

5. The method of claim 4, wherein after determining the access heat information of each of a plurality of data blocks in the current disk, the method further comprises: In the case where the read frequency or the write frequency of the target data block is greater than a first threshold value, hot spot abnormal information of the target data block is determined. In the case where the data read delay time length or the data write delay time length of the target data block is greater than a second threshold value, delay abnormal information of the target data block is determined. In the case where the read frequency or the write frequency of the target data block increases from a low frequency state to a high frequency state in the target detection period, cold data mutation information of the target data block is determined. In the case where the read frequency or the write frequency between the target data block and the reference data block satisfies a target difference condition in a plurality of time intervals in the target detection period, strip imbalance information of the target data block is determined, wherein the target data block and the reference data block are two adjacent data blocks in the current disk.

6. The method of claim 1, wherein before writing a target data set in at least one target data block in the current disk to the persistent memory, the method further comprises: According to the data access mode information of the current disk, a target running environment is determined, wherein the data access mode is used to indicate the way of performing data read operation or data write operation on the current disk. ​ ​ ​ ​ determining reference running state information of the current disk in the target running environment and reference access hotness information of the data block in the current disk at the reference time point; determining the fault warning threshold according to the reference running state information and the reference access hotness information.

7. The method of claim 1, wherein, in a case where it is detected that the current disk has a fault, performing data reconstruction on the fault data in the current disk according to the target data set obtained from the persistent memory, comprising: performing an exclusive or check operation according to check data and the target data set to determine a missing data set, wherein the check data is calculated according to data sets in a plurality of the data blocks in the current disk; reconstructing the fault data in the current disk according to the missing data set to obtain fault recovery data; storing the fault recovery data in a candidate disk, wherein the candidate disk is determined according to running states of a plurality of disks in a disk array.

8. An electronic device, comprising: comprising: a memory for storing a computer program; a processor for implementing the steps of the data reconstruction method of any one of claims 1 to 7 when executing the computer program.

Images