A method for managing a disk array of a large-capacity high-speed data recorder

Abstract

The application discloses a kind of disk array management methods of large capacity high-speed data recorder, it is related to data management technical field, comprising: according to real-time running state parameter calculation each physical disk's health score, and the physical disk that health score is lower than preset degradation threshold is marked as early warning disk;According to the write priority level determination corresponding strip granularity, the data stream to be written is segmented into strip unit according to strip granularity and is rotated and distributed to each physical disk;With preset sampling period monitoring the write queue depth of each physical disk, when the write queue depth of any physical disk exceeds preset congestion threshold, the strip unit of the physical disk that has not been written into disk is re-routed to the physical disk with the lowest current write load rate;The write channel of early warning disk and inventory data are migrated to hot spare disk;Locate the damaged strip of failed verification;The application improves the data write real-time performance of large capacity high-speed data recorder in long time multi-channel continuous recording scene.

Description

Technical Field

[0001] This invention relates to the field of data management technology, and specifically to a disk array management method for a high-capacity, high-speed data logger. Background Technology

[0002] With the rapid development of aerospace, radar detection, high-speed communication and scientific experiments, multi-channel, high-sampling-rate data acquisition systems have placed extremely stringent requirements on back-end storage devices. As the core storage terminal of the above systems, high-capacity high-speed data loggers need to continuously receive and persist multiple parallel high-speed data streams in a short period of time. Their storage media usually adopts a disk array composed of multiple physical disks to obtain sufficient write bandwidth and storage capacity.

[0003] Currently, existing disk array management solutions mainly follow RAID management strategies used in general-purpose servers or enterprise-level storage. When facing the specific application scenarios of high-speed data loggers, these solutions have the following shortcomings: First, existing solutions rely heavily on a single SMART threshold alarm mechanism to assess disk health. Alarms are triggered only when disk parameters exceed manufacturer-preset fixed thresholds, lacking the ability to dynamically track and predict disk degradation trends. This makes it difficult to take timely data protection measures when disk performance is gradually declining but not yet completely failing, resulting in a high risk of data loss. Second, existing solutions typically use fixed striping granularity during data writing. The existing solution uses a round-robin allocation method without considering the differences in priority and real-time performance of multi-channel data streams in high-speed data loggers. High-priority channels share the same stripe granularity and write scheduling strategy with low-priority channels, which can easily increase the write latency of high-priority data streams and affect the real-time disk write of critical data. Third, in multi-channel concurrent write scenarios, the existing solution lacks a dynamic balancing scheduling mechanism for the write queues of each physical disk. When individual disks experience queue congestion due to excessive instantaneous write pressure, data continues to be allocated to those disks according to a fixed round-robin strategy, further exacerbating the write bottleneck and significantly reducing the overall array's write throughput.

[0004] Therefore, there is an urgent need for a high-capacity, high-speed data logger disk array management method that can comprehensively assess disk health status, achieve priority-aware adaptive stripe writing, dynamically balance write load, quickly complete data migration and stripe-level reconstruction of faulty disks, and have automatic capacity management capabilities.

[0005] Beneficial effects Summary of the Invention

[0006] To address the aforementioned technical problems, a disk array management method for a high-capacity, high-speed data logger is provided. This technical solution resolves the problems described above.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] A disk array management method for a high-capacity, high-speed data logger includes:

[0009] Obtain the real-time operating status parameters of each physical disk in the disk array, calculate the health score of each physical disk based on the real-time operating status parameters, and mark physical disks with health scores lower than a preset degradation threshold as warning disks;

[0010] Obtain the channel number and data rate of the data stream to be written, determine the write priority level based on the channel number, determine the corresponding stripe granularity according to the write priority level, divide the data stream to be written into stripe units according to the stripe granularity and distribute them to each physical disk in a round-robin fashion;

[0011] The write queue depth of each physical disk is monitored at a preset sampling period. When the write queue depth of any physical disk exceeds the preset congestion threshold, the stripe units of that physical disk that have not yet been written to the disk are rerouted to the physical disk with the lowest current write load rate.

[0012] In response to the health score of the warning disk dropping to a preset failure threshold, the write channel and existing data of the warning disk are migrated to the hot standby disk.

[0013] Perform periodic check code comparisons on the stripe units already written to each physical disk, locate the damaged stripes that failed the check, perform XOR reconstruction based on the redundant check data of the other disks in the same stripe group, and write the reconstructed stripe units to the replacement disk.

[0014] Furthermore, the calculation of the health score of each physical disk based on the real-time operating status parameters includes:

[0015] Read the SMART attribute data of each physical disk and extract the remapped sector count, the number of sectors to be repaired, the cumulative power-on time, and the historical temperature peak.

[0016] The remapped sector count, the sector count to be repaired, the cumulative power-on time, and the historical temperature peak are each assigned a preset weight coefficient and then weighted and summed to obtain the disk degradation index.

[0017] The real-time spindle vibration amplitude of each physical disk is obtained. If the real-time spindle vibration amplitude exceeds a preset vibration safety threshold, a vibration penalty factor is added to the disk degradation index.

[0018] The disk degradation index, after being superimposed with a vibration penalty factor, is normalized to the range of 0 to 100 to obtain the health score;

[0019] Linear regression fitting is performed on the health score for multiple consecutive sampling periods to calculate the downward slope of the health score; if the absolute value of the downward slope is greater than a preset slope threshold, an accelerated degradation warning signal is generated.

[0020] Furthermore, according to the write priority level, the corresponding stripe granularity is determined, and the data stream to be written is divided into stripe units according to the stripe granularity and distributed to each physical disk in a round-robin fashion, including:

[0021] The corresponding number of stripe bytes is read from the preset granularity configuration table according to the write priority level, wherein the number of stripe bytes corresponding to the high priority data stream is less than the number of stripe bytes corresponding to the low priority data stream;

[0022] The data stream to be written is cut into stripe units of equal length according to the number of bytes in the stripe;

[0023] Obtain the current available capacity and cumulative write volume of each physical disk in the disk array, and calculate the capacity balancing coefficient of each physical disk;

[0024] The target write disk number of each stripe unit is determined according to the capacity balancing coefficient, and the stripe unit is sequentially and round-robin distributed to each physical disk.

[0025] Write the channel number, timestamp, and incremental sequence number at the beginning of each stripe unit, and append a cyclic redundancy check code to the end of each stripe unit.

[0026] Furthermore, the step of monitoring the write queue depth of each physical disk at a preset sampling period, and when the write queue depth of any physical disk exceeds a preset congestion threshold, rerouting the unwritten stripe units of that physical disk to the physical disk with the lowest current write load rate, includes:

[0027] The write queue depth and instantaneous write bandwidth of each physical disk are polled at the preset sampling period;

[0028] Calculate the write load rate of each physical disk, where the write load rate is the ratio of the current write queue depth to the maximum queue depth;

[0029] The physical disk whose write load rate exceeds the preset congestion threshold is marked as a congested disk;

[0030] Select the physical disk with the lowest write load rate from the remaining physical disks as the migration target disk;

[0031] The stripe units in the write queue of the congested disk that have not yet been written to the disk are reallocated to the migration target disk;

[0032] The allocation weight of the congested disk in the striping strategy is temporarily set to zero until the write load rate of the congested disk falls below the preset recovery threshold, after which the allocation weight is gradually restored with a preset step value.

[0033] Furthermore, in response to the health score of the warning disk dropping to a preset failure threshold, migrating the write channel and existing data of the warning disk to a hot standby disk includes:

[0034] When the health score drops to the preset failure threshold, a warning disk offline command is sent to the disk array controller;

[0035] In response to the warning disk going offline command, the disk array controller switches the write channel of the warning disk to the hot spare disk, and the subsequent newly written stripe units are directly written to the hot spare disk.

[0036] Obtain the data distribution table of the warning disk and determine the physical offset address of each stripe unit in the data.

[0037] The stored data is read strip by strip from the warning disk according to the physical offset address, and the stored data is written to the corresponding offset address of the hot standby disk;

[0038] During the migration of existing data, sector-by-sector read verification is performed on the warning disk; if any sector read fails, the content of the sector is recovered by XOR operation using the redundant verification data of the other physical disks in the same stripe group and then written to the hot spare disk.

[0039] After all the existing data has been migrated, the warning disk is marked as offline and the hot standby disk is registered as a formal member disk of the array.

[0040] Furthermore, the step of performing periodic checksum comparisons on the stripe units already written to each physical disk to locate corrupted stripes that failed the checksum verification includes:

[0041] The stripe units already written to each physical disk are read sequentially according to the preset inspection cycle;

[0042] For each stripe unit, the cyclic redundancy check code is recalculated and compared with the original check code stored at the end of the stripe unit;

[0043] If the recalculated checksum is inconsistent with the original checksum, the stripe unit is marked as a suspected damaged stripe.

[0044] Perform a second reading verification on the suspected damaged strip, continuously read a preset number of times and compare the results of each reading;

[0045] If the reading results of the preset number of times are inconsistent with the original check code, then the suspected damaged stripe is confirmed to be the damaged stripe.

[0046] Record the physical disk number, stripe sequence number, and physical offset address of the damaged stripe, and generate a damaged stripe location report.

[0047] Furthermore, the step of performing XOR reconstruction based on the redundancy check data of the other disks in the same stripe group, and writing the reconstructed stripe unit to the replacement disk, includes:

[0048] The strip group to which the damaged strip belongs is determined based on the damaged strip location report;

[0049] Read the corresponding data stripe units and check stripe units from the remaining physical disks in the stripe group;

[0050] Perform an XOR operation byte-by-byte on the data stripe unit and the check stripe unit to calculate the original data content of the damaged stripe;

[0051] Regenerate a cyclic redundancy check code from the calculated original data content;

[0052] The physical disk with the largest available capacity and the lowest write load rate in the disk array is selected as the replacement disk;

[0053] Write the original data content and the regenerated cyclic redundancy check code to the corresponding offset address of the replacement disk;

[0054] Update the disk mapping table of the stripe group to point the physical address of the damaged stripe to the replacement disk.

[0055] Furthermore, after generating the accelerated degradation warning signal, the method further includes:

[0056] In response to the accelerated degradation warning signal, the historical temperature curve and historical vibration curve of the warning disk are acquired;

[0057] The historical temperature curve and the historical vibration curve are input into a pre-trained disk life prediction model, which outputs the estimated remaining usable time of the warning disk.

[0058] If the estimated remaining available time is less than the preset safe time threshold, the inspection frequency of the warning disk will be increased to a preset multiple of the default inspection frequency.

[0059] Prioritize migrating the existing data on the warning disk that has not yet been redundantly backed up to the physical disk with the highest health score;

[0060] Before the existing data migration is completed, the write rate of new data to the warning disk is limited to a preset limit value.

[0061] Furthermore, after writing the reconstructed stripe units to the replacement disk, the method further includes:

[0062] Obtain the total available capacity and the currently used capacity of the disk array, and calculate the remaining storage ratio;

[0063] If the remaining storage ratio is lower than the preset insufficient capacity alarm threshold, an insufficient capacity alarm signal is generated;

[0064] In response to the insufficient capacity alarm signal, each storage file is scanned in order of data writing time from earliest to latest;

[0065] The storage file whose data was written earliest and has been verified is marked as an archiveable file;

[0066] The archiveable files are migrated from the disk array to an external archiving storage medium;

[0067] After the archiveable files are migrated, the disk space occupied by the archiveable files is released and the available capacity table of the disk array is updated.

[0068] Furthermore, after temporarily setting the allocation weight of the congested disk in the striping strategy to zero, the method further includes:

[0069] Continuously monitor the write load rate and disk temperature of the congested disk;

[0070] If the write load rate of the congested disk is lower than the preset recovery threshold for multiple consecutive sampling periods and the disk temperature drops below the preset temperature safety threshold, then the write freeze state of the congested disk is lifted.

[0071] The allocation weight of the congested disk is gradually restored using the preset step value, and a preset observation period is waited after each level of weight is restored.

[0072] Monitor the write latency and read / write error rate of the congested disk within the preset observation period;

[0073] If the write latency or the read / write error rate exceeds a preset abnormal threshold, the allocation right of the congested disk is reset to zero and a disk abnormality report is generated.

[0074] If the write latency and read / write error rate are both within the normal range during the preset observation period, then the next level of weights will continue to be restored until the allocated weights are restored to the initial equilibrium value.

[0075] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0076] 1. This invention comprehensively reads multi-dimensional parameters such as SMART attribute data of the disk, real-time spindle vibration amplitude, and historical temperature peaks, and calculates a weighted disk degradation index and normalizes it into a health score. Simultaneously, it performs linear regression fitting on the health score over multiple consecutive sampling periods to calculate the decline slope. This allows for the generation of early warning signals when the disk performance has not yet completely failed but shows an accelerated degradation trend, providing sufficient lead time for data migration and disk replacement, effectively reducing the risk of data loss due to sudden disk failures. Furthermore, this invention determines the write priority level based on the channel number of the data stream to be written and selects the corresponding stripe byte count from a preset granularity configuration table accordingly. This allows high-priority data streams to be written with smaller stripe granularity to reduce single-write latency, while low-priority data streams are written with larger stripe granularity to improve sequential write efficiency, thus balancing the differentiated needs of multi-channel data streams in terms of real-time performance and throughput.

[0077] 2. This invention continuously monitors the write queue depth and write load rate of each physical disk at a preset sampling period. When a congested disk is detected, the stripe units that have not yet been written to the disk are rerouted to the disk with the lowest load rate, and the allocation weight of the congested disk is temporarily set to zero. After the congestion is relieved, the allocation weight is gradually restored with a preset step value. An observation period is set for each restoration level to verify the write latency and error rate, which avoids the congested disk from being overloaded again due to instantaneous pressure in the early stage of restoration, and ensures a smooth transition of array write performance. When the health score of the warning disk drops to a preset failure threshold, this invention immediately switches the write channel to the hot standby disk to ensure that new data is not interrupted in writing. Then, the existing data is migrated from the warning disk to the hot standby disk stripe by stripe. During the migration process, read verification is performed on each sector. For sectors that fail to read, the redundant verification data of the other disks in the same stripe group is recovered by XOR operation, which ensures that all data is safely transferred before the disk fails completely, and avoids the risk of long-term degraded operation caused by full disk reconstruction.

[0078] 3. This invention recalculates the cyclic redundancy check (CRC) code of the written stripe units according to a preset inspection cycle and compares it with the original check code. After eliminating occasional reading interference through secondary reading verification, it accurately locates the damaged stripe. Only the stripe group to which the damaged stripe belongs is XORed and reconstructed and written to the replacement disk. Compared with the existing whole disk reconstruction method, this significantly shortens the data recovery time, reduces the additional read and write pressure on the remaining disk, and can promptly detect silent data corruption, ensuring the long-term integrity of stored data. After detecting the accelerated degradation warning signal, the historical temperature curve and historical vibration curve of the warning disk are input into the pre-trained disk life prediction model to output the estimated remaining usable time. Based on this, the inspection frequency is increased, existing data that has not completed redundant backups is migrated first, and the new data write rate is limited, forming a complete data protection link from trend prediction to proactive protection, further improving data security. Attached Figure Description

[0079] Figure 1 This is a flowchart illustrating the steps of the present invention. Detailed Implementation

[0080] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.

[0081] Example 1: Refer to Figure 1 As shown, a disk array management method for a high-capacity, high-speed data logger includes:

[0082] Step 101: Obtain the real-time operating status parameters of each physical disk in the disk array, calculate the health score of each physical disk based on the real-time operating status parameters, and mark physical disks with health scores lower than the preset degradation threshold as warning disks;

[0083] Step 102: Obtain the channel number and data rate of the data stream to be written, determine the write priority level based on the channel number, determine the corresponding stripe granularity according to the write priority level, divide the data stream to be written into stripe units according to the stripe granularity and distribute them to each physical disk in a round-robin fashion.

[0084] Step 103: Monitor the write queue depth of each physical disk at a preset sampling period. When the write queue depth of any physical disk exceeds the preset congestion threshold, reroute the stripe units of that physical disk that have not yet been written to the physical disk to the physical disk with the lowest current write load rate.

[0085] Step 104: In response to the health score of the warning disk dropping to the preset failure threshold, migrate the write channel and existing data of the warning disk to the hot spare disk.

[0086] Step 105: Perform periodic check code comparison on the stripe units that have been written to each physical disk, locate the damaged stripe that failed the check, perform XOR reconstruction based on the redundant check data of the other disks in the same stripe group, and write the reconstructed stripe unit to the replacement disk.

[0087] The above scheme achieves dynamic tracking and early prediction of disk degradation trends by real-time monitoring of the operating status parameters of each physical disk in the disk array and calculating health scores; it takes into account the differentiated needs of multi-channel data streams in terms of real-time performance and throughput by determining the write priority level and matching the corresponding stripe granularity based on the channel number; it avoids the overall array write throughput from decreasing due to excessive instantaneous write pressure on individual disks by monitoring the write queue depth and dynamically routing stripe units at a preset sampling period; it ensures that new data is written without interruption and that existing data is safely migrated by initiating a hot standby disk takeover process when the warning disk health score drops to a preset failure threshold; and it promptly detects and accurately recovers silent data corruption by performing periodic check code comparisons on written stripe units and performing stripe-level XOR reconstruction based on redundant check data, effectively ensuring the data security and storage reliability of high-speed data loggers in long-term continuous recording scenarios.

[0088] In some embodiments, a health score for each physical disk is calculated based on real-time operating status parameters, including:

[0089] Read the SMART attribute data of each physical disk and extract the remapped sector count, the number of sectors to be repaired, the cumulative power-on time, and the historical temperature peak.

[0090] The disk degradation index is obtained by assigning preset weight coefficients to the remapped sector count, the unrepaired sector count, the cumulative power-on time, and the historical temperature peak, and then summing them up by weight.

[0091] The real-time spindle vibration amplitude of each physical disk is obtained. If the real-time spindle vibration amplitude exceeds the preset vibration safety threshold, a vibration penalty factor is added to the disk degradation index.

[0092] The disk degradation index, after being superimposed with a vibration penalty factor, is normalized to the range of 0 to 100 to obtain a health score;

[0093] Linear regression fitting is performed on the health scores for multiple consecutive sampling periods to calculate the decline slope of the health scores; if the absolute value of the decline slope is greater than the preset slope threshold, an accelerated deterioration warning signal is generated.

[0094] Specifically, SMART, or Self-Monitoring, Analysis and Reporting Technology, is a built-in health monitoring mechanism for disks. During operation, the disk continuously records various operating parameters and stores them in the SMART attribute data area. The disk array controller periodically reads the SMART attribute data of each physical disk through ATA or SCSI commands, extracting four key parameters: remapped sector count, unrepaired sector count, cumulative power-on time, and historical peak temperature. The remapped sector count refers to the cumulative number of times the disk has migrated data to spare sectors due to physical sector damage; a higher value indicates more severe disk media degradation. The unrepaired sector count refers to the number of sectors that have detected read / write anomalies but have not yet been remapped, reflecting the immediate risk of disk failure. The cumulative power-on time refers to the total number of hours the disk has been powered on since manufacturing, used to assess the fatigue level of the disk's mechanical components. The historical peak temperature refers to the highest disk surface temperature recorded since manufacturing; high temperatures accelerate the degradation of the disk's magnetic media and the aging of electronic components.

[0095] The disk degradation index is obtained by assigning preset weight coefficients to the remapped sector count, the number of sectors to be repaired, the cumulative power-on time, and the historical temperature peak, and then summing them up. Specifically, the preset weight coefficients are pre-calibrated according to the contribution of each parameter to disk failure. For example, the weight coefficient for the remapped sector count is set to 0.35, the weight coefficient for the number of sectors to be repaired is set to 0.30, the weight coefficient for the cumulative power-on time is set to 0.20, and the weight coefficient for the historical temperature peak is set to 0.15. Before the weighted summation, the four parameters need to be normalized by dividing the original value of each parameter by its corresponding maximum allowable value, so that each parameter is normalized to the range of 0 to 1, eliminating the influence of dimensional differences. The formula for the weighted summation is: Disk degradation index D equals w1 multiplied by N. remap Divide by N remapmax Add w2 multiplied by N pending Divide by N pendingmax Add w3 multiplied by T power Divide by T powermax Add w4 multiplied by T peak Divide by T peakmax Where w1, w2, w3, and w4 are the preset weighting coefficients for the four parameters, and N remap For the remapped sector count, N pending T is the number of sectors to be repaired. power T represents the cumulative duration of power-on. peak This represents the historical peak temperature; the subscript 'max' indicates the corresponding maximum allowable value.

[0096] The real-time spindle vibration amplitude of each physical disk is acquired through vibration sensors located within the disk array chassis. These sensors collect the vibration acceleration signal of the disk spindle motor during operation at a preset sampling frequency, and the vibration amplitude is calculated using the root mean square (RMS) method. If the real-time spindle vibration amplitude exceeds a preset vibration safety threshold, it indicates potential wear on the disk spindle bearings or excessive external mechanical impact. In this case, a vibration penalty factor is added to the disk degradation index. The value of the vibration penalty factor is proportional to the degree to which the vibration amplitude exceeds the preset vibration safety threshold. The specific calculation formula is: the vibration penalty factor Pvib equals the difference between α multiplied by Avib minus Ath, where α is the penalty coefficient, Avib is the real-time spindle vibration amplitude, and Ath is the preset vibration safety threshold. The resulting disk degradation index D' equals D plus Pvib.

[0097] The disk degradation index, after being superimposed with a vibration penalty factor, is normalized to the range of 0 to 100 to obtain a health score. The normalization formula is: the health score H equals 100 multiplied by 1 minus the difference between D' and Dmax, where Dmax is the theoretical maximum value of the disk degradation index, that is, the degradation index when all parameters reach the maximum allowable value and the maximum vibration penalty factor is superimposed. A health score of 100 indicates that the disk is in brand new condition, and a health score of 0 indicates that the disk has completely failed. Physical disks with health scores below a preset degradation threshold are marked as warning disks. The preset degradation threshold is set according to the reliability requirements of the application scenario, for example, set to 60 points. That is, disks with health scores below 60 points are marked as warning disks and enter a key monitoring state.

[0098] Linear regression fitting is performed on the health scores for multiple consecutive sampling periods to calculate the decline slope of the health scores. Specifically, the health score data sequences H1, H2 to HM of the most recent M sampling periods are selected, and the least squares method is used to fit the straight line H, which is equal to k multiplied by t plus b, where k is the decline slope, t is the sampling period number, and b is the intercept. If the absolute value of the decline slope k is greater than the preset slope threshold, it indicates that the disk health status is deteriorating rapidly, and an accelerated degradation warning signal is generated. The preset slope threshold is set according to the disk type and application scenario, for example, it is set to decrease by 2 points per sampling period. The accelerated degradation warning signal is used to trigger subsequent proactive data protection processes, including increasing the inspection frequency, prioritizing the migration of existing data, and limiting the write rate.

[0099] In some embodiments, determining the corresponding stripe granularity according to the write priority level, dividing the data stream to be written into stripe units according to the stripe granularity and distributing them to each physical disk in a round-robin fashion includes:

[0100] The number of stripe bytes corresponding to the write priority level is read from the preset granularity configuration table, where the number of stripe bytes corresponding to the high priority data stream is less than the number of stripe bytes corresponding to the low priority data stream;

[0101] The data stream to be written is divided into stripe units of equal length according to the number of bytes per stripe.

[0102] Obtain the current available capacity and cumulative write volume of each physical disk in the disk array, and calculate the capacity balancing coefficient of each physical disk;

[0103] The target write disk number for each stripe unit is determined according to the capacity balancing coefficient, and the stripe units are sequentially distributed to each physical disk in a round-robin fashion.

[0104] Write the channel number, timestamp, and incremental sequence number at the beginning of each stripe unit, and append a cyclic redundancy check code to the end of each stripe unit.

[0105] Specifically, high-speed data loggers typically have multiple data acquisition channels, and the data acquired by each channel has different real-time requirements. For example, the target echo channel in a radar system is extremely sensitive to write latency and is a high-priority data stream; while the data in the system status monitoring channel allows for a certain write latency and is a low-priority data stream. When the disk array controller receives a data stream to be written, it first obtains the channel number and data rate of the data stream, and then queries the pre-stored channel priority mapping table based on the channel number to determine the write priority level corresponding to the data stream. The channel priority mapping table is pre-configured during the system initialization phase according to the service importance of each channel, mapping each channel number to the corresponding priority level, such as high, medium, and low levels.

[0106] The corresponding number of stripe bytes is read from the preset granularity configuration table based on the write priority level. The preset granularity configuration table is a pre-configured mapping table that associates each priority level with the corresponding number of stripe bytes. The number of stripe bytes corresponding to a high-priority data stream is less than the number of stripe bytes corresponding to a low-priority data stream. For example, the number of stripe bytes for a high-priority data stream is set to 64KB, the number of stripe bytes for a medium-priority data stream is set to 256KB, and the number of stripe bytes for a low-priority data stream is set to 1MB. The reason for using a smaller stripe granularity for high-priority data streams is that smaller stripe units can complete a single write operation faster, reducing write latency and meeting high real-time requirements. Using a larger stripe granularity for low-priority data streams can reduce stripe switching overhead and improve the throughput efficiency of sequential writes.

[0107] The data stream to be written is divided into stripe units of equal length according to the number of bytes in the stripe. Specifically, the disk array controller divides the received continuous data stream into stripe units of equal length according to the number of bytes in the stripe, and the length of each stripe unit is equal to the number of bytes in the stripe. If the length of the last data segment is less than the number of bytes in a complete stripe, zero-value bytes are padded to the end of the data segment to the length of the complete stripe, and a valid data length field is marked at the beginning of the stripe.

[0108] Obtain the current available capacity and cumulative write volume of each physical disk in the disk array, and calculate the capacity balancing coefficient of each physical disk. The capacity balancing coefficient reflects the remaining write capacity of each disk, and the calculation formula is: Capacity balancing coefficient Ci equals V. availi Divide by V totali The quotient, multiplied by 1 and minus W cumi Divide by W cummax The difference; where V availi V represents the current available capacity of the i-th disk. totali Let W be the total capacity of the i-th disk. cumi W represents the cumulative write volume of the i-th disk. cummax This represents the maximum cumulative write volume across all disks; a higher capacity leveling factor indicates that the disk has stronger remaining write capacity and should be allocated more stripe units.

[0109] The target disk number for each stripe unit is determined according to the capacity equalization coefficient, and the stripe units are then allocated to each physical disk in a round-robin fashion. Specifically, the capacity equalization coefficient of each disk is normalized to obtain the allocation weight of each disk. The frequency of each disk appearing in the round-robin sequence is determined according to the allocation weight. The disk with a higher capacity equalization coefficient appears more frequently in the round-robin sequence, thus receiving more stripe units. For example, if the capacity equalization coefficient of disk A is 0.8, disk B is 0.6, and disk C is 0.4, then the normalized allocation weights are 0.44, 0.33, and 0.23, respectively. The round-robin sequence can be set to ABACABAC, so that disk A receives more stripe units.

[0110] The channel number, write timestamp, and strip increment sequence number are written at the beginning of each stripe unit, and a cyclic redundancy check (CRC) code is appended to the end of each stripe unit. The stripe header information is used for subsequent data retrieval and integrity verification: the channel number identifies the data acquisition channel to which the stripe unit belongs, facilitating data retrieval by channel; the write timestamp records the write time of the stripe unit, accurate to the millisecond level, for data timing recovery; the strip increment sequence number is the incrementing number of the stripe unit within that channel, used to detect stripe loss or out-of-order data; the cyclic redundancy check (CRC) code is calculated using the CRC-32 algorithm on the data field of the stripe unit, generating a 32-bit check value that is appended to the end of the stripe for subsequent periodic check code comparison to verify data integrity.

[0111] In some embodiments, the write queue depth of each physical disk is monitored at a preset sampling period. When the write queue depth of any physical disk exceeds a preset congestion threshold, the stripe units of that physical disk that have not yet been written to are rerouted to the physical disk with the lowest current write load rate, including:

[0112] The write queue depth and instantaneous write bandwidth of each physical disk are polled at a preset sampling period.

[0113] Calculate the write load rate for each physical disk. The write load rate is the ratio of the current write queue depth to the maximum queue depth.

[0114] Physical disks whose write load exceeds a preset congestion threshold are marked as congested disks;

[0115] Select the physical disk with the lowest write load rate from the remaining physical disks as the migration target disk;

[0116] The stripe units in the write queue of the congested disk that have not yet been written to the disk are reallocated to the migration target disk;

[0117] The allocation weight of congested disks in the striping strategy is temporarily set to zero until the write load rate of the congested disks falls below the preset recovery threshold, after which the allocation weight is gradually restored with a preset step value.

[0118] Specifically, the disk array controller polls the write queue depth and instantaneous write bandwidth of each physical disk at a preset sampling period, for example, once every 100 milliseconds. The write queue depth refers to the number of stripe units waiting to be written to the disk in the write queue, reflecting the current write pressure of the disk. The instantaneous write bandwidth refers to the actual data write rate completed by the disk in the most recent sampling period, in MB / s. The disk array controller obtains the write queue depth by reading the queue status register of each disk and calculates the instantaneous write bandwidth by counting the number of bytes written in the sampling period.

[0119] Calculate the write load rate for each physical disk. The write load rate is the ratio of the current write queue depth to the maximum queue depth. The maximum queue depth is the maximum number of concurrent write requests supported by the disk hardware, determined by the disk firmware parameters. The formula for calculating the write load rate is: Li equals Qi divided by Qmaxi, where Li is the write load rate of the i-th disk, Qi is the current write queue depth, and Qmaxi is the maximum queue depth. The write load rate ranges from 0 to 1, with values ​​closer to 1 indicating greater disk write pressure.

[0120] Physical disks with write load rates exceeding a preset congestion threshold are marked as congested disks. The preset congestion threshold is set according to the overall write performance requirements of the disk array. For example, if it is set to 0.85, then disks with write load rates exceeding 85% are considered congested disks. Among the remaining physical disks, the physical disk with the lowest write load rate is selected as the migration target disk. The remaining physical disks refer to disks that have not been marked as congested disks. If multiple disks have the same write load rate and are all at the lowest value, then the disk with the largest available capacity is selected as the migration target disk.

[0121] The stripe units in the write queue of the congested disk that have not yet been written to the disk are reallocated to the migration target disk. Specifically, the disk array controller takes all stripe units in the write queue of the congested disk that have not yet started writing operations, modifies their target write disk number to the number of the migration target disk, and appends these stripe units to the tail of the write queue of the migration target disk. For stripe units that have already started writing operations, they are waited for to complete without interruption to avoid data inconsistency.

[0122] In the striping strategy, the allocation weight of congested disks is temporarily set to zero. This means that no new stripe units will be allocated to congested disks during subsequent stripe rotation allocation processes until the write load rate of the congested disks falls below the preset recovery threshold. Then, the allocation weight is gradually restored with a preset step value. The preset recovery threshold is lower than the preset congestion threshold, for example, it is set to 0.60, to avoid the congested disks from re-enabling the allocation weight when the load rate just drops to near the congestion threshold, which would cause congestion again. The preset step value is the weight increment for each restoration. For example, if the initial equilibrium weight is 1.0 and the preset step value is 0.2, then it will be restored to the initial equilibrium value in 5 steps.

[0123] In some embodiments, in response to the health score of the warning disk falling to a preset failure threshold, the write channels and existing data of the warning disk are migrated to the hot standby disk, including:

[0124] When the health score drops to a preset failure threshold, a warning disk disconnection command is sent to the disk array controller;

[0125] In response to the warning disk going offline command, the disk array controller switches the write channel of the warning disk to the hot spare disk, and subsequent newly written stripe units are directly written to the hot spare disk.

[0126] Obtain the data distribution table of the warning disk and determine the physical offset address of each stripe unit in the data.

[0127] Read the existing data from the warning disk strip by strip according to the physical offset address, and write the existing data to the corresponding offset address of the hot spare disk;

[0128] During the migration of existing data, sector-by-sector read verification is performed on the warning disk; if any sector read fails, the content of the sector is recovered by XOR operation using the redundant verification data of the other physical disks in the same stripe group and then written to the hot spare disk.

[0129] After all existing data migration is completed, the warning disk will be marked as offline and the hot standby disk will be registered as a formal member disk of the array.

[0130] Specifically, the preset failure threshold is lower than the preset degradation threshold, for example, set to 30 points. When the health score of the warning disk continues to drop from the preset degradation threshold to the preset failure threshold, it indicates that the disk is close to complete failure and the data migration process needs to be started immediately. Taking a preset degradation threshold of 60 points and a preset failure threshold of 30 points as an example, migration is triggered when the health score continues to drop from 60 points to 30 points. When the disk array manager detects that the health score has dropped to the preset failure threshold, it sends a warning disk offline command to the disk array controller. The warning disk offline command includes the physical disk number of the warning disk and the offline reason code.

[0131] In response to the warning disk going offline command, the disk array controller first switches the write channel of the warning disk to the hot spare disk. The hot spare disk is a pre-reserved idle disk in the disk array, which is in a power-on standby state but has not participated in normal data writing. The specific operation of the write channel switching is as follows: modify the channel-disk mapping table of the disk array, and redirect all write channels that originally pointed to the warning disk to the hot spare disk. After the switching is completed, subsequent newly written stripe units are directly written to the hot spare disk to ensure that data writing is uninterrupted. The write channel switching process is completed in milliseconds, is transparent to the upper-layer data acquisition system, and does not affect the continuous recording of the data stream.

[0132] Obtain the existing data distribution table of the warning disk and determine the physical offset address of each stripe unit in the existing data. The existing data distribution table is a metadata table maintained by the disk array controller, which records the channel number, stripe number, physical offset address and data length of all stripe units written on the warning disk. By querying this table, the precise physical location of all existing stripe units on the warning disk can be determined.

[0133] The system reads the existing data stripe by stripe from the warning disk according to the physical offset address and writes the existing data to the corresponding offset address of the hot spare disk. Specifically, the disk array controller sends read commands to the warning disk in ascending order of physical offset address, reading the data of one stripe unit at a time. After a successful read, the stripe unit is written to the corresponding offset address of the hot spare disk. The read and write operations are executed in a pipelined manner, that is, the previous stripe unit is written while the next stripe unit is read, in order to improve migration efficiency.

[0134] During the migration of existing data, a sector-by-sector read verification is performed on the warning disk. Specifically, after reading each stripe unit, a CRC check is performed on each sector within that stripe unit. The sector size is typically 512 bytes or 4096 bytes. The recalculated check value is compared with the original check code stored at the end of the stripe unit. If any sector read fails, including in cases of read timeout, read error code return, or CRC check inconsistency, the sector content is recovered using redundant check data from the other physical disks in the same stripe group through an XOR operation. A stripe group refers to a group of stripe units and their corresponding check stripe units distributed across different physical disks in the same striping write. The principle of XOR recovery is as follows: In RAID-5 or similar redundancy schemes, the check stripe unit stores the byte-by-byte XOR value of all data stripe units in the same group. Therefore, the damaged data stripe can be recovered by XORing the remaining data stripe units in the same group with the check stripe unit byte-by-byte. The recovered sector content is then written to the corresponding offset address on the hot spare disk.

[0135] After all existing data migration is completed, the disk array controller performs integrity checks on all stripe units on the hot spare disk. Once the migrated data is confirmed to be correct, the warning disk is marked as offline, which means that the disk will no longer participate in any read or write operations of the array. At the same time, the hot spare disk is registered as a formal member disk of the array, and the disk member list and stripe mapping table of the disk array are updated, so that the hot spare disk officially takes over the array role of the warning disk.

[0136] In some embodiments, periodic checksum comparisons are performed on stripe units written to each physical disk to locate corrupted stripes that fail the checksum comparison, including:

[0137] Read the stripe units that have been written to each physical disk in sequence according to the preset inspection cycle;

[0138] For each stripe unit, recalculate the cyclic redundancy check code and compare it with the original check code stored at the end of the stripe unit;

[0139] If the recalculated checksum is inconsistent with the original checksum, the stripe unit is marked as a suspected damaged stripe.

[0140] Perform a second reading verification on suspected damaged strips, continuously reading a preset number of times and comparing the results of each reading;

[0141] If the reading results of the preset number of times are inconsistent with the original checksum, then the suspected damaged stripe is confirmed to be a damaged stripe.

[0142] Record the physical disk number, stripe sequence number, and physical offset address of the damaged stripe, and generate a damaged stripe location report.

[0143] Specifically, the disk array controller initiates a full data inspection task according to a preset inspection cycle, for example, once every 24 hours; during the inspection, the stripe units written in each physical disk are read sequentially according to the preset inspection cycle; in order to reduce the impact of inspection on normal write performance, the inspection read operation is performed during the disk write idle period, and the number of stripe units read each time does not exceed the preset batch limit, and a preset interval time is inserted between batches;

[0144] For each stripe unit, the cyclic redundancy check (CRC) code is recalculated and compared with the original check code stored at the end of the stripe unit. Specifically, the data field of the stripe unit is read. The data field does not contain the channel number, write timestamp, stripe increment sequence number, or the original check code at the end. The same CRC-32 algorithm as when writing is used to recalculate the check code in the data field. The recalculated check code is then compared bit by bit with the original check code stored at the end of the stripe unit.

[0145] If the recalculated checksum is inconsistent with the original checksum, the stripe cell is marked as a suspected damaged stripe. The reason for marking it as suspected rather than directly confirming it as damaged is that a single read may cause occasional bit flips in the read data due to momentary disk interference. Momentary interference includes vibration or electromagnetic interference, which is not actual media damage.

[0146] For suspected damaged stripes, a second reading verification is performed. The stripes are read a preset number of times and the results of each reading are compared. For example, the preset number of readings is set to 3 times. Specifically, multiple reading commands are sent to the physical sector where the suspected damaged stripe is located. After each reading, the CRC checksum is recalculated and compared with the original checksum. If the reading results of the preset number of readings are inconsistent with the original checksum, the possibility of occasional reading interference is ruled out, and the suspected damaged stripe is confirmed to be a damaged stripe. If at least one reading result in the preset number of readings is consistent with the original checksum, the stripe unit is determined to be an occasional reading error, and the suspected damaged mark is removed.

[0147] Record the physical disk number, stripe sequence number, and physical offset address of the damaged stripe, and generate a damaged stripe location report. The damaged stripe location report includes the following fields: the physical disk number to which the damaged stripe belongs, the stripe's incremental sequence number in the corresponding channel, the stripe's physical offset address on the disk in bytes, the timestamp of the damage discovery, and the stripe group number to which it belongs. The damaged stripe location report serves as the input for subsequent XOR reconstruction operations.

[0148] In some embodiments, performing XOR reconstruction based on the redundancy check data of the remaining disks in the same stripe group, and writing the reconstructed stripe units to a replacement disk includes:

[0149] Determine the strip group to which the damaged strip belongs based on the damaged strip location report;

[0150] Read the corresponding data stripe units and parity stripe units from the remaining physical disks in the stripe group;

[0151] Perform an XOR operation byte-by-byte on the data stripe unit and the check stripe unit to calculate the original data content of the damaged stripe;

[0152] Regenerate cyclic redundancy check codes from the calculated original data content;

[0153] Select the physical disk with the largest available capacity and the lowest write load rate in the disk array as the replacement disk;

[0154] Write the original data content and the regenerated cyclic redundancy check code to the corresponding offset address on the replacement disk;

[0155] Update the disk mapping table of the stripe group to point the physical address of the damaged stripe to the replacement disk.

[0156] Specifically, the stripe group to which the damaged stripe belongs is determined based on the stripe group number in the damaged stripe location report. A stripe group consists of N-1 data stripe units and 1 parity stripe unit distributed across N physical disks in the same striping write operation. For example, in a RAID-5 array consisting of 4 disks, a stripe group contains 3 data stripe units and 1 parity stripe unit, with the parity stripe unit distributed in a rotating manner among the 4 disks.

[0157] Read the corresponding data stripe units and parity stripe units on the remaining physical disks in the stripe group; specifically, query the stripe mapping table according to the stripe group number to determine the physical disk number and physical offset address of each stripe unit in the stripe group, and send read commands to the corresponding physical disks in sequence to read all data stripe units and parity stripe units in the same group except for the damaged stripe.

[0158] The XOR operation is performed byte-by-byte on the data stripe units and the parity stripe units to calculate the original data content of the damaged stripe. Specifically, all data stripe units and parity stripe units in the same group are aligned byte by byte, and an XOR operation is performed on each byte position. Taking a 4-disk RAID-5 as an example, if data stripe D2 on disk 2 is damaged, then D2 is equal to D1 XOR D3 XOR P, where D1 and D3 are the remaining data stripes in the same group, and P is the parity stripe. The XOR operation is performed bit by bit byte by byte until the complete stripe data is recovered.

[0159] A new cyclic redundancy check code is regenerated on the calculated original data content. Specifically, the CRC-32 algorithm is used to calculate a new check code on the recovered original data content, which is then appended to the end of the stripe when writing to the replacement disk to ensure the integrity and verifiability of the reconstructed data.

[0160] Select the physical disk with the largest available capacity and the lowest write load rate in the disk array as the replacement disk; the selection criteria are as follows: among all the normally working physical disks in the disk array, first filter the disks with an available capacity greater than one stripe unit size, and then select the disk with the lowest write load rate among the disks that meet the capacity criteria as the replacement disk; if there are multiple disks with the same write load rate, select the one with the largest available capacity.

[0161] Write the original data content and the regenerated cyclic redundancy check code to the corresponding offset address on the replacement disk; the corresponding offset address is the physical offset address of the original damaged stripe on the damaged disk, and the same offset address is kept on the replacement disk to simplify the subsequent data retrieval logic; after writing is completed, perform a read verification on the stripe unit just written on the replacement disk to confirm that the written data is correct.

[0162] Update the disk mapping table of the stripe group to point the physical address of the damaged stripe to the replacement disk; specifically, modify the disk number field corresponding to the stripe group in the stripe mapping table maintained by the disk array controller, replacing the original damaged disk number with the replacement disk number, so that subsequent read operations on the stripe group can correctly locate the reconstructed data on the replacement disk.

[0163] In some embodiments, after generating the accelerated degradation warning signal, the method further includes:

[0164] In response to the accelerated degradation warning signal, the historical temperature curve and historical vibration curve of the warning disk are obtained;

[0165] Input the historical temperature curve and historical vibration curve into the pre-trained disk life prediction model, and output the estimated remaining usable time of the warning disk;

[0166] If the remaining available time is expected to be less than the preset safe time threshold, the inspection frequency of the warning disk will be increased to a preset multiple of the default inspection frequency.

[0167] Prioritize migrating existing data that has not yet been redundantly backed up on the warning disk to the physical disk with the highest health score;

[0168] Before the migration of existing data is completed, the write rate of new data to the warning disk is limited to no more than the preset speed limit.

[0169] Specifically, in response to the accelerated degradation warning signal, the disk array manager acquires the historical temperature curve and historical vibration curve of the warning disk; the historical temperature curve is the disk temperature time series data recorded by the warning disk at a preset recording interval since it was powered on, for example, once every 5 minutes; the historical vibration curve is the spindle vibration amplitude time series data recorded by the vibration sensor within the same time span; the two curves reflect the accumulation of thermal and mechanical stress on the disk during its life cycle;

[0170] Historical temperature and vibration curves are input into a pre-trained disk life prediction model, which outputs the estimated remaining usable time of the warning disk. The disk life prediction model is pre-trained based on a large amount of historical operating data and failure records of the same type of disk. The model input is the feature vectors of the temperature and vibration curves, which include statistical features such as mean, peak value, rate of change, and cumulative time exceeding the threshold. The output is the estimated remaining usable time in hours. The model is trained using survival analysis or deep neural network regression methods. After training is completed offline, the model parameters are deployed on the disk array controller.

[0171] If the remaining available time is less than the preset safe time threshold, it indicates that the warning disk may fail completely in a short period of time, and active protection measures should be taken immediately. The preset safe time threshold is set to 72 hours for example. First, the inspection frequency of the warning disk is increased to a preset multiple of the default inspection frequency. For example, if the default inspection cycle is 24 hours and the preset multiple is 4, the inspection cycle will be shortened to 6 hours after the increase, so as to monitor the integrity of the stripe cells on the warning disk at a more intensive frequency.

[0172] Prioritize migrating existing data on the warning disk that has not yet completed redundant backups to the physical disk with the highest health score. Specifically, query the existing data distribution table of the warning disk, filter out the stripe units that do not have redundant copies on other disks, and read and write them sequentially to the physical disk with the highest health score in ascending order of stripe number. The disk with the highest health score is selected as the migration target because it has the longest remaining lifespan and the highest reliability, which can ensure the safety of the migrated data to the greatest extent.

[0173] Before the migration of existing data is completed, the write rate of new data to the warning disk is limited to a preset limit value. The preset limit value is set as a certain percentage of the normal write rate of the warning disk, such as 30% of the normal rate. The purpose of limiting the write rate is to reduce the write load of the warning disk, reduce the mechanical stress of the read / write head seeking and media writing, slow down the deterioration of the disk, and buy more time for the migration of existing data.

[0174] In some embodiments, after writing the reconstructed stripe units to a replacement disk, the method further includes:

[0175] Obtain the total available capacity and currently used capacity of the disk array, and calculate the remaining storage ratio;

[0176] If the remaining storage percentage is lower than the preset insufficient capacity alarm threshold, an insufficient capacity alarm signal will be generated.

[0177] In response to the insufficient capacity alarm signal, scan each storage file in order of data write time from earliest to latest;

[0178] Mark the storage file whose data was written earliest and has been verified as an archiveable file;

[0179] Migrate archiveable files from the disk array to an external archive storage medium;

[0180] After the archiveable files are migrated, release the disk space occupied by the archiveable files and update the available capacity table of the disk array;

[0181] Specifically, after each stripe rebuild operation is completed, the disk array controller obtains the total available capacity of the disk array and the current used capacity, and calculates the remaining storage ratio. The formula for calculating the remaining storage ratio is: R equals the difference between Vtotal and Vused, divided by Vtotal, and then multiplied by 100%, where Vtotal is the total available capacity of the disk array, which is the sum of the capacities of all member disks, and Vused is the current used capacity, which is the sum of the space occupied by all written stripe units.

[0182] If the remaining storage percentage is lower than the preset insufficient capacity alarm threshold, an insufficient capacity alarm signal is generated. The preset insufficient capacity alarm threshold is set to 15% for example. The insufficient capacity alarm signal is reported to the upper monitoring system through the disk array management interface, and the automatic archiving process is triggered at the same time.

[0183] In response to the insufficient capacity alarm signal, each storage file is scanned in order of data write time from earliest to latest. Specifically, the disk array controller reads the file index table, sorts each storage file in ascending order according to its creation time, i.e., data write time, and checks each storage file one by one starting from the storage file with the earliest write time.

[0184] The storage file with the earliest data write time and which has been verified is marked as an archiveable file; verification means that all stripe cells in the storage file have been compared with periodic check codes and no damage has been found, and the data integrity has been verified; only files that have been verified are marked as archiveable files in order to ensure that the data archived to the external storage medium is complete and reliable, and to avoid archiving files that have not been repaired and are damaged.

[0185] The archiveable files are migrated from the disk array to an external archive storage medium. The external archive storage medium can be a tape library, optical disc library, or network attached storage (NAS) or other high-capacity, low-cost storage devices. During the migration, the data of the archiveable files is read tape by tape and written to the external archive storage medium. After the writing is completed, the data on the external archive storage medium is verified to confirm that the archived data is consistent with the original data on the disk array.

[0186] After the archiveable files are migrated, the disk space occupied by the archiveable files is released and the available capacity table of the disk array is updated. Specifically, all stripe cell data of the archiveable files on the disk array is deleted to release the corresponding disk space; the file index table is updated to mark the storage location of the file as the address of the external archive storage medium; the total available capacity and the current used capacity of the disk array are recalculated and the available capacity table is updated; if the remaining storage ratio after archiving is still lower than the preset insufficient capacity alarm threshold, the next storage file that has been verified and confirmed is selected in the order of write time to perform the archiving operation until the remaining storage ratio recovers to above the preset insufficient capacity alarm threshold.

[0187] In some embodiments, after temporarily setting the allocation weight of congested disks in the striping strategy to zero, the method further includes:

[0188] Continuously monitor the write load rate and disk temperature of congested disks;

[0189] If the write load rate of the congested disk is lower than the preset recovery threshold for several consecutive sampling periods and the disk temperature drops below the preset temperature safety threshold, then the write freeze state of the congested disk will be lifted.

[0190] The allocation weight of congested disks is gradually restored using a preset step value, and a preset observation period is waited after each level of weight is restored.

[0191] Monitor the write latency and read / write error rate of congested disks within a preset observation period;

[0192] If the write latency or read / write error rate exceeds the preset abnormal threshold, the allocation rights of the congested disk will be reset to zero and a disk abnormality report will be generated.

[0193] If the write latency and read / write error rate are both within the normal range during the preset observation period, then continue to restore the next level of weights until the weight allocation is restored to the initial equilibrium value.

[0194] Specifically, after temporarily setting the allocation weight of congested disks to zero in the striping strategy, the disk array controller continuously monitors the write load rate and disk temperature of the congested disks; the disk temperature is read by the temperature sensor built into the disk, reflecting the heat dissipation recovery status of the disk after it stops receiving new write tasks.

[0195] If the write load rate of the congested disk is lower than the preset recovery threshold for several consecutive sampling periods and the disk temperature drops below the preset temperature safety threshold, the congested disk is considered to have fully recovered, and the write freeze state of the congested disk is lifted. Several consecutive sampling periods are, for example, set to 5 consecutive sampling periods, the preset recovery threshold is, for example, set to 0.60, and the preset temperature safety threshold is, for example, set to 55℃. The purpose of satisfying both the write load rate and disk temperature conditions is to ensure that the disk has not only cleared the backlog at the queue level, but also has been sufficiently cooled at the physical level, so as to avoid the accumulation of heat and accelerated media deterioration caused by resuming writing when the disk temperature is still high.

[0196] The allocation weight of the congested disk is gradually restored using preset step values, with a preset observation period after each weight level is restored. For example, if the initial equalization weight is 1.0, the preset step value is 0.2, and the preset observation period is 30 seconds, the restoration process is as follows: first, the allocation weight is restored from 0 to 0.2, and then observed for 30 seconds; if normal, it is restored to 0.4, and then observed for another 30 seconds; and so on, until it is restored to 1.0. The purpose of gradually restoring the weight rather than restoring it to the initial value all at once is to avoid the congested disk being subjected to full write pressure immediately after being unfrozen, which could lead to renewed congestion.

[0197] The write latency and read / write error rate of congested disks are monitored within a preset observation period. The write latency refers to the time interval from when a stripe unit enters the write queue to when it completes writing to the disk, and is the average of the write latency of all stripe units within the observation period. The read / write error rate is the ratio of the number of read / write error codes returned by the disk to the total number of read / write operations within the observation period.

[0198] If the write latency or read / write error rate exceeds the preset anomaly threshold, it indicates that the congested disk is experiencing performance abnormalities even under a small amount of write pressure. The allocation rights of the congested disk are then reset to zero, and a disk anomaly report is generated. The preset anomaly threshold is, for example, set to a write latency exceeding 50 milliseconds or a read / write error rate exceeding 0.1%. The disk anomaly report includes information such as the disk number, the time of the anomaly, the write latency value, the read / write error rate, and the current disk temperature, and is reported to the upper-level monitoring system for maintenance personnel to handle.

[0199] If the write latency and read / write error rate are both within the normal range during the preset observation period, the next level of weight will continue to be restored until the allocated weight is restored to the initial equilibrium value. Once the allocated weight is fully restored to the initial equilibrium value, the disk will participate in stripe round-robin allocation with normal weight, and the congestion handling process will end.

[0200] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

[0201] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A disk array management method for a high-capacity, high-speed data logger, characterized in that, include: Obtain the real-time operating status parameters of each physical disk in the disk array, calculate the health score of each physical disk based on the real-time operating status parameters, and mark physical disks with health scores lower than a preset degradation threshold as warning disks; Obtain the channel number and data rate of the data stream to be written, determine the write priority level based on the channel number, determine the corresponding stripe granularity according to the write priority level, divide the data stream to be written into stripe units according to the stripe granularity and distribute them to each physical disk in a round-robin fashion; The write queue depth of each physical disk is monitored at a preset sampling period. When the write queue depth of any physical disk exceeds the preset congestion threshold, the stripe units of that physical disk that have not yet been written to the disk are rerouted to the physical disk with the lowest current write load rate. In response to the health score of the warning disk dropping to a preset failure threshold, the write channel and existing data of the warning disk are migrated to the hot standby disk. Perform periodic check code comparisons on the stripe units already written to each physical disk, locate the damaged stripes that failed the check, perform XOR reconstruction based on the redundant check data of the other disks in the same stripe group, and write the reconstructed stripe units to the replacement disk.

2. The disk array management method for a high-capacity, high-speed data logger according to claim 1, characterized in that, The calculation of the health score of each physical disk based on the real-time operating status parameters includes: Read the SMART attribute data of each physical disk and extract the remapped sector count, the number of sectors to be repaired, the cumulative power-on time, and the historical temperature peak. The remapped sector count, the sector count to be repaired, the cumulative power-on time, and the historical temperature peak are each assigned a preset weight coefficient and then summed to obtain the disk degradation index. The real-time spindle vibration amplitude of each physical disk is obtained. If the real-time spindle vibration amplitude exceeds a preset vibration safety threshold, a vibration penalty factor is added to the disk degradation index. The disk degradation index, after being superimposed with a vibration penalty factor, is normalized to the range of 0 to 100 to obtain the health score; Linear regression fitting is performed on the health score for multiple consecutive sampling periods to calculate the downward slope of the health score; if the absolute value of the downward slope is greater than a preset slope threshold, an accelerated degradation warning signal is generated.

3. The disk array management method for a high-capacity, high-speed data logger according to claim 1, characterized in that, The step of determining the corresponding stripe granularity according to the write priority level, dividing the data stream to be written into stripe units according to the stripe granularity, and allocating them to each physical disk in a round-robin fashion includes: The corresponding number of stripe bytes is read from the preset granularity configuration table according to the write priority level, wherein the number of stripe bytes corresponding to the high priority data stream is less than the number of stripe bytes corresponding to the low priority data stream; The data stream to be written is cut into stripe units of equal length according to the number of bytes in the stripe; Obtain the current available capacity and cumulative write volume of each physical disk in the disk array, and calculate the capacity balancing coefficient of each physical disk; The target write disk number of each stripe unit is determined according to the capacity balancing coefficient, and the stripe unit is sequentially and round-robin distributed to each physical disk. Write the channel number, timestamp, and incremental sequence number at the beginning of each stripe unit, and append a cyclic redundancy check code to the end of each stripe unit.

4. The disk array management method for a high-capacity, high-speed data logger according to claim 1, characterized in that, The step of monitoring the write queue depth of each physical disk at a preset sampling period, and when the write queue depth of any physical disk exceeds a preset congestion threshold, rerouting the unwritten stripe units of that physical disk to the physical disk with the lowest current write load rate, includes: The write queue depth and instantaneous write bandwidth of each physical disk are polled at the preset sampling period; Calculate the write load rate of each physical disk, where the write load rate is the ratio of the current write queue depth to the maximum queue depth; The physical disk whose write load rate exceeds the preset congestion threshold is marked as a congested disk; Select the physical disk with the lowest write load rate from the remaining physical disks as the migration target disk; The stripe units in the write queue of the congested disk that have not yet been written to the disk are reallocated to the migration target disk; The allocation weight of the congested disk in the striping strategy is temporarily set to zero until the write load rate of the congested disk falls below the preset recovery threshold, after which the allocation weight is gradually restored with a preset step value.

5. The disk array management method for a high-capacity, high-speed data logger according to claim 1, characterized in that, The step of migrating the write channels and existing data of the warning disk to a hot standby disk in response to the health score of the warning disk dropping to a preset failure threshold includes: When the health score drops to the preset failure threshold, a warning disk offline command is sent to the disk array controller; In response to the warning disk going offline command, the disk array controller switches the write channel of the warning disk to the hot spare disk, and the subsequent newly written stripe units are directly written to the hot spare disk. Obtain the data distribution table of the warning disk and determine the physical offset address of each stripe unit in the data. The stored data is read strip by strip from the warning disk according to the physical offset address, and the stored data is written to the corresponding offset address of the hot standby disk; During the migration of existing data, sector-by-sector read verification is performed on the warning disk; if any sector read fails, the content of the sector is recovered by XOR operation using the redundant verification data of the other physical disks in the same stripe group and then written to the hot spare disk. After all the existing data has been migrated, the warning disk is marked as offline and the hot standby disk is registered as a formal member disk of the array.

6. The disk array management method for a high-capacity, high-speed data logger according to claim 1, characterized in that, The process of performing periodic checksum comparisons on stripe units already written to each physical disk to locate corrupted stripes that have failed checksum verification includes: The stripe units already written to each physical disk are read sequentially according to the preset inspection cycle; For each stripe unit, the cyclic redundancy check code is recalculated and compared with the original check code stored at the end of the stripe unit; If the recalculated checksum is inconsistent with the original checksum, the stripe unit is marked as a suspected damaged stripe. Perform a second reading verification on the suspected damaged strip, continuously read a preset number of times and compare the results of each reading; If the reading results of the preset number of times are inconsistent with the original checksum, then the suspected damaged stripe is confirmed to be the damaged stripe. Record the physical disk number, stripe sequence number, and physical offset address of the damaged stripe, and generate a damaged stripe location report.

7. The disk array management method for a high-capacity, high-speed data logger according to claim 1, characterized in that, The step of performing XOR reconstruction based on the redundancy check data of the other disks in the same stripe group, and writing the reconstructed stripe units to the replacement disk, includes: The strip group to which the damaged strip belongs is determined based on the damaged strip location report; Read the corresponding data stripe units and check stripe units from the remaining physical disks in the stripe group; Perform an XOR operation byte-by-byte on the data stripe unit and the check stripe unit to calculate the original data content of the damaged stripe; Regenerate a cyclic redundancy check code from the calculated original data content; The physical disk with the largest available capacity and the lowest write load rate in the disk array is selected as the replacement disk; Write the original data content and the regenerated cyclic redundancy check code to the corresponding offset address of the replacement disk; Update the disk mapping table of the stripe group to point the physical address of the damaged stripe to the replacement disk.

8. The disk array management method for a high-capacity, high-speed data logger according to claim 2, characterized in that, After generating the accelerated degradation warning signal, the method further includes: In response to the accelerated degradation warning signal, the historical temperature curve and historical vibration curve of the warning disk are acquired; The historical temperature curve and the historical vibration curve are input into a pre-trained disk life prediction model, which outputs the estimated remaining usable time of the warning disk. If the estimated remaining available time is less than the preset safe time threshold, the inspection frequency of the warning disk will be increased to a preset multiple of the default inspection frequency. Prioritize migrating the existing data on the warning disk that has not yet been redundantly backed up to the physical disk with the highest health score; Before the existing data migration is completed, the write rate of new data to the warning disk is limited to a preset limit value.

9. A disk array management method for a high-capacity, high-speed data logger according to claim 7, characterized in that, After writing the reconstructed stripe units to the replacement disk, the method further includes: Obtain the total available capacity and the currently used capacity of the disk array, and calculate the remaining storage ratio; If the remaining storage ratio is lower than the preset insufficient capacity alarm threshold, an insufficient capacity alarm signal is generated; In response to the insufficient capacity alarm signal, each storage file is scanned in order of data writing time from earliest to latest; The storage file whose data was written earliest and has been verified is marked as an archiveable file; The archiveable files are migrated from the disk array to an external archiving storage medium; After the archiveable files are migrated, the disk space occupied by the archiveable files is released and the available capacity table of the disk array is updated.

10. A disk array management method for a high-capacity, high-speed data logger according to claim 4, characterized in that, After temporarily setting the allocation weight of the congested disk in the striping strategy to zero, the method further includes: Continuously monitor the write load rate and disk temperature of the congested disk; If the write load rate of the congested disk is lower than the preset recovery threshold for multiple consecutive sampling periods and the disk temperature drops below the preset temperature safety threshold, then the write freeze state of the congested disk is lifted. The allocation weight of the congested disk is gradually restored using the preset step value, and a preset observation period is waited after each level of weight is restored. Monitor the write latency and read / write error rate of the congested disk within the preset observation period; If the write latency or the read / write error rate exceeds a preset abnormal threshold, the allocation right of the congested disk is reset to zero and a disk abnormality report is generated. If the write latency and read / write error rate are both within the normal range during the preset observation period, then the next level of weights will continue to be restored until the allocated weights are restored to the initial equilibrium value.

Images